Something happened at BSidesSF 2026 that nobody saw coming. The top ten teams in the Capture The Flag competition didn't just use AI to help them solve challenges. They fully automated the entire process. An autonomous agent, running multiple AI models in parallel, solved all 52 challenges and won first place. Most challenges fell within minutes of being released.
A year earlier, at the same event, roughly half the players had ChatGPT open as a helper. It could handle easy challenges and free up brainpower for harder ones. That felt like a meaningful shift at the time. But the jump from 2025 to 2026 wasn't incremental. It was a complete transformation of how CTF competitions work.
What Actually Happened
The winning team at BSidesSF 2026 open-sourced their tool after the competition. Their system works by polling a CTF platform for new challenges, then spinning up parallel AI agents in isolated Docker containers. Each challenge gets attacked simultaneously by multiple models. A coordinator model shares insights between agents, and if one gets stuck, it feeds discoveries from the others back in. The result is a system that solves cryptography, binary exploitation, web security, and reverse engineering challenges faster than any human team could.
One competitor wrote afterward that he placed fifth the year before playing solo. In 2026, he estimated he would have finished seventy-fifth without AI assistance. The skill gap didn't change. The tools did.
Why This Matters Beyond Competitions
CTF competitions have been the backbone of cybersecurity skill development for decades. Universities use them to train students. Companies use them to assess candidates. Security teams use them to stay sharp. The underlying assumption has always been that if someone can solve these challenges, they have the skills to handle real threats.
That assumption is breaking down. If an AI agent can solve a standard jeopardy-style CTF challenge in minutes, then the challenge is no longer measuring a uniquely human skill. It's measuring something a machine does better and faster. This doesn't mean cybersecurity skills are obsolete. It means the way we measure and develop them needs to change.
What AI Still Can't Do
The research coming out of BSidesSF and academic institutions tells a consistent story. AI excels at bounded, well-defined problems with clear success criteria. That describes most jeopardy-style CTF challenges perfectly. Find the flag, submit it, move on.
But professional security work rarely looks like that. Penetration testers need to manage scope, avoid false positives, understand business context, and communicate findings to non-technical stakeholders. Incident responders need to coordinate across teams under pressure, triage competing priorities, and make judgment calls with incomplete information. SOC analysts need to distinguish real threats from noise across thousands of alerts. None of these skills has a hidden flag at the end.
Researchers at NYU found something interesting in their study of AI-assisted CTF competitions. The bottleneck wasn't the AI's reasoning capability. It was the human's ability to provide context and direction. When humans tried to guide the AI, ineffective prompting actually slowed things down. Autonomous agents that directed themselves performed better. That's a revealing finding, because it means the human skill that matters most in an AI-augmented world isn't technical execution. It's strategic thinking, context-setting, and knowing what questions to ask.
Where Cybersecurity Training Needs to Go
The implications for training are clear. Programs built entirely around solving static, flag-based challenges are teaching skills that AI already does better. That doesn't make those skills worthless, but it does mean they're becoming table stakes rather than differentiators.
Training needs to shift toward the things AI struggles with. Live attack-and-defense exercises where the environment changes in real time. Multi-day cyber drills that require coordination between teams and communication with leadership. Incident response simulations where there's no single right answer, just better and worse decisions under uncertainty. Scenarios that test judgment, not just technical knowledge.
This shift is already happening. Organizations that run cyber drills and simulation-based training are finding that these exercises reveal capabilities and gaps that traditional CTFs never exposed. Can your team communicate clearly during a crisis? Can they prioritize when everything seems urgent? Can they explain technical risk to a board member? These are the skills that matter when AI handles the routine technical work.
What This Means for You
If you're running CTF competitions, this doesn't mean you should stop. CTFs remain excellent for learning fundamentals, building community, and sparking interest in cybersecurity. But if you're using them as your primary method for assessing skills or measuring readiness, it's time to add simulation-based exercises to the mix.
If you're hiring cybersecurity talent, a candidate's CTF ranking tells you less than it used to. What matters more is how they think through ambiguous problems, how they communicate under pressure, and how they work with others. Hands-on simulations and scenario-based assessments reveal these qualities in ways that flag-based challenges cannot.
The cybersecurity professionals who thrive in the next few years won't be the ones who can solve the most puzzles. They'll be the ones who can think strategically, coordinate effectively, and make good decisions with imperfect information. AI is already solving the puzzles. The question is whether your training is preparing people for everything else.
Simulations Labs is a cybersecurity simulations platform for hosting CTFs, cyber ranges, and cyber drills.
FAQ
What happened at BSidesSF 2026?
At BSidesSF 2026, a team used autonomous AI agents to fully automate solving CTF challenges. Their system solved all 52 challenges faster than human competitors and won first place.
Does this mean traditional CTF competitions are dead?
No. CTFs still play an important role in learning cybersecurity fundamentals, building practical skills, and growing communities. However, they are becoming less effective as the only way to assess real-world cybersecurity readiness.
Why are AI models so effective at solving CTF challenges?
Most jeopardy-style CTF challenges are structured, well-defined problems with clear objectives and success criteria. AI performs exceptionally well in environments where tasks are bounded and measurable.
What cybersecurity skills can AI still not replace?
AI still struggles with human-centered and strategic tasks such as:
- Decision-making under pressure
- Communication during incidents
- Prioritization and risk assessment
- Team coordination
- Understanding business context
- Explaining technical issues to non-technical stakeholders
These skills remain essential in real-world cybersecurity operations.
How should cybersecurity training evolve?
Cybersecurity training should increasingly focus on:
- Live attack-and-defense simulations
- Incident response exercises
- Multi-team cyber drills
- Real-time decision-making scenarios
- Communication and leadership during crises
These environments better reflect the realities of modern cybersecurity work.
Are CTF rankings still useful for hiring?
CTF rankings can still demonstrate technical curiosity and foundational skills, but they should no longer be the primary measure of cybersecurity capability. Employers should also evaluate problem-solving, collaboration, communication, and strategic thinking through simulations and scenario-based assessments.
What is the biggest takeaway for cybersecurity professionals?
The future belongs to professionals who can combine technical understanding with strategic thinking, teamwork, and decision-making. As AI automates routine technical tasks, human judgment becomes even more valuable.
How can organizations prepare for this shift?
Organizations should complement traditional CTF programs with simulation-based training platforms that test operational readiness, collaboration, and response capabilities in realistic environments.
