Search

In both the CTF and cyber defense challenges, Claude demonstrated both promise and clear limitations. In the CTF competitions, Claude usually struggled on the same tasks as other competitors; the one task it (and every other AI team) ultimately failed on in HackTheBox was also the challenge for which the human teams had the lowest solve rate (only about 14% of the participating human teams solved it). In PlaidCTF, Claude did not solve any challenges–but this was also true of about 70% of the teams who entered. Although Claude performed as well or better than human teams in some aspects of the

AI is poised to transform the domain of cybersecurity. Anthropic’s Safeguards team recently identified and banned a user with limited coding abilities leveraging Claude to develop malware. Research suggests that this lowering of the bar for expertise needed to pose a threat, combined with the falling costs of large language models (LLMs), presages a dramatic shift in the economics of cyberattacks.[1] To understand the present state of AI cyber capabilities and gain insight into their trajectory, we pursue different approaches to model evaluation, including publicly available and custom-made be

Claude Sonnet 4.5 represents a meaningful improvement, but we know that many of its capabilities are nascent and do not yet match those of security professionals and established processes. We will keep working to improve the defense-relevant capabilities of our models and enhance the threat intelligence and mitigations that safeguard our platforms. In fact, we have already been using results of our investigations and evaluations to continually refine our ability to catch misuse of our models for harmful cyber behavior. This includes using techniques like organization-level summarization to und