Cyber evaluations of Claude 4
Frontier Red Team Detailed cyber evaluations of Claude 4 Jul 15, 2025 Anthropic (with Pattern Labs ) We believe we are at a crucial period for cybersecurity and AI, with models advancing toward…
Tracked topic
Claude is an AI assistant developed by Anthropic that uses large language models to generate text and support conversational tasks.
In our previous Economic Index reports, we separated out “automation,” where AI directly performs tasks, from “augmentation,” where AI collaborates with a user to perform a task. Here, we found that Claude Code showed dramatically higher automation rates—79% of conversations involved some form of automation, compared to 49% on Claude.ai. We also split automation and augmentation into several subtypes (as discussed in our previous work). “Feedback Loop” patterns, where Claude completes tasks autonomously but with help of human validation (for example, where the user sends any errors back to Cla
Anthropic Economic Index: AI's impact on software developmentOverall, we found that developers commonly use Claude for building user interfaces and interactive elements for websites and mobile applications. Although no single language dominated, the primarily web-focused development languages of JavaScript and TypeScript together accounted for 31% of all queries, and HTML2 and CSS (other languages for user-facing code) together added another 28%. Back-end development languages (used for behind-the-scenes logic, databases, and infrastructure, as well as API and AI development) were also represented: notably, Python was at 14% of queries. However, Python
Anthropic Economic Index: AI's impact on software developmentWe also analyzed which groups of developers might be using Claude. We used our analysis system to identify the type of project (e.g. a personal project vs. a project done for a startup) that best described users’ coding-related interactions. Because we don’t know the real-world context in which Claude’s responses were being used, these analyses rely on uncertain inferences from incomplete data. We therefore treat these findings as more preliminary than the ones described above. Startups appear to be the primary early adopters of Claude Code, and enterprise adoption lags behind. Startup work ac
Anthropic Economic Index: AI's impact on software developmentIn our last report, we analyzed how AI usage varied between augmentative uses, like learning or iterating on an output, and automative uses, like asking the model to directly complete a task or debug errors. Our analysis shows the balance of augmentation and automation is essentially unchanged in our new data, with augmentation still comprising 57% of usage. However, we did see some change in types of automation and augmentation uses—for example, we saw learning interactions, where the user asks Claude for information or explanation about different topics, rise from ~23% to ~28%. We received a
Anthropic Economic Index: Insights from Claude 3.7 SonnetFrontier Red Team Detailed cyber evaluations of Claude 4 Jul 15, 2025 Anthropic (with Pattern Labs ) We believe we are at a crucial period for cybersecurity and AI, with models advancing toward…
…However, the fact that Claude could succeed at automatically developing a crude browser exploit, even if only in a few cases, is concerning. “Crude” is an important caveat here. The exploits Claude…
…For CodeRabbit’s code review workloads, Claude Opus 4.7 is the sharpest model we’ve tested. Recall improved by over 10%, surfacing some of the most difficult-to-detect bugs in…
…The Security team often uses Claude Code for code understanding (48.9%), specifically analyzing and understanding the security implications of different parts of the codebase. Non-technical employees often use Claude Code…
…Claude Code offers the opposite tradeoff. Because Claude Code is our own product, we can link requests across sessions and understand entire agent workflows from start to finish. This makes Claude Code…
…Claude, Claude Cowork, and Claude Code. It's starting with regulated industries—banking, insurance, healthcare—where accuracy and auditability matter most. Engagements range from targeted help with specific finance tasks to top…
…At Bending Spoons, one of Italy’s largest technology companies, the majority of code changes are now co-authored with Claude Code. Anthropic is also putting Claude in the hands of Italy…
…Much of the code it wrote was effective on the first try (which was not the case for Team Claude or Team Claude-less in the original experiment). Indeed, we can see…
…Extending tools with code : Claude Code can write scripts, plugins, and generative systems for the software you already use. You can ask it to build a custom shader, script a procedural animation…
…the framework to automatically construct inputs that might fail this property. Our Property-Based Testing Agent Our property-based testing agent is built as a custom Claude Code command. The agent takes…