Developing Nuclear Safeguards for AI
… 1 Along with the concrete importance of securing frontier AI models against nuclear misuse, this first-of-its-kind effort shows the power of public-private partnerships. …
Search
People also ask
What does AI have to do with dangerous weapons at all?
We worry about how AI might assist malicious actors with weapon acquisition and development both because of how it is similar to historical information and communication technologies and how it is different. In recent years, terrorist groups have rapidly adopted technologies like encrypted communications, cryptocurrency, and social media. We should expect nothing different from AI. Just as those seeking information about how to build weapons shifted from needing to acquire physical pamphlets or manuals to searching the internet, we can expect that they will query AI. What is different, though,
LLMs and bioriskRedeploying Claude Fable 5
… 3 We expect that some jailbreaks will be found for our models, and that they will vary in severity: there will be many minor jailbreaks, some narrow harmful ones, and although no universal jailbreaks for Fable 5 have been discovered at the time of writing, expert safety researchers continue to red-… …
Building safeguards for Claude
… We operate across multiple layers: developing policies, influencing model training, testing for harmful outputs, enforcing policies in real-time, and identifying novel misuses and attacks. …
Mapping the mind of a large language model
… It’s therefore particularly interesting that, in addition to the aforementioned scam emails feature, we found features corresponding to: Capabilities with misuse potential code backdoors, developing biological weapons Different forms of bias gender discrimination, racist claims about crime Potentia… …
LLMs and biorisk
… At the same time, AI is fundamentally a dual-use technology. A key tenet of our effort to develop AI responsibly is to identify, measure, and mitigate the prospects for malicious actors to misuse the same capabilities that make AI so promising for scientists and innovators. …
Mapping AI-enabled cyber threats: Insights from the LLM ATT&CK Navigator
… It is calculated based on the actor's activity across Claude.ai, Claude Code, and our API, drawing on our safety classifiers alongside open-source and internal threat-intelligence indicators. The higher the score, the higher-risk the AI enabled actor is. …
Claude Fable 5 and Claude Mythos 5
… Fable 5 comes with a new set of classifiers : separate AI systems that detect potential misuse, including jailbreak attempts, and prevent the main model in this case Fable 5 from responding. …
Claude Opus 4.6
… A step forward on safety These intelligence gains do not come at the cost of safety. On our automated behavioral audit, Opus 4.6 showed a low rate of misaligned behaviors such as deception, sycophancy, encouragement of user delusions, and cooperation with misuse. …
Introducing Claude Sonnet 5
… 01 / 10 Safety evaluations Our pre-deployment safety evaluations found that Sonnet 5 was overall an improvement on Sonnet 4.6. On agentic safety, the model is better at refusing malicious requests and resisting hijack attempts in prompt injection attacks. …
An update on our election safeguards
… To measure how well Claude handles election-related risks, we run a series of tests examining its responses to questions about candidates, voting, and election administration, and how it holds up against attempts at misuse. …