Anthropic said it had blocked several accounts that were using its artificial intelligence models in ways that could have supported the development of biological weapons capable of killing people.
In a report published on Thursday, the AI company behind the Claude chatbot said it viewed the misuse of biological resources as one of the "most serious risks" facing its models.
The report outlines five real case studies in which users circumvented controls that block access from certain regions and took other steps to obscure the purpose of their research and evade Anthropic's safeguards. The examples include possible gain-of-function research and cover both infectious diseases, such as avian flu, and new poisons and toxins.
Reviewing 30 days of activity, Anthropic said it identified about 35 "distinct research efforts" involving potentially concerning activity. The company acknowledged it could not be certain whether the users intended to cause harm or were carrying out legitimate scientific research. It said the individuals in the case studies were "active scientists," but did not name the research institutions or countries involved.
Gain-of-function research attempts
One case involved a request for help drafting a funding proposal for gain-of-function research into the transmissibility and immune evasion of the chikungunya virus.
In another, a researcher outside the United States was using Claude for work on avian flu, focused on how the virus adapts to mammals and causes severe disease. The report also details research into orthopoxviruses, a family of viruses that includes smallpox and mpox, as well as venom toxins.
The double-edged promise of AI drug discovery
Drug development is considered one of the most promising applications of AI, and industry leaders often point to its potential to help cure cancer and other diseases. But the same qualities that make AI powerful for discovering new materials and medicines can also make it useful for developing biological weapons in the wrong hands.
Anthropic said it wants its models to remain useful for scientific research, but newer versions have become capable of far more complex work than earlier ones, and the potential for misuse grows alongside the potential benefits. It added that malicious actors could try to use the beneficial biological applications of AI to maintain a kind of "plausible deniability" around potentially harmful research.
As its models become increasingly capable, approaching or exceeding expert performance on complex scientific tasks, Anthropic said it expects their impact to grow in both beneficial and potentially harmful contexts. It said that as AI models are used more widely, providers will keep gaining visibility into real-world threats that even governments and international organizations lack.
The company said its newer models, including Claude Fable 5, have more robust safeguards restricting access to a wide range of dual-use biological research queries. Anthropic said there is evidence the safety measures are working, but that even stronger systems will be needed as AI models and potential threats grow more sophisticated.
Surveillance, scams and missile guidance also blocked
Anthropic said it had also stopped attempts to use its models for surveillance operations, scams and the development of conventional weapons such as drones and missiles, allegedly involving malicious actors in China, Russia and Yemen.
According to the report, operators suspected of ties to the Chinese government used Claude to surveil and target Uyghur minorities in Xinjiang, as well as religious communities including Catholic cardinals across Asia, the Presbyterian Church in Taiwan and Tibetan Buddhists.
One user, allegedly linked to the Chinese military, was using what they believed was the Kimi model from the Chinese AI lab Moonshot to assist surveillance work. But the queries, which included sensitive Moonshot customer information, were instead redirected to Claude, according to the report. Anthropic said there were other similar attempts by Chinese companies including DeepSeek to secretly divert user requests to Claude. CNN contacted Chinese authorities and DeepSeek for comment; Moonshot declined to comment.
Anthropic added that groups based in northern Yemen used Claude to develop guidance software for missile systems. The report did not directly identify who was responsible, but appeared to point to the Iran-backed Houthi rebels, who control much of northern Yemen.
Former employees warn of runaway AI risk
The disclosure comes amid a growing chorus of warnings from AI company employees who have recently left their jobs over safety concerns. Some have said they fear the technology is advancing too fast and could one day outpace humans' ability to control it, and have called on governments to consider ways to slow development if the situation gets out of hand.
Jacob Coxon, a former Anthropic employee, told Anderson Cooper of CNN on Wednesday that if the current capability level of AI systems were extrapolated into the future, they could cause extreme havoc.
The 27-year-old AI researcher, in a series of posts about his resignation on X this week, said the people developing AI genuinely believe it could kill everyone by the end of the decade. In a statement on Wednesday, Coxon said the technology could be used to hack critical infrastructure or build "extinction-level biological weapons," adding that there are many ways AI could manifest in the world.
Calls for tighter regulation
In July, nearly 1,400 employees of AI companies signed an open letter urging the US government to regulate the technology, rein in big tech companies and slow the pace of AI development to ensure its safety. This week, OpenAI's chief scientist, Jakub Pachocki, warned that AI capabilities are advancing faster than researchers' ability to reliably monitor and control them. OpenAI, the maker of ChatGPT, is one of Anthropic's main rivals in the AI industry.
For now, AI companies are largely regulating themselves. The Trump administration has worked to undermine state-level AI regulations, and Congress has so far been reluctant to rein in the technology.
In a statement on Wednesday, Anthropic said it believed the world would benefit if the industry adopted a legal and verifiable way of working together to set the pace of releasing robust models.
