Anthropic warns the risk of AI-created weapons could slow frontier models down
Anthropic is preparing to go public, and new documents show how the lab hopes to control its technology while also pushing the limits of what AI can do
By Adam Kovac edited by Claire Cameron
Samuel Boivin/Shutterstock
As artificial intelligence pioneer Anthropic prepares to go public, the company has warned users—including scientists—on Monday that advanced models may be delayed because of cybersecurity and safety threats that risk eclipsing the models’ benefits.
In documents filed with the Securities and Exchange Commission ahead of Anthropic’s expected initial public offering, the AI lab vowed to weigh the effects of its models on the public good before issuing any new products.
“Moving quickly brings AI’s benefits sooner, but moving too quickly could outrun the world’s ability to make AI systems safe,” the company wrote.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
In the filing, Anthropic acknowledged that its future relies on outpacing its competition in developing increasingly capable and complex large language models. But it also raised the risk of its AI being used for harmful purposes, such as developing biological or chemical weapons.
“We deploy technical safeguards, including automated classifiers, specifically intended to detect and block these categories of misuse,” the company warned. “But such safeguards are incomplete, may be outdated in light of evolving threat models, have been and may in the future again be circumvented.” The comments reflect a debate among AI experts over what adequate safety testing looks like in light of repeated AI breaches and hacks.
Still, Anthropic highlighted the potential for new discoveries in biology and medical sciences as a key pathway to profitability—the company currently runs at a loss in the billions of dollars. Anthropic cited its life sciences laboratory, which recently discovered a segment of DNA that bears some resemblance to CRISPR, as an example of how it could access new markets. Even that expansion presents dangers, the company acknowledged, because more traditional science and research is outside its current area of expertise.
“These activities expose us to categories of liability, including for workplace injury and environmental contamination, that differ from those associated with our core business,” the company acknowledged.
Anthropic also warned investors that the company will like make investments into compute power that could appear “irresponsibly large at first glance.” Those vast sums of money are part of a bet that AI will have “wild exponential” growth.
“In order to fully capture the opportunity in front of us, we need to invest more aggressively than would a business that operates in a more slowly changing technological regime,” the company wrote.
Subscribe to Support Independent Journalism
Great science journalism requires human expertise, time, effort and creativity. And it costs money. That’s why I and the journalists here at Scientific American hope you’ll join our community.
When you subscribe, you are supporting staff and freelance journalists who are passionate about telling science stories that are true, important and compelling. Our editors and reporters are often experts in their fields, which means they understand the nuances of big discoveries and can untangle the breakthroughs from the hype. With a subscription, you are also supporting rigorous fact-checking to ensure the words we publish are precise and accurate. And you’re supporting original illustrations, graphics and photos that bring you closer to an advanced laboratory, an ice sheet in Antarctica or a space mission in orbit. You’re helping us craft other types of high-quality journalism as well: Our newsletters are carefully written, edited and curated by staffers you have or will come to know and love. Our Science Quickly podcast is based on original reporting, collaboration with editors and scientists and exacting production.
Subscriptions keep this engine running so we can continue to deliver thoughtful, rigorous and independent science journalism to you. In an era of viral misinformation, this work is crucial. If you value what we do, I hope you’ll consider joining us as a subscriber