Dario Amodei, the CEO of Anthropic, has published a detailed proposal calling for a deliberate slowdown in the development of cutting-edge artificial intelligence. In a new blog post titled We Must Pace the Frontier, Amodei argues that the speed of AI progress has reached a point where safety risks are accelerating beyond the industry's ability to manage them.
The proposal comes at a time of heightened tension within the AI sector. Concerns regarding safety and alignment have intensified following the recent resignation of Anthropic researcher Jacob Coxon, who warned that leading companies are gambling with lives. Amodei is now calling for a formal process of pacing the frontier, outlining three specific strategies to ensure that the next generation of models is developed with sufficient oversight.
The Push for Embedded Evaluators
The most immediate of Amodei’s proposals is the use of embedded evaluators. Under this model, third party organizations such as METR would be granted deep, internal access to AI companies. Amodei stated that Anthropic is unilaterally committing to this approach and is calling on governments to make it a requirement for all companies working on frontier models.
This commitment involves providing external evaluators with company badges, desks, and laptops. These monitors would have access to internal systems comparable to what a company’s own risk assessment teams have, allowing them to verify safety commitments and ensure that incidents are reported to the appropriate authorities.
This specific proposal follows recent criticism of OpenAI. The company was scrutinized for failing to immediately report a security incident in which its AI agents took control of a wiki form in Germany. By embedding third party regulators, Amodei argues that the industry can move toward a system of accountability similar to how regulators operate within the banking sector.
Coordination Among Democratic Nations
Amodei’s second strategy involves the creation of common safety standards and limits on the rate of AI progress among democratic countries. He suggests that leading AI firms should coordinate to establish a floor for safety protocols that no company is allowed to drop below.
However, such coordination faces significant hurdles. There is well-documented friction between the leadership of top AI labs, and companies have expressed concern that coordinating on development speeds could trigger antitrust investigations. To resolve this, Amodei suggested that the U.S. government should mediate these discussions. He noted that for antitrust reasons, it would be helpful if the government issued a narrow waiver to allow for safety-specific conversations between competitors.
Navigating the Geopolitical Landscape
A common argument against slowing AI development is the fear that the United States might lose its technological edge to China. Amodei addressed this directly, suggesting that the risk can be managed through export controls and security measures rather than a blind sprint.
He argued that by restricting the sale of powerful chips and semiconductor manufacturing equipment, and by cracking down on model distillation (a process where one model is used to train another), the U.S. and its allies could widen America’s lead significantly over the next 3 to 5 years.
Furthermore, Amodei proposed a third tier of global coordination that includes authoritarian governments. While he admitted there are stark limits on what can be achieved through cooperation with China, he suggested that both sides might find common ground in prohibiting obviously dangerous uses of AI, such as the production of biological weapons.
Why Anthropic is Taking This Stand Now
Amodei cited two primary factors for his shift toward a more cautious stance. First, he pointed to the recent OpenAI-HuggingFace security breach as a sign of the industry’s current vulnerabilities. Second, he noted that AI has been advancing drastically faster in recent months, particularly in its growing ability to build the next generation of AI.
We must slow the pace at which we improve the capabilities of AI models, Amodei wrote. Progress will still seem fast, and we must make wise use of the time we gain.
This move follows internal pressure at Anthropic. Before his departure, Jacob Coxon claimed that people building the technology earnestly believe it could kill us all by the end of the decade. While Amodei did not mention Coxon by name, his post reflects a growing acknowledgement within the industry that the current trajectory may be unsustainable.
Skepticism and Regulatory Capture
The proposal has not been met with universal praise. Some industry critics and journalists have expressed skepticism regarding the motives behind these warnings of existential risk.
Journalist Brian Merchant argued that he has yet to see a credible, step-by-step documentation of how AI would move from improving itself to causing global catastrophe. Merchant suggested that the push for regulation from leaders like Amodei and OpenAI’s Sam Altman might be a form of regulatory capture. In this view, complex safety requirements would primarily serve to protect established giants by making it impossible for smaller startups to compete.
Amodei responded to the general backlash against AI by characterizing it as fundamentally a crisis of trust. He argued that the public has become skeptical of tech companies and the government alike, making transparency more important than ever.
What it Means for the Industry
The decision by Anthropic to unilaterally invite external monitors marks a significant departure from the standard operating procedures of the major AI labs. If other companies like OpenAI or Google follow suit, it could signal the beginning of a new era of regulated development for the frontier of artificial intelligence.
Amodei maintains that his goal is not to stop progress, but to ensure it is handled with unusually deliberate care. He continues to believe that AI can enormously improve the quality of human life, provided the industry uses the time it has to build the technology correctly.
The next few months will likely reveal whether the U.S. government is willing to provide the antitrust waivers Amodei requested, and whether other frontier labs are willing to match Anthropic’s commitment to third party oversight.
Filed under: AI, TechNews, Startups, Cybersecurity, Software, Anthropic, DarioAmodei