Claude Automates Code Safety Proofs
Anthropic’s model generates massive mathematical proofs to verify open-source software, uncovering hidden bugs in a matter of weeks.
By Madison Lewis ·
Software verification has historically been a slow, resource-intensive process reserved for critical systems, but new developments are rapidly changing that landscape. Anthropic has announced that its large language model, Claude, is now capable of mathematically proving the correctness of existing software code without significant human intervention. This breakthrough addresses a fundamental challenge in AI-generated programming: ensuring that automated code is actually safe to run.
The initiative, conducted by Anthropic’s Frontier Red Team in partnership with Theorem, focused on generating formal proofs rather than writing new applications. Over a period of just a few weeks, Claude processed test programs from 123 open-source libraries associated with Google’s OSS-Fuzz security project. The result was the creation of approximately 180 million lines of mathematical proof code. This volume of work would have taken a team of human specialists decades to complete manually, highlighting the dramatic shift in efficiency brought by AI.
The method works by translating standard crash tests into mathematical problems. Claude analyzes the compiled machine code of the target software and constructs extensive proofs to demonstrate its correctness. These proofs are then automatically checked by an independent verification program known as Rocq. According to Anthropic, the process is largely autonomous, requiring very few human reviewers to oversee the output, although the underlying formal foundations and tools were developed by experts at Theorem.
One of the most significant outcomes of this project was the discovery of numerous software defects. While generating the proofs, Claude identified hundreds of bugs that conventional testing methods had failed to detect. Many of these issues were latent, appearing only under specific conditions such as unusually large input files or particular sequences of memory access. The model was permitted to fix these errors as part of the verification process, effectively improving the stability of the tested libraries.
The impact was particularly evident during the verification of a widely used PDF reader. In this instance, the team found as many security vulnerabilities in a single week as are typically reported for that specific software over an entire year. In another case, Claude uncovered a logical error that had remained undetected for more than a decade. Anthropic notes that even when a complete proof cannot be generated, the attempt itself often surfaces real bugs, making the process valuable regardless of the final outcome.
Despite these successes, the approach has strict practical limitations. Anthropic emphasizes that a "formally verified" status does not guarantee that the software is error-free in all possible scenarios. The proofs are valid only under specific conditions, including a particular processor architecture, single-threaded execution, and input files smaller than 64 MB. In real-world environments, these libraries often run on diverse hardware configurations, meaning the verification does not cover every possible use case.
The computational cost of this process remains high. Individual proofs can consume hundreds of millions of tokens, though Anthropic has not disclosed the total financial cost of the project. However, the company argues that the savings in developer time and review costs will outweigh these expenses. If developers can trust AI-generated software to be correct under defined conditions, they can spend less time on manual code review, streamlining the entire development pipeline.
Recent data also suggests that efficiency is improving rapidly. A smaller test run indicated that proofs are reusable across different contexts. Furthermore, newer model versions are becoming more efficient; Opus 5.5, for example, solved half of the required proofs using approximately 40 percent fewer tokens than its predecessor, Opus 5. This trend points toward a future where formal verification becomes a standard, routine part of the software development lifecycle rather than a specialized task.
