What Claude’s Formal Verification Work Reveals—and What It Doesn’t

🔍 Read the full analysis: What Claude’s Formal Verification Work Reveals—and What It Doesn’t on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on tech for your team

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

Anthropic’s Frontier Red Team and collaborator Theorem reported on October 22 that Claude produced proofs for 239 fuzz-test entry points across 123 open-source libraries, establishing a limited no-crash property for specified compiled programs. The reported results matter because they apply formal verification at scale to machine code and surfaced bugs that testing had not found, but they cover particular builds and conditions—not the libraries’ overall security.

The stated property is that each tested program, compiled as a RISC-V binary, does not crash for any input smaller than 64 MB under the project’s model and assumptions. The specification also covers the absence of AddressSanitizer memory errors and failed assertions. Claude generated proofs in Rocq, a proof assistant; the Rocq checker then checked them against the specifications. The checker’s role is distinct from Claude’s: the result is not accepted merely because the model says it has proved something.

The workflow starts from existing fuzz tests rather than asking researchers to define software correctness from scratch. The team compiled the library and its fuzz-test entry point, instrumented the program, and used formal models of RISC-V instructions and Linux processes to generate a specification. Claude attempted a Rocq proof; when that effort exposed a possible bug, the code could be patched, the specification regenerated, and the proof attempted again. Anthropic says the effort ran for a few weeks and produced roughly 180 million lines of proof. A typical proof was about 600,000 lines, for a binary with a median length of 18,000 RISC-V instructions.

Anthropic reports that the work found hundreds of bugs, including issues it says ordinary fuzzing was unlikely to reach. Examples in the source account include a decompressor bug requiring an unusually large input, an image-decoder read requiring a 64 MB input and more than 2 GB of memory, and a flaw dependent on a sequence of calls involving two handles to the same file. It also reports rediscovering CVE-2026-56132 in libexpat and finding a logic error in a file-loading restriction. These are examples reported by the project, not evidence that every bug class or use of each library was checked.

Proofs Reach Beyond Fuzzing

Fuzzers test many generated or altered inputs, but their findings depend on which paths those inputs reach. A formal proof, if its specification and model correctly represent the target, aims to establish the stated property for every input within the defined range, not just sampled cases. That distinction helps explain why the team says its proof attempts exposed bugs tied to rare input layouts, large files, or particular call sequences.

The result is also a test of whether AI can reduce the labor involved in formal methods. The source article contrasts this work with verification projects that have required substantial human effort, including the seL4 microkernel, for which it cites two person-years to build and 20 person-years to verify. Here, the volume of generated proof is large, but the claimed trust burden is concentrated in the formal models, theorem statements, and Rocq checker. That could make machine-checked assurance more accessible if the method becomes cheaper and applies to useful software beyond these test cases.

For maintainers and users, the immediate value is narrower: the work supplies evidence about particular entry points and builds, and a set of reported defects to investigate. It does not replace code review, testing, or security analysis. Nor does a no-crash result by itself show that a program returns correct answers, enforces every policy, or has no exploitable behavior.

From Existing Tests to Machine Code

Formal verification checks whether a program satisfies a precise mathematical statement. Its difficulty often begins before the proof: teams must define what correct behavior means. The method described by Anthropic and Theorem takes a more limited route, turning a developer-written fuzz test into the property to prove. A fuzz test is an entry point that accepts inputs and often treats a crash or failed assertion as evidence of a defect.

In this project, the claim is therefore about what happens when those entry points run under a particular setup. The target is compiled RISC-V code instrumented with AddressSanitizer, with inputs bounded below 64 MB and behavior represented through formal models of the processor and Linux processes. The approach does not directly certify a source repository in every configuration. Anthropic says the results do not cover the x86 or ARM builds, or the glibc environments, that many users rely on.

The source article describes the work as, to Anthropic’s knowledge, the first large-scale use of AI for formal verification of production software at the assembly-code level. That is the organization’s characterization, not an independently established historical finding in the material provided. Anthropic says it is publishing proofs that did not require patches, previously unpublished trusted components, and specifications for fuzz tests the system could not prove.

Where the Proofs Stop

The most important limit is the scope of the theorem. The reported proofs address single-threaded, static RISC-V binaries linked against musl, under the project’s models and assumptions. They do not establish equivalent behavior for commonly used x86 or ARM builds linked against glibc. The source material also does not provide enough detail to independently assess every assumption in the formal models or the full validation process for the reported bug count.

“No crash” is not the same as “correct” or “secure.” A program can produce the wrong result, mishandle permissions, or contain a flaw that does not trigger the specified crash conditions. The reported file-restriction logic bug illustrates that some findings may concern behavior beyond memory safety, but the proof still depends on the particular assertions and properties encoded in each test. The source account says some fuzz tests could not be proved; it does not specify a complete success rate or explain how often proof attempts stalled.

Cost is another unresolved issue. The source says proofs can consume hundreds of millions of tokens, while newer models and shared proofs can reduce the burden. It does not provide a per-library cost, a total financial figure, or enough information to compare the approach’s cost and coverage directly with conventional auditing and testing. The proofs also rely on trusted components, including the formal models and checker, as well as supporting software and hardware.

Published Proofs and Follow-Up

Anthropic says it is releasing the proofs that did not require code patches, trusted components that were not already public, and specifications for tests it could not prove. Those materials give other researchers and maintainers a way to inspect the claims, examine the assumptions, and pursue the disclosed defects. The source article does not state a date for further releases or set out a timetable for checking other architectures.

The next practical test is whether the method can be reproduced and extended beyond this batch: to more libraries, different processor architectures, and configurations closer to those deployed by users. Maintainers will also need to confirm fixes and determine whether the reported issues affect their releases. Until comparable proofs cover those builds and properties, the findings should be treated as bounded evidence about selected entry points, not a general security certification.

Key Questions

What did Claude prove?

The project says Claude proved that 239 fuzz-test entry points across 123 libraries do not crash for inputs under 64 MB in specified, instrumented RISC-V builds, including no AddressSanitizer memory error or failed assertion under stated assumptions.

Does this mean the libraries are bug-free?

No. The result applies to specific properties, entry points, builds, and input bounds. It does not prove that the libraries have no bugs or that they are secure in every use.

How were the proofs checked?

Claude generated proofs in Rocq, and Rocq’s proof checker checked them against formal specifications. The result also depends on the accuracy of the specifications and trusted models of the machine and operating system.

Why did the work find bugs fuzzing missed?

Fuzzing samples inputs and execution paths. A proof attempt must account for all behavior in its specified scope, which can expose defects tied to rare inputs or call sequences. The project’s reported examples remain subject to the limits of that scope.

Do the findings apply to ordinary x86 and ARM installations?

Not based on the reported proofs. The source says the work covered static RISC-V binaries linked against musl, not the x86 or ARM builds with glibc that many users run.

Source: ThorstenMeyerAI.com