How the Claude Code Team Verifies AI-Generated Code: E2E, Property and Differential Tests
Addy Osmani details the three test types the Claude Code team invests in, end-to-end, property and differential, and how they use verification loops to run dozens of agents in parallel.

By TechWhoKnows
4 min read
The Claude Code team at Anthropic has published a detailed breakdown of how they verify AI-generated code, revealing that approximately 90% of their code is written and reviewed by Claude itself. At that scale, verification loops become the critical building block that enables running dozens or hundreds of agents in parallel.
Author Addy Osmani outlines three test types the team invests in most heavily:
End-to-end tests are the top priority. Unlike unit tests, which can pass while the feature remains broken, E2E tests describe real user flows and are difficult to fool, either the whole feature works or it does not. The team gives agents "hands and eyes" on the running system using tools like Chrome DevTools Protocol, simulators or API harnesses. A feature map serves as a grading rubric listing features from the user's point of view with paths, keyboard shortcuts, selectors and expected results.
Property tests look at edge cases humans and agents tend to miss. Using libraries such as fast-check (JavaScript/TypeScript), Hypothesis (Python) or proptest (Rust), developers write statements they believe cannot fail, round-trip encoding, invariants like "account balance can never become negative", idempotency, equivalence between optimised and naive implementations, and let the library generate thousands of inputs to try to prove otherwise. Property libraries keep agents honest with edge cases not created with the same bias as the agent that wrote the original code.
Differential tests are particularly effective for large-scale migrations where functionality must remain identical. Both the old and new systems run with the same randomly generated inputs or production traffic samples, with irrelevant factors normalised and a list of approved changes for expected differences.
The team's verification process includes a verify skill in each repository that agents run before committing. Failing tests require human attention after three automated fix attempts. Code owners write their own checks, refined until they flag what the owner would flag. A 39-item reliability checklist triggers on diff patterns, for example, when session storage or message types change, the system checks whether new fields load safely from older transcripts.
Reviews use a single reviewer with multiple rounds building on earlier context rather than re-checking from scratch. Changes cannot modify their own checks, the REVIEW.md is read from the base branch, never from the pull request. The team runs cheap checks first (tests, build, linter) before opening a pull request, enforced by hooks.
A commit gate operates in observe mode (logging what would be blocked) before moving to enforce mode, with a safety valve allowing commits through after two blocks so agents do not get stuck. The team evaluates reviewers by tracking reverted merges, maintainer rejections, and a formal evaluation recreating repo state at the time of past incidents.
Sources
- 1Claude Code team blogclaude.dev
- 2Hacker News discussionnews.ycombinator.com