The team behind Claude Code has published a deep dive into their internal verification strategies, revealing that approximately 90% of their codebase is now written and reviewed by Claude agents. This high degree of automation necessitates robust verification loops, which the team argues are the critical building block for scaling agent usage from a few parallel instances to dozens or hundreds. The blog post outlines the specific testing methodologies and workflow integrations required to maintain quality at this velocity.
The Necessity of Automated Evidence
According to internal testing cited in the article, equipping agents with a separate grader to check work against a rubric improved task success rates by up to 10 points compared to standard prompting loops, with the most significant gains observed on complex problems. The team posits that verification transforms expensive tasks like generating benchmarks and specs into cheap, repeatable software operations. By closing the loop, agents can use failure data to update their own rules in CLAUDE.md, effectively preventing recurring issues without constant human intervention.
Three Pillars of Effective Testing
The article identifies three specific test types as worth investing in, dismissing flaky or slow tests that train agents to retry rather than fix code. First, end-to-end (E2E) tests are prioritized because they are difficult to fool and provide definitive done criteria, often revealing broken features that unit tests miss. Second, property tests use libraries like fast-check or Hypothesis to generate thousands of edge-case inputs, ensuring invariants like round-trip encoding or idempotency hold true. Third, differential tests serve as an oracle during migrations by comparing old and new system outputs on identical inputs to catch breaking changes.
Verification Means Running the App
While test suites are important, the source clarifies that running the app end to end matters more than tests. To codify this, the team uses a specific skill called 'verify', stored in a .claude/skills/ folder within the repo. This skill instructs Claude on how to start the simulator or run the app locally, acting as a smoke test harness. Claude is expected to use the verify skill before committing changes, mirroring how a human QA engineer would inspect the running system rather than just relying on unit test passes.
Targeted Checks and Human Escalation
Operationalizing verification involves strict gates and escalation paths. The team maintains a reliability checklist with 39 items, each associated with a specific trigger pattern in a diff. Crucially, Claude reads the diff and checks only the items whose trigger matchesβusually 3 to 10 itemsβrather than running all 39 checks automatically. This targeted approach ensures efficiency while capturing specific risks, such as changes to session storage that might break old transcripts. When checks fail, the workflow enforces human attention. If a check is still failing after three rounds of agent attempts, Claude stops and brings in a human reviewer. This prevents agents from getting stuck in infinite retry loops and ensures that persistent issues are addressed by someone who can provide context. Additionally, the commit gate operates in 'observe' or 'enforce' modes; in enforce mode, unverified commits are blocked unless an explanation is provided, with a grace period allowing a commit through after two blocks to prevent stagnation.
Key Takeaways
- Running the app end to end via a 'verify' skill is prioritized over simply passing test suites, as it mimics real user flows.
- The 39-item reliability checklist is triggered selectively based on diff patterns, with Claude checking only 3 to 10 relevant items per change.
- A hard stop occurs after three failed verification rounds, automatically escalating the issue to a human reviewer to prevent agent stagnation.
- Checks must be immutable relative to the change being reviewed, with REVIEW.md read from the base branch to prevent agents from gaming the system.
The Bottom Line
Anthropicβs approach proves that AI coding is no longer about generating code, but about building the infrastructure to trust it. By integrating targeted checks, mandatory app verification, and clear human escalation paths, teams can scale agents safely. If you aren't treating verification as a first-class engineering problem with defined stop-gaps, you're just generating noise.
Sources
Hacker News: How the Claude Code team thinks about verification