METR’s independent assessment of Anthropic, published November 5, 2026, claims a 1.9x acceleration in AI research and development since October 2025. However, this single number is a statistical mirage. The report reveals that the actual speedup ranges from 1.1x to 2.9x depending entirely on the link function used to model benchmark task difficulty. METR admits the data fits most of these models with similar accuracy, leaving the public with a massive uncertainty range rather than a definitive metric of progress.

The Statistical Fragility of the Index

The core metric, the Anthropic Epoch Capabilities Index (AECI), measures capability growth in index points per year. Before October 7, 2025, growth averaged 15 points annually; afterward, it jumped to 29 points. But translating raw benchmark scores into this index requires assuming how difficult tasks are distributed. If you assume benchmarks contain many easy tasks (complementary log-log), the speedup drops to 1.1x. If you assume many hard tasks (log-log), it balloons to 2.9x. METR’s default logistic curve sits at 1.9x, but the out-of-sample fit for these conflicting models is nearly identical, hovering around 86-87% R-squared.

Redactions Hide the Evidence

Anthropic retained unilateral redaction rights, stripping away the specific names of the 15 benchmarks used for the analysis. METR lists only broad categories like math, software engineering, and multimodal tasks. Furthermore, a planned check on how compute is split between capabilities and safety research was cancelled midway at Anthropic’s request. Without these details, the public cannot verify whether the index is tracking genuine capability leaps or merely saturation effects on specific, undisclosed tests.

Subjective Speed Versus Measured Progress

A glaring disconnect exists between measured capability and employee sentiment. While the index suggests a ~1.9x improvement, Slack polls among Anthropic staff estimated their personal workflow speedup at roughly 12x for Model 4. METR dismisses these polls as 'very loose evidence' because self-reported efficiency does not equate to lab-wide research acceleration. The interviews themselves were coded with the help of Claude, introducing a layer of circularity that METR acknowledges could distort results, though they deem deliberate misleading unlikely.

The 80 Percent Attribution Judgment

METR estimates that 80% of the observed speedup is attributable to AI itself, but this is a stated judgment, not a measured variable. The report notes that non-AI factors drove much of the late-2025 change, though the specifics remain redacted. METR found no convincing bottlenecks in algorithms, compute, or data for the next 12 months, but warns that benchmarks are running out of room to measure progress as Model 4 scores above 50% on all tested metrics.

Key Takeaways

  • The 1.9x headline is fragile; the true range is 1.1x to 2.9x based on modeling assumptions.
  • Anthropic redacted benchmark names and cancelled a key compute allocation check.
  • Staff feel 12x faster, but METR argues this subjective speed doesn't match the 1.9x capability trend.
  • The 80% AI attribution is a subjective estimate by METR staff, not a hard measurement.
  • Benchmarks are saturating, making future index readings increasingly difficult to interpret.

The Bottom Line

Stop quoting the 1.9x number without the 1.1x-2.9x asterisk. This report is less a verdict on Anthropic’s progress and more a case study in how redacted data and arbitrary statistical choices can manufacture precision where none exists.