The widely cited range for fresh data requirements in self-consuming LLM training loopsβ€”roughly 1% to 30%β€”is not a disagreement about optimal values, but a failure to distinguish between two fundamentally different operational states. A new reproduction study of the paper 'Prevention Is Not Cure' demonstrates that maintaining a loop in a stable state requires 13.4x to 27.6x more fresh data than recovering a fully collapsed loop. This finding challenges the standard assumption that prevention is the cheaper strategy, revealing that the cost of 'intact support' is vastly higher than the cost of 'recoverability.'

The Illusion of the Universal Rate

The study identifies that the 'folk closed form' often used to estimate per-generation loss is actually a lower bound on unconditional loss probability, not the conditional rate practitioners experience. Direct simulation across 602 per-symbol readings showed that the measured rate exceeded this theoretical bound in 519 cases, with a median ratio of 0.08437. This means the standard formula understates the loss rate by a median of 11.9x for symbols that are frequently present, leading to significant under-provisioning of fresh data in healthy pipelines.

Pool Windows and Accumulation Risks

A critical control variable often overlooked is the pool window (w), which determines whether a training pipeline replaces old data or accumulates history. The research shows that accumulating history (w > 1) pushes a loop two to three orders of magnitude closer to collapse compared to replacement (w = 1) at the same fresh-data budget. For example, in a uniform distribution with n=50, the stability boundary drops from 0.2937 at w=1 to 0.000977 at w=64. This indicates that standard fractions calibrated for replacement scenarios are dangerously inaccurate for the accumulating pipelines common in production.

Prevention vs. Recovery Metrics

The core discovery lies in the disparity between 'prevention' (Ξ»_stationary) and 'recovery' (Ξ»_heal) thresholds. Prevention is defined as the largest rate where the worst symbol's absence stays ≀ 1%, while recovery is the smallest rate where a collapsed loop returns to full support with β‰₯ 0.5 probability. In a uniform distribution with w=4, the prevention threshold was 0.0504741, while the recovery threshold was only 0.00377383β€”a 13.4x difference. In a two-hot distribution with w=1, the prevention criterion was unsatisfiable within the domain, highlighting that protecting a loop's integrity is a far stricter demand than allowing it to heal.

Key Takeaways

  • Prevention is expensive: Keeping a loop stable costs 13-28x more fresh data than rescuing a collapsed one.
  • Accumulation hurts: Using a historical pool (w > 1) brings the stability boundary 2-3 orders of magnitude closer to collapse.
  • No point of no return: The process is ergodic for any Ξ» > 0, meaning collapse is reversible given enough time, though practical horizons may make it seem permanent.
  • Standard formulas fail: The 'folk closed form' is a lower bound that often understates actual loss rates by an order of magnitude.

The Bottom Line

If you are provisioning data for a self-consuming LLM pipeline, you are likely underestimating the cost of stability by an order of magnitude because you are optimizing for recovery rather than prevention. Stop quoting the 1-30% range as a universal constant and start auditing your pool window and stability criteria.