Long-context AI models are often praised for their ability to maintain coherence over thousands of tokens, but a new test run by Shalendra Gaur exposes a critical vulnerability in how standard chat architectures handle state persistence. The experiment, conducted on October 22, 2026, compared a standard kept-chat implementation against the AXI architecture developed by Rajnish Choubey. While the standard model maintained 99.5% reply accuracy over 400 turns of ambiguous office chatter, a single erroneous reply at turn 191 corrupted the underlying list state for the subsequent 140 turns. This highlights a fundamental disconnect between conversational fluency and reliable state management in production-grade AI agents.

The Experiment: Ambiguity Over Structure

The test moved beyond rigid graph structures to simulate real-world noise. Using Gemini 3.8 Flash, Gaur constructed a conversation consisting of 400 turns centered on a hidden list of event items, each containing specific attributes like owner, day, amount, and status. Each turn was accompanied by 30 sentences of irrelevant office chatter designed to introduce ambiguity through loose references and conflicting details. The 'kept chat' arm re-sent the entire history with every prompt, applying each model reply directly to the list. In contrast, the AXI arm maintained the exact list on the host side, presenting it to the model at each step and validating replies before application, allowing for up to two re-asks if checks failed.

Performance Metrics: Accuracy vs. Efficiency

The results revealed a stark trade-off between state integrity and computational cost. The kept chat processed an average of 99,809 prompt tokens per step, totaling 39.9 million tokens for the run. Its time-to-first-token degraded significantly as context grew, rising from 1.86 seconds in the early turns to 5.57 seconds once the prompt exceeded 150,000 tokens. Conversely, the AXI implementation consumed only 1,177 prompt tokens per step on average, totaling 0.47 million tokens. AXI maintained a consistent time-to-first-token of 2.41 seconds throughout the run, demonstrating that host-side state validation can drastically reduce latency and token usage while ensuring the list remained 100% accurate across all 400 steps.

The Cost of a Single Slip

The most compelling finding is the difference between reply accuracy and state accuracy. The kept chat produced correct replies for 398 out of 400 turns (99.5%). However, the error at turn 191 involved changing the wrong item in the list. Because the architecture applied replies immediately without validation, this single mistake persisted in the state until a later message inadvertently overwrote it at turn 331. Consequently, the held list was incorrect for 140 consecutive turns, resulting in an overall state accuracy of only 65.0%. This demonstrates that in long-running agentic workflows, a low error rate in generation does not guarantee a low error rate in the final system state.

Key Takeaways

  • State validation is critical: A 99.5% reply accuracy can still result in a 65% state accuracy if errors are not caught and corrected before persistence.
  • Token efficiency: Host-side state management (AXI) reduced token consumption by approximately 85 times compared to full history re-sending.
  • Latency stability: Standard chats suffer from increasing latency as context grows, whereas validated architectures maintain consistent response times.
  • Architecture matters: The AXI layer, by checking replies against the allowed set before applying them, prevented state corruption entirely.

The Bottom Line

Reply accuracy is a vanity metric for agents; state accuracy is the only metric that matters in production. Until developers adopt host-side validation layers like AXI, long-context chats remain fragile systems prone to silent, persistent corruption.