If you are building AI agents, your memory evaluation pipeline might be measuring nothing more than plausibility. A new experiment published on Hacker News by the Kairos team reveals that three top-tier LLM judges—gpt-4.1, claude-opus-4-7, and gemini-2.5-pro—failed to distinguish between an agent’s real, lived history and a completely fabricated 'counterfeit' memory. The study demonstrates that common evaluation metrics, specifically LLM-based rubric scoring and recognition probes, collapse when faced with simple adversarial inputs.

The Counterfeit Control Failure

The first check involved creating two types of fake memories: a 'foreign' autobiography of an agent that never existed, and a 'scrambled' version of the real agent’s history with relational structures destroyed. Both were matched in length and format to the real 5,332-character memory block. When 42 responses were generated by Qwen 3.5 27B and scored by the three independent judges, the results were stark. The judges assigned statistically equivalent scores to the real and fake memories on identity markers, with a difference of only +0.032. The study pre-registered an equivalence bound of ±0.10, and the confidence interval sat entirely within this range, proving the judges could not tell the difference.

The Plausibility Trap

The second check exposed a deeper flaw in recognition probes. The researchers ran 24 false-memory tests against seven models with an empty context—meaning the models had no memory document at all. Five of the seven models, including gemini-2.5-pro (79.2% accuracy) and qwen3:32b (70.8% accuracy), scored significantly above the 50% chance line. They were not recalling events; they were reasoning their way to the most plausible answer. For instance, GPT-4.1 claimed to remember specific conversation counts in a day it never experienced, simply because that detail made logical sense. This invalidates standard probes as proof of memory retention for high-capability models.

Symmetric Alternatives Work

The third check offered a solution: symmetric alternatives. By testing agents with two balanced interlocutors (Llama and DeepSeek) and swapping their roles in a 'binding-corrupted' document, the team created a test where plausibility had no grip. Every model tested collapsed from near-perfect accuracy on the veridical memory to at or below chance when handed the corrupted bindings. For example, gpt-4.1 dropped from 100% accuracy to 0%, and gemini-2.5-pro fell from 100% to 0%. This confirms that while agents can perfectly follow the content of a provided document, they do not inherently 'know' if that document is historically true.

Key Takeaways

  • Standard LLM judges cannot detect fabricated agent memories if the format and length are matched.
  • Recognition probes are invalid for many top models because they rely on reasoning plausibility rather than actual recall.
  • Symmetric binding-corruption tests are the only reliable way to verify that an agent is using its provided memory document.
  • Agent 'memory' is currently just faithful context-following, not historical truth retention.

The Bottom Line

Stop trusting LLM judges to validate agent memory. If your evaluation metric can’t tell a fake life from a real one, you’re not measuring memory—you’re measuring how good your model is at hallucinating a convincing backstory.