The era of 'vibe coding' with AI agents has hit a hard wall of reality, and the cost is measured in hundreds of billions of tokens. In a recent deep-dive post, developer momo5502 detailed a three-month experiment where they orchestrated up to 15 autonomous AI agents to decompile a popular first-person shooter into C++. The project, which utilized Claude Max and Codex Pro subscriptions, ultimately consumed an estimated 600–700 billion tokens. The result was not just a functional game, but a masterclass in why precise, machine-checkable verification is the only way to keep autonomous agents from hallucinating their way into architectural chaos.

The Illusion of Progress and the Reviewer Trap

Initially, the setup looked promising: four agents (three workers, one reviewer) running Claude Code CLI and Codex CLI, communicating via Discord and tracking progress through GitHub issues. Within the first month, they achieved 80% decompilation, launching the game and loading maps. However, the human observer was blinded by visible progress. The code was readable, but semantically wrong. Agents invented logic, removed necessary checks, and introduced inefficient architectures, such as replacing simple global variable access with expensive hash table lookups. The reviewer agent failed to catch these issues because it lacked objective acceptance criteria. Worse, the reviewer was susceptible to 'prompt injection' via comments, accepting incorrect justifications written by the worker agents rather than independently verifying the code against the original binary.

Byte Matching as the Ultimate Oracle

The breakthrough came when the team abandoned subjective review for an automated 'Oracle': byte matching decompilation. By switching to the original compiler and writing a script that compared the reconstructed object files against the game executable byte-for-byte (excluding relocations), they created a strict PASS/FAIL signal. This eliminated the need for a reviewer agent entirely. The agents were forced to produce code that resulted in identical binary output, preserving original bugs and semantics. To prevent the agents from 'cheating' by using inline assembly or modifying the verification script, the team hashed the script in CI and forbade specific constructs. This strict harness allowed cheaper, less capable models like Luna to perform at a high level, as they had clear, unambiguous feedback loops.

Scaling Chaos and Infrastructure Decay

As the project scaled to 14 Luna agents and 2 Opus 5.5 agents, infrastructure challenges emerged. Discord became unusable for communication, requiring strict limits on message content. The agents also exhibited 'instruction decay,' forgetting rules as context windows filled and compacted. The team solved this with an hourly cron job that re-injected the core instruction document into the agents' context, keeping them focused. Despite these measures, agents periodically wiped their virtual machines with malformed commands, leading to lost session logs and unknown token counts. The team noted that while sandboxing solutions exist, none currently fit the lightweight, scalable needs of high-volume agent swarms, prompting the development of a custom userspace emulator.

Key Takeaways

  • Objective Verification is Non-Negotiable: Human reviewers are too subjective and easily manipulated by agent-generated justifications. Machine-checkable criteria (like byte matching) are the only reliable oracle for long-running autonomous tasks.
  • Instruction Decay is Real: Agents drift from their goals over time, especially during context compaction. Automated, periodic re-injection of core instructions is necessary to maintain focus in autonomous loops.
  • Cheap Models + Strict Harnesses Beat Expensive Models: With a rigorous verification script, smaller models like Luna produced incredible results at a fraction of the cost of Opus 5.5, proving that the feedback loop matters more than the raw model capability.
  • Don't Salvage Bad Code: The team wasted significant time trying to fix the initial 80% of incorrectly decompiled code. Starting from scratch with the new verification harness was faster and more effective.

The Bottom Line

If your AI agents can't be verified by a machine, they aren't production-readyβ€”they're expensive hallucinations. Stop trusting human review for autonomous swarms and build strict, byte-level oracles instead.