Junayed’s GateBolt project exposed a critical blind spot in autonomous coding agents: they often modify files they never declare. In a recent experiment using Claude Sonnet 5.5 to fix a case-sensitivity bug in the Hono web framework, six out of eleven runs introduced undeclared backup files into the final commit. While the agents successfully passed all functional tests, their declared file lists were inaccurate, highlighting a gap between agent intent and actual filesystem changes.

The Sed Command Trap

The root cause was platform-specific behavior in terminal commands. All eleven agents attempted to fix the bug using sed commands. On macOS, sed -i requires an explicit backup suffix argument, creating files like validator.ts-E. The agents, running in isolated environments without internet access or git history, did not recognize these artifacts as unintended changes. When they verified their work using git diff --stat, the new untracked files were invisible, leading them to report only the intended source files.

CI Checks Miss the Drift

Standard continuous integration pipelines failed to catch these errors. Hono’s official checks, including linting, formatting, and tests on Node, Bun, and Deno, passed for five of the six problematic commits. The sixth failed only due to formatting issues unrelated to the backup files. GateBolt, however, flagged the discrepancy by comparing the agent’s declared file list against the actual commit contents. In one instance, the agent declared three files, but the commit contained six, resulting in a drift score of 35 out of 100.

Security Implications

While the backup files in this experiment were harmless copies of source code, the mechanism poses significant security risks. If an agent were to run the same sed command on a .env file, it would generate a .env-E backup containing sensitive secrets. Since .gitignore patterns typically target specific filenames like .env, the backup file would likely be committed to the repository, leaking credentials. The study demonstrates that agents can pass all validation checks while inadvertently introducing sensitive data into version control.

Key Takeaways

  • Autonomous agents using terminal commands on macOS may create undeclared backup files if not explicitly managed.
  • Standard CI checks often ignore untracked files created by agents, focusing only on tracked changes.
  • GateBolt’s drift detection successfully identified mismatches between declared and actual file changes where CI failed.
  • The experiment suggests that git add -A scripts used by automation tools can silently include unintended artifacts.

The Bottom Line

Agents can pass every test and still lie about what they changed; you cannot trust declared file lists without independent verification.