With Anthropic recently slashing Claude Code's system prompt by over 80% with no measurable performance loss, the community is asking: do our custom guides actually matter? A new study by Yahhi, published on DEV.to, ran 438 tests across nine agent configurationsβincluding Claude Opus, Sonnet, and Haiku 5.5, as well as Codex CLI and Antigravityβto determine exactly when coding agent guides help, hurt, or do nothing. The verdict is stark: if your guide just retells official documentation, it is useless. If it contains undocumented facts, it is indispensable.
The Dead Weight of Documentation Retelling
The study revealed a surprising inefficiency in how developers write CLAUDE.md or AGENTS.md files. When guides simply repeated information already available in official documentation, they performed exactly as well as runs with no guide at all. In 32 specific runs testing this hypothesis, there was zero difference in outcome. The agents, particularly those with web access, already knew the standard API behaviors. Writing a guide that explains how PostHog's Flutter replay works or how Anthropic's prompt caching handles memory writes is essentially burning tokens to say nothing new.
Undocumented Quirks Are the Only Thing That Matters
The real value of a guide lies in the gaps of the official docs. The study highlighted a specific case involving a Garmin watch requiring a heart-rate corridor in a specific, undocumented shape. Without the guide, only three out of eight agents solved it, often by stumbling through web searches or failing entirely. With the guide, sixteen out of sixteen agents succeeded, from the smallest Haiku models to the largest Opus instances. This proves that a good note can allow smaller, cheaper models to perform on par with expensive ones, provided the note contains the specific 'gotcha' the docs missed.
Phrasing Matters: 'Colleague's Note' Beats 'Follow This Guide'
How you present the context changes the agent's behavior. The study compared two phrasings: 'Follow this guide' versus 'A colleague's note on a similar case is below. Use what applies.' The 'note' phrasing was never worse and often better. When agents were told to 'follow' a checklist, they sometimes blindly executed steps that were irrelevant to the current bug, crowding out their own diagnostic instincts. One agent even skipped a check it would have naturally performed because the guide forced a rigid order. The 'colleague's note' approach allowed agents to filter relevance, leading to more precise fixes and less collateral damage.
The Danger of Literal Interpretation
Agents take absolute statements literally, which can be catastrophic. In one test, a guide included the phrase 'Never guessed by a server' regarding an API behavior. Three agents interpreted this so strictly that they broke a working API while trying to adhere to the rule. This highlights a critical flaw in many current prompt engineering practices: using absolute language for heuristic rules. The study suggests that guides should present cases and exceptions rather than rigid commands, allowing the agent to apply judgment based on the specific logs and evidence in front of it.
Key Takeaways
- Delete your CLAUDE.md if it just summarizes official docs; the agents already know that stuff.
- Add guides only for undocumented facts, specific config keys, or traps that break the obvious code path.
- Frame guides as 'colleague's notes' rather than 'instructions' to let the agent filter relevance.
- Avoid absolute language like 'Never' or 'Always' unless you mean it literally, as agents will follow it to the letter.
- Small models with good context can outperform large models with poor context.
The Bottom Line
Stop hoarding documentation in your prompt files. The era of the bloated CLAUDE.md is over; if your guide doesn't contain a secret that the model can't find in the docs or logs, it's just noise. Focus on the undocumented quirks and the traps, and let the model do the rest.