A recent analysis from Jeff PDC on DEV.to exposes a critical failure mode in current agentic coding workflows: the disconnect between internal code consistency and external business correctness. When an agent is tasked with adding a refund endpoint for cancelled orders, it reliably generates routes, service methods, migrations, and tests that all pass. However, the resulting code may fail to account for real-world states, such as orders that were cancelled before payment was captured. The agent satisfies the literal ticket but ignores the implicit business logic, leading to bugs that only surface weeks later when finance teams notice refunds being issued for unpaid orders.

The Hallucination Loop in Testing

The core issue is that agents verify their own work using tests they generate themselves. If an agent misinterprets a requirement, it creates a test that confirms the misinterpretation, locking the wrong behavior in as "expected." This self-referential loop means the agent never checks against the actual payment ledger or state machine; it only checks if the code contradicts its own generated logic. The agent has never seen the real-world boundary conditions, only the ticket text and existing code patterns, leading to a false sense of security when the CI pipeline goes green.

Ambiguity Is the Enemy of Agentic Code

Failures consistently stem from agents picking one plausible reading of an ambiguous requirement without seeking clarification. Common examples include sorting by created_at instead of updated_at because it matches other endpoints, or returning a 200 status for unknown webhooks to avoid breaking senders. In each case, the code is clean and reviewable, but the bug lies in the interpretation. No amount of diff reading catches these errors without deep context, and agents rarely pause to ask, "Which of these three interpretations did you mean?"

Contracts Over Clarification

The solution isn't to make agents ask better questions, but to remove ambiguity from the input. Jeff PDC recommends defining explicit contracts, such as OpenAPI specs with strict status codes and error responses, before implementation begins. By constraining the agent with a machine-readable spec, developers prevent it from quietly deciding that an unknown event should return a 200. Additionally, naming edge cases directly in the ticket, like "skip refunds for orders that were never captured," forces the agent to handle specific branches rather than guessing based on pattern matching.

Key Takeaways

  • Agents verify internal consistency (syntax, types, self-generated tests) but not external correctness (business rules, real-world state).
  • Self-generated tests create a hallucination loop that locks in misinterpretations of ambiguous requirements.
  • Defining strict API contracts and explicit edge cases in tickets reduces agent error rates more effectively than prompt engineering.
  • Human review must focus on failure tests and business logic alignment, not just code quality or passing CI.

The Bottom Line

Stop trusting green checkmarks from agents as proof of correctness; move ambiguity upstream into strict contracts so the machine can actually check itself against reality.