When you hand off implementation work to a coding agent, the definition of 'done' needs to be rock solid before the code gets written. But what happens when the machine-readable spec that defines 'done' silently loses half its meaning due to a syntax quirk? That is exactly what happened in the open-source tool spec-lane, where a YAML comment truncated a success criterion, yet the verification gate still passed. The bug wasn't in the comparison logic; it was in the data ingestion pipeline.

The Invisible Truncation

The issue, tracked as Issue #45, started with a simple line in an intent.yaml file: success: - ledger has exactly one PhaseGate row # include the negative case too. To a human reader, the comment clarifies that the negative case is part of the requirement. But to the yaml@2.9.0 parser, that text after the hash is just a comment. The parsed JavaScript value became simply 'ledger has exactly one PhaseGate row'. The source file hadn't changed, but the data reaching the downstream verification code had.

Gates Pass on Partial Data

Because the verification matrix in verification.yaml also contained the shortened string 'ledger has exactly one PhaseGate row', the gate compared two identical values. It didn't know the original intent included the negative case. The CLI reported success with exit code 0, allowing the workflow to advance from phase 3_implement to 4_verify. The gate wasn't broken; it was faithfully comparing the truncated inputs it received. This highlights a dangerous gap: schema validation ensures the shape of the data, but it cannot verify that the source representation was preserved.

The v0.11.0 Fail-Closed Fix

Maintainer shiki-yusuke shipped a fix in spec-lane v0.11.0 via PR #48 that addresses this at the boundary. Instead of trying to recover lost intent later, the tool now walks the YAML AST and inspects the original source text. If it detects an unquoted plain scalar followed by a hash, it rejects the input before it ever reaches the gate. This moves the check from 'does the value match?' to 'was the value silently modified during parsing?' The fix is intentionally narrow, trading some false positives for a fail-closed safety mechanism.

Key Takeaways

  • YAML comments are invisible to parsed values; unquoted strings with hashes will truncate silently.
  • Schema validation cannot detect if information was lost during the parse step.
  • Spec-lane v0.11.0 now rejects unquoted success criteria containing inline comments to prevent silent truncation.
  • Always quote strings in YAML that contain special characters like #, even if they look like comments.

The Bottom Line

If your AI agent's definition of 'done' can be altered by a single character, your verification pipeline is fragile. Stop trusting that the gate passed; start auditing what actually reached the gate.