A new documentation workflow proposed on DEV.to establishes a hard boundary for LLM-assisted technical writing: models may draft explanatory prose and propose code, but only humans can assert the observable output of that code. The core mechanism is a Python-based executor that parses Markdown files, identifying fenced code blocks and enforcing strict labeling conventions. This approach targets the most persistent defect in model-drafted documentationβ€”confident examples that have never actually been executed.

The Unit of Trust Is the Fence

The author argues that code blocks carry a higher burden of proof than surrounding text because a wrong command wastes an engineer's afternoon, whereas a wrong sentence costs only a minute of confusion. To manage this, every fenced block must be labeled as runnable, output, or illustrative owner=@handle. The runnable label indicates the block will be executed in a throwaway directory during CI, while output blocks contain the stdout/stderr verified by a human. Illustrative blocks are sketches that must never execute, requiring a named owner to vouch for their shape.

CI Integration and Exit Codes

The proposed tool, docex, runs on standard library Python and executes runnable blocks in temporary directories to prevent state leakage. It compares actual execution results against output blocks, failing the build if there is a mismatch. The CI gate uses specific exit codes: 0 for success, 1 for execution failures or output mismatches, and 2 for policy violations like unlabeled fences or missing owners. This structure forces developers to decide who owns the accuracy of each example before merging, turning documentation review from a stylistic debate into a factual diff.

Limitations and Security Implications

Executing documentation code in CI introduces security considerations, as it requires running code from pull requests. The author advises against using this on untrusted forks without disposable sandboxes and notes that examples requiring cloud credentials or GUIs should remain illustrative. Furthermore, the workflow acknowledges that while free model access can reduce drafting costs, it cannot replace the human verification step, as models cannot observe what a CLI prints on a specific machine configuration.

Key Takeaways

  • Models draft prose; humans verify outputs: A strict division of labor is enforced by CI.
  • Labeling is mandatory: Every code block must be tagged as runnable, output, or illustrative with an owner.
  • CI blocks merges on errors: Exit codes 1 and 2 prevent merging if code fails or labels are missing.
  • Security is a constraint: Running code in CI requires sandboxing for untrusted sources.

The Bottom Line

This workflow correctly identifies that AI's weakness is not in writing but in hallucinating execution results. By enforcing a human-owned output block, we transform documentation from a guess into a verified artifact.