Prompting an LLM to "be careful" with legal citations is like asking a compiler to guess at syntax errors. It doesn't work. A developer on DEV.to recently watched their AI assistant confidently cite 18 U.S.C. Β§ 301 for a data-protection claim and quote Consumer Credit Act 1974 s. 157 for a rule that exists nowhere in that statute. Both citations looked authoritative. Both were completely wrong. The fix wasn't a better system promptβ€”it was treating every legal norm as a structured data record with verifiable provenance.

Treating Legal Facts as Data Structures

The solution builds a corpus where each legal norm is stored as a record containing the official source URL, the exact article number, a content hash of the text, the last verification date, and a summary written only after reading the source. A scheduled job re-reads sources and compares hashes. If a source changes, the norm gets flagged for review. If a source becomes unreachable, the system doesn't silently keep citing itβ€”it tells the user the source couldn't be confirmed and withholds any deadline information. This is infrastructure thinking applied to a hallucination problem.

The Corpus Is Small, Deliberately

The current corpus contains 92 verified norms across six jurisdictions: Russia (25), United States (26), India (14), Hong Kong (11), United Kingdom (9), and Canada (7). Coverage thresholds matter here. The UK and Canada exist in the corpus but remain hidden from the user interface until at least 10 norms are verified for those jurisdictions. A country with three norms invites questions the system can't answer, so it stays silent. India's coverage is also explicitly limited to central Actsβ€”state law, customary law, and non-Hindu personal law statutes are marked as out of scope rather than silently extrapolated.

Refusal Is a Feature, Not a Bug

The system refuses to output article numbers unless the text was read, refuses to give deadlines unless the norm permits one, and refuses to generalize from central Acts to state law. It also refuses to say "the law says" when a source can't be confirmed, naming the gap instead. The developer acknowledges this makes the product less impressive in a demo but more useful in actual conversations. The interface supports 26 languages, though legal content currently exists only in Russian and English. The language boundary is displayed rather than hidden.

The Bottom Line

This is the right pattern for any domain where hallucinated citations cause real harm. Stop trying to prompt your way out of a data integrity problem. Build the verification pipeline, enforce the refusal logic, and let the system say "I don't know" when it should. The developer is asking users to try to make the summaries say something the source doesn't supportβ€”find the bugs that matter for this design, not just broken buttons. We are early on this pattern, but it's the one that actually scales beyond demos.

Key Takeaways

  • LLM legal hallucinations are a data provenance problem, not a prompting problem.
  • The fix uses content hashes, scheduled re-verification, and explicit refusal logic.
  • The corpus covers 92 norms across six jurisdictions with deliberate coverage thresholds.
  • The system hides jurisdictions until 10 norms are verified, and marks gaps explicitly.
  • Refusal to cite unverified sources makes the product less flashy but more trustworthy.