We have all been there. The prototype demoed perfectly, but the moment you shipped that internal Q&A tool, the support channel turned into a queue of confused users. One person gets an answer from an unrelated contract, another sees three copies of the same page, and someone else is wondering why the bot is spitting out customer phone numbers. The instinct is to debug the model layer: swap embedding models, retune chunk sizes, or bolt on a reranker. But a new field guide from DEV.to argues that the root cause is rarely the AI model itself. Instead, it is the document pipeline upstream of the embeddings.

Failure Mode 1: Scan Rot and Silent Errors

A massive share of enterprise documents are essentially images wearing a PDF costume. These are scanned contracts, photocopied HR files, and digitized faxes. When standard chunkers process these, they slice pages and compute embeddings on whatever OCR text is emitted. The result is 'scan rot': noisy text like '1?.3?4?5?6?.78' or missing table columns. The failure is silent because the embedding of this noisy text still lands close enough to the correct region in vector space to return a high cosine similarity score. The generator then confidently answers based on a corrupted premise, and no exception is thrown anywhere in the stack.

Failure Mode 2: The Duplicate Chunk Crisis

Vector stores do not deduplicate by default. Consider the lifecycle of a typical corporate contract: draft, reviewed draft, final, final-final, and the copy emailed to a vendor. All five versions describe near-identical content, causing their embeddings to cluster tightly. When a query hits this region, the top-k retrieval window fills with clones, crowding out the unique, authoritative passage. This is not an edge case; a survey by M-Files found that 96% of employees struggle to find the most current version of a document. The corpus you are indexing is manufactured by a process that creates duplicates, so your retrieval ranking must account for this statistical reality.

Failure Mode 3: Leaky Indexes and PII Exposure

Retrieval is an access-control bypass waiting to happen. Document-level permissions often do not survive chunk-level indexing, meaning the vector store flattens 'finance team only' into 'anyone whose query matches.' This leads to sensitive fields leaking simply because they match a query. IBM's 2025 Cost of a Data Breach report highlights the risk, noting that 63% of breached organizations had no formal AI governance policy. Furthermore, customer personally identifiable information made up 53% of data stolen in breaches. If your index contains raw phone numbers or ID cards from contracts and invoices, you are exposing your organization to shadow AI risks that cost an average of $670,000 more per breach.

The Fix: Clean, Mask, Compress, Index

The solution is a three-stage upstream pipeline: clean, mask, and compress. Cleaning involves exact-duplicate hashing and layout recovery for scans. Masking requires detecting sensitive fields before embedding, ensuring the processing runs locally to avoid third-party leaks. Compression focuses on fidelity-preserving reduction of files. For scan-heavy libraries, this can cut file size by 60% to 80%. More importantly, it reduces token costs. On a GPT-4o basis, image-heavy content saw an 85.9% cut in token costs after compression. Unlike one-time storage savings, these per-token savings compound across every query, every day, for as long as the library runs.

Key Takeaways

  • Sample OCR output by eye at full zoom; never trust average confidence scores.
  • Measure the duplicate ratio of retrieved chunks in your test query sets.
  • Grep your index for PII patterns; assume you will find some.
  • Keep masking inside your own network to prevent data exfiltration.
  • Judge pipelines on fidelity, masking coverage, throughput, integration, and observability.

The Bottom Line

The model gets the attention, but the pipeline gets the answers. Stop tweaking your rerankers and start debugging your data ingestion; your RAG system is only as good as the clean, masked, and compressed data you feed it.