Language models are fluent, but they are fundamentally blind to your proprietary data. During an internship at Valentius Kryptix, developer Samruddhi Shirgave built a Retrieval-Augmented Generation (RAG) system to solve this exact problem. The goal was not to retrain the model, but to see how much accuracy improves when the LLM is forced to read specific documents before answering. Using Python, ChromaDB, and Gemini, the project proved that context injection is often more effective than model swapping.

The Pipeline: Python, Chroma, and Gemini

The implementation was lean and focused on practical results. Shirgave ingested two real-world documents totaling approximately 9,000 words: a prompt engineering report and notes on agribusiness forecasting. The text was processed into 40 overlapping chunks of roughly 2,000 characters each. These chunks were embedded and stored in a local Chroma database. For every user query, the system retrieves the top four most relevant chunks and injects them into the prompt, ensuring the model answers only from the provided context.

Lesson 1: Chunking Is a Design Decision, Not a Checkbox

The most critical insight from the build was that chunking strategy dictates success. Chunks that are too large introduce noise, while chunks that are too small strip away necessary context. Shirgave found that overlapping chunks, cut specifically at paragraph or sentence boundaries, maintained readability for the model. This isn't just a technical detail; it is the difference between a model that understands a full argument and one that hallucinates based on fragmented data points.

Lesson 2: Debug Retrieval Before Blaming the LLM

Developers often rush to blame the generation model when answers are wrong. However, Shirgave printed the retrieved chunks before connecting the generation step. In one test, the fourth retrieved chunk came from an unrelated section of the notes. While the final answer remained correct, this revealed that top-k search is not flawless. Inspecting the retrieval pipeline independently allows builders to isolate whether the failure lies in finding the right data or in generating the response.

Lesson 3: Instructions Are Your Safety Net

Behavioral control is essential for trust. The system was explicitly instructed to answer only from the given context and to admit when information was missing. This constraint makes the model's behavior predictable and testable. In a quick validation test, the plain model claimed it had no access to files, while the RAG version correctly retrieved the exact schema match rate from the report. The difference wasn't in the model's intelligence, but in the discipline of the prompt structure.

Key Takeaways

  • Pipeline Choice: Python, ChromaDB, and Gemini provide a robust, low-code entry point for RAG.
  • Chunking Strategy: Use overlapping chunks cut at semantic boundaries to preserve context.
  • Debugging Workflow: Always inspect retrieved chunks separately from generated answers.
  • Prompt Discipline: Strictly constrain models to use only provided context to prevent hallucinations.

The Bottom Line

Shirgaveโ€™s project is a reminder that you don't need massive datasets to validate RAG. Start small, inspect your retrieval, and respect the chunking process. If your model is guessing, your retrieval is likely broken.

Limits and Next Steps

The current system is intentionally minimal, handling two documents with a fixed top-4 retrieval. Shirgave plans to expand the test set to measure retrieval quality more rigorously. Future iterations will include re-ranking strategies and comparisons of different chunk sizes to further optimize performance.