Kevin Leary, founder of the Laravel SaaS product ai-track.app, has implemented a support architecture that bypasses traditional help articles entirely. Instead of relying on static documentation, his system derives answers directly from the application's codebase and the user’s real-time account data. This approach eliminates the staleness and coverage gaps inherent in manual documentation, allowing the AI to resolve abstract support tickets instantly.

The Failure of Documentation-First AI

Standard AI support bots typically use retrieval-augmented generation (RAG) over embedded help articles. Leary argues this design has two fatal flaws: documentation is a lossy, lagging copy of the product, and it sets a hard ceiling on what the bot can answer. If a feature changes weekly but the docs do not, the bot confidently provides outdated instructions. By treating the product itself as the single source of truth, the new system ensures that every answer reflects the current state of the application logic.

Architecture: Code, Context, and Confidence

The system combines three distinct data streams before invoking the LLM. First, a distilled product brief is generated from the codebase using AI coding tools, capturing exact rules like billing providers and feature thresholds. Second, the system queries the user’s full account context, including organization settings, active resources, and recent errors. Third, automatic ticket metadata records the user's browser, OS, and viewport. This enriched prompt is sent to a fast, cheap LLM API, which must return a structured response including a confidence rating.

Handling Abstract Questions

Because the model reasons over the product’s actual rules and the user’s actual state, it handles tickets that keyword-matching bots cannot. For example, if a user asks why their performance chart is empty, the system sees the account is only two days old and knows the chart requires three days of data. A standard doc-search bot would return a generic article or hallucinate. Similarly, for vague queries like "something failed overnight," the system checks for recent errors on the user's resources and explains the retry cycle, deriving answers the way a senior support engineer with database access would.

Safeguards Prevent Hallucinations

Leary emphasizes that intelligence is useless without discipline. The system employs strict confidence gating: only replies marked as "high" confidence are posted automatically. Medium and low-confidence answers are discarded and routed to human agents, ensuring no guessed answers reach customers. Hard topic exclusions prevent the AI from answering billing specifics or ambiguous queries. Furthermore, the design ensures transparency; auto-replies come from a real admin account and flip tickets to a feedback state, allowing users to easily escalate to a human if needed.

Minimal Infrastructure, Maximum Leverage

Contrary to complex RAG pipelines, this solution required minimal infrastructure. It consists of one service class, one LLM API integration, and a context builder that gathers account state via standard database queries. There is no vector database, no embedding pipeline, and no retraining cycle when features ship. The hard part wasn't the code, but the product judgment required to define what belongs in the product brief and where the AI must refuse to answer.

Key Takeaways

  • Code is the Source of Truth: Regenerating product briefs from the codebase eliminates documentation drift.
  • Context is King: Answering "why is your chart empty" requires live account data, not generic articles.
  • Confidence Gating is Essential: Auto-posting only high-confidence answers prevents hallucination damage.
  • Simplicity Wins: The system uses standard Laravel queries and a single LLM API call, avoiding complex vector stores.

The Bottom Line

Leary’s approach proves that the most effective AI support isn't built on massive vector databases, but on rigorous product judgment and tight integration with live data. By treating code as the source of truth and enforcing strict confidence gates, this architecture offers a scalable, low-infrastructure path to accurate automation that static documentation can never match.