A recent DEV.to post outlines a practical Node.js pattern for website chatbots that eliminates the risk of hallucinated business details. The core problem is simple: when a general-purpose LLM lacks specific context, it often invents plausible but incorrect answers, such as claiming a clinic is open on Saturdays when the website never mentions weekend hours. For customer-facing applications, a confident wrong answer is significantly worse than a refusal to answer, potentially leading to frustrated users and lost trust.

Curate Your Knowledge Base

The first step in this workflow is strict curation. Developers should not point the bot at the entire site. Instead, define an array of allowedPages containing only high-value content like contact details, service descriptions, and FAQs. This approach explicitly excludes outdated blog posts or job ads that could confuse the model. By splitting longer pages into short chunks and tagging each with a page ID, you create a structured dataset that is easier to retrieve and audit.

Refuse Before Generating

Efficiency and accuracy both improve when you filter results before calling the model. The proposed code uses a search function to score page chunks against the user's question, applying a minimum threshold score of 0.75. If no chunks meet this relevance bar, the system immediately returns a standard "I don't have that information" message. This refusal step prevents unnecessary API costs and ensures the model is never forced to guess when no relevant context exists.

Validate Numerical Facts

Even with good retrieval, models can hallucinate specific numbers. The article recommends a post-generation validation step that uses regex to extract facts like phone numbers, emails, and times from the model's response. It then checks if these extracted strings actually exist in the source chunks. If a generated answer contains a phone number that wasn't in the retrieved text, the response is rejected. This blunt check prioritizes safety over flexibility, accepting that some correct but reworded answers might be filtered out.

Key Takeaways

  • Restrict chatbot knowledge to curated, specific pages rather than the whole site.
  • Implement a relevance threshold to refuse answering when context is missing.
  • Validate generated numerical facts against source text to catch hallucinations.
  • Treat unanswered queries as a to-do list for improving website content.

The Bottom Line

Prompt engineering alone is not enough for business-critical chatbots; you need deterministic retrieval and validation layers to prevent costly errors.