In the wild world of AI agent deployment, the most dangerous bugs aren't the ones that crash the systemβ€”they're the ones that whisper nonsense while everything looks green. Developer Nabeel Baghoor recently uncovered a silent failure in his Retell voice agent for Airheads HVAC. By diffing two JSON exports, he discovered that while the prompt correctly identified the business as an HVAC service, the underlying configuration was still bleeding dental keywords like 'Invisalign' and 'cavity' into the speech-to-text pipeline.

The Ghost in the Machine

The root cause was lazy cloning. Five agent exports shared the same last_modification_timestamp and version number, indicating they were duplicated from a single base template originally built for Memorial Family Dental. While Baghoor updated the prompt for the HVAC client, he missed the boosted_keywords field, which still contained twenty dental terms instead of HVAC-specific phrases like 'refrigerant' or 'compressor.' Worse, the post_call_analysis_data was still asking if callers had a 'legitimate dental appointment need,' causing every HVAC call to potentially return qualified: false in the dashboard, even if the booking was successful.

Why Testing Failed to Catch It

Standard conversational testing is blind to this kind of config drift. If you call the agent and ask to book an AC repair, the prompt guides the model to the right action, and the transcript looks perfect. But boosted_keywords operate at the transcription layer, biasing the ASR (Automatic Speech Recognition) toward dental terms. An ambiguous audio clip might get transcribed as 'crown' instead of 'cooling,' leading the model to answer a question that wasn't asked. Additionally, post-call analysis runs after the hangup, so the client only sees the error when their analytics dashboard reports zero qualified leads for a perfectly functioning agent.

Deploying Configuration, Not Just Prompts

Baghoor argues that voice agents are deployable artifacts with 42+ top-level fields, and the prompt is just one of them. He now treats publishing as a full deploy, sorting changes by 'blast radius.' High-risk fields like keywords, analysis schemas, and model temperatures require strict regression testing that checks downstream effectsβ€”such as verifying that qualified returns true and appointment_type matches HVAC values. He also implemented a JSON diff script to flatten and compare every field between exports, ensuring that cloning an agent doesn't carry over the previous client's vertical-specific baggage.

Key Takeaways

  • Clone Carefully: Cloning voice agents often carries over hidden config like boosted_keywords and analysis_fields that aren't visible in the prompt.
  • Diff Everything: Use scripts to flatten and diff full JSON exports before publishing, not just the prompt text.
  • Test Downstream: Conversational tests miss post-call analysis errors; verify that dashboard metrics and CRM entries reflect the correct business vertical.
  • Version Control Matters: Empty version_title fields in Retell exports highlight the need for manual change logs to track what actually changed between versions.

The Bottom Line

Prompts are not the product; the entire configuration object is. If you aren't diffing the full JSON export before every publish, you are deploying invisible bugs that corrupt your data before they ever hit the transcript.