Stop buying '24/7' as a feature label. It is marketing fluff, not a service level. A new deep-dive by Isabelle Dubuis on DEV.to argues that an AI agent answering at 02:00 often fails the customer promise simply because its knowledge is stale, its authority is undefined, or its escalation path is a black hole. Availability is a customer outcome, not a green uptime badge on a dashboard.
The Illusion of Always-On Support
The core problem is conflating endpoint reachability with resolution coverage. Dubuis breaks this down into six distinct concepts: endpoint availability, agent availability, answer availability, action availability, human coverage, and resolution coverage. A 99.9% endpoint metric means nothing if the agent gives a supported answer from outdated data or if a human queue sits unmonitored until Monday. The article warns that 'deflection' metrics are particularly dangerous if they count a frustrated customer who gives up as a success.
Defining the Coverage Contract
Before configuring any agent, businesses must write a coverage contract in plain language. This document must specify which channels accept requests, which categories the agent may answer without a person, and which must stop and show a human or emergency path. It must also define who owns queued requests while closed and what response time can actually be promised. A sentence in a prompt is not an operational control if the runtime never checks the calendar or queue.
A Synthetic 12-Case Coverage Test
To cut through the vendor noise, the piece introduces an original, synthetic acceptance set of 12 scenarios designed to test coverage boundaries. These are not live benchmarks but constructed tests to verify if an agent can safely acknowledge, resolve, or route requests. Case 2, for example, tests how an agent handles a refund request when the team is closedβit must explain the request is recorded, not approved, and create an owned review item. Case 10 checks if the agent denies unauthorized actions, like issuing credits, unless explicitly authorized by policy.
Email Agents Are Not Chatbots
Email introduces unique risks that chat interfaces often hide. An inbound email contains quoted history, forwarded text, and attachments that can leak internal notes or unrelated customer threads into a draft. Dubuis recommends restricting the read set to specific folders and requiring explicit approval before sending until the team proves it can handle sender identity and recipient lists. A simple 'reply-all' error or an auto-reply reopening a closed case can turn a helpful agent into a compliance nightmare.
Measure Outcomes, Not Theater
The article demands a shift from 'availability theater' to real outcome measurement. Dashboards must report four separate time series: endpoint success, agent processing success, supported-answer rate, and human queue completion. Crucially, the piece advises against averaging away critical failures. If eleven cases pass but an urgent safety message receives fabricated reassurance, the system is not ready. Keep the system in a narrower class rather than renaming a failure as an edge case.
Key Takeaways
- Define a coverage contract before configuration, specifying which categories the agent may answer, classify, or must stop.
- Treat '24/7' as separate service levels: automated response is not human coverage.
- Use synthetic tests to verify authority boundaries and recovery paths, not just response speed.
- Email agents require stricter provenance checks to prevent data leakage from quoted threads and attachments.
The Bottom Line
Don't let vendors sell you 'always-on' when you can only guarantee 'always-answered.' True coverage requires a contract that maps every request type to a specific, owned outcome, whether that is an instant FAQ or a ticket for Monday morning.