Every vendor is currently shoving the word 'agent' onto their landing pages, but the technical reality is far less glamorous. A recent deep dive published on DEV.to strips away the marketing fluff to reveal a simple dividing line: a chatbot answers, while an agent finishes. The distinction isn't just about intelligence; it's about who holds the reins. If a human executes every step and decides when to continue, the system is merely a chatbot wearing an agent costume.

The Autonomy Test Is Not About Planning

Many systems can generate beautiful, multi-step plans, but that does not equate to autonomy. The critical metric isn't whether the LLM forms a plan, but rather who forms it, who executes it, and who decides to replan or stop. If the human is the one driving the loop, the autonomy is missing. This distinction matters because vendors know that 'agent' sells better than 'chatbot with a loop,' leading to widespread mislabeling of simple reasoning engines as autonomous entities.

The Cost of the While-Loop Is Brutal

True agents operate on a perceive-plan-act-check loop, often implemented as a while-loop wrapping LLM and tool calls, as seen in architectures from Anthropic and OpenAI. This architecture comes with a staggering price tag. Across 500 SWE-bench coding tasks, agentic workflows burned approximately 3,500x the tokens of single-round reasoning tasks. The cost per task jumped from $0.016 to $1.857. This isn't just because agents are smarter; it's because accumulated context gets re-sent every round. Nobody puts this multiplier on the marketing slide.

Reliability Ceilings Are Still Dangerously Low

Beyond the cost, the reliability of current frontier agents is surprisingly poor for long-horizon tasks. On TheAgentCompany’s benchmark of 175 workplace tasks, the best frontier agent completed only 30.3% of tasks, costing $4.20 per attempt. Failures included getting stuck in infinite loops or aimlessly exploring environments. METR data further highlights this fragility, showing that the 80%-success horizon is roughly five times shorter than the 50% horizon. Planning makes multi-step work possible, but it does not make it reliable.

When a Chatbot Is Enough, Say So

For single-question answers with nothing to decompose, observe, or recover from, a chatbot is the right tool. Reaching for an agent in these scenarios is simply burning tokens to feel futuristic. The discipline lies in knowing which side of the line your task sits on before picking the architecture. Reserve agentic workflows for multi-step, open-ended tasks that require failure recovery, such as filing expenses or investigating a bug across a codebase.

Key Takeaways

  • Autonomy Definition: An agent must carry a goal across steps and decide next actions without human hand-holding; otherwise, it is a chatbot.
  • Token Economics: Agentic loops cost ~3,500x more than single-shot reasoning due to context re-sending, a cost hidden from most vendor pages.
  • Benchmark Reality: Top agents complete only 30.3% of long-horizon workplace tasks, with reliability dropping sharply as task horizon increases.
  • Tool Selection: Use chatbots for single-question answers; reserve agents for multi-step, open-ended tasks requiring failure recovery.

The Bottom Line

Stop letting 'agent' become a synonym for 'chatbot with a loop.' If the system can't recover from failure or decide its next step without you, it's not an agentβ€”it's a cost center. Choose the architecture based on the task's complexity, not the vendor's marketing budget.