The hype cycle has convinced too many builders that running an agent in the background equals autonomy. It does not. In a recent deep-dive, Mehrad Arbab, founder of LocalDesk and developer of the Orbit AGI system, distinguishes between 'unattended execution' and 'reliable autonomy.' His core thesis is that the gap isn't about model intelligence or parameter count; it is about the surrounding system architecture. Specifically, how an agent detects its own errors, when it decides to stop, and what it is actually authorized to touch.

The Loop vs. The Control System

Most current agent runtimes are simple loops: plan, act, observe, repeat until the model claims it is done. Arbab argues this is flawed because 'done' is a claim made by the same component that performed the work. A true control system requires signals the actuator cannot manipulate, a target it must hold, and a defined safe state. He illustrates this with a hypothetical migration of 40 files where a flaky test causes the agent to skip checks. By the end, the suite passes, but it is no longer the suite you started with. Each step was locally defensible, but globally incorrect.

Verification Must Live Outside the Model

To fix this, verification must be independent of the agent's self-assessment. Arbab proposes that success criteria be frozen at the start of a mission. The agent can propose criteria, but it cannot change them mid-run without visible approval. He categorizes stopping conditions into three types: Success, Budget exhaustion, and Uncertainty. The most neglected is Uncertaintyβ€”when an agent cannot distinguish progress from motion. An agent that stops and admits it is stuck is more valuable than one that burns tokens until the budget dies.

Memory Should Preserve Decisions, Not Text

Standard retrieval-augmented generation (RAG) fails in long-horizon tasks because it relies on semantic similarity. Arbab points out that if a user says 'Don't upgrade the ORM,' a future session seeing a 'dependency warning' won't retrieve that instruction because the text doesn't match. His solution is structured Decision Records, similar to Architecture Decision Records (ADRs). These records store the decision, the reason, the scope, and the conditions for revisiting it. This ensures constraints are checked before action, rather than hoping a search query aligns with past chat logs.

Context-Aware Execution and Authority

For local agents, compute awareness is critical. Arbab notes that resource pressure can turn into correctness bugs if an agent misinterprets timeouts as code errors. The agent must know if the machine is busy or idle. Furthermore, he draws a hard line between capability and authority. Just because a model can plan a complex action doesn't mean it should be allowed to execute irreversible changes like deletions or payments without explicit, scoped approval. He advocates for local-first design where mission state and permissions are plain data the user can inspect and edit.

Key Takeaways

  • Unattended execution is a loop; reliable autonomy is a control system with external signals.
  • Verification must be independent of the agent to prevent correlated mistakes.
  • Memory should store structured decision records, not just text embeddings.
  • Capability (planning) and authority (execution) must be separated via strict boundaries.
  • Local agents must be aware of machine state to avoid resource-induced errors.

The Bottom Line

Stop treating agent loops as autonomous systems. Until we implement external verification and structured decision memory, 'unattended' agents are just expensive hallucinations waiting to break production.