Most engineering teams are obsessing over the wrong metric. While per-token costs are public and precise, they often obscure the true financial impact of AI agents. A recent post from KarmaBox engineer highlights a vendor-neutral schema that tracks three specific fields—attempt_cost, review_minutes, and accepted status—to reveal where money is actually leaking from the P&L.

Why Per-Token Metrics Fail

The arithmetic of token pricing is straightforward, but it ignores the reality of agent loops. Retries, tool-call round trips, and context re-reads mean that 'unit price × task count' misses real bills by an order of magnitude. The author’s October data showed that model fees accounted for only 18% of total costs, while human time spent checking and redoing output represented 41%. Furthermore, external pricing volatility, such as Grok’s recent quota throttling, makes token-based forecasting unreliable.

The Three-Field Schema

The proposed solution logs three critical data points. First, attempt_cost captures everything burned in a job, including model fees or amortized hardware costs, metered at the client wrapper to catch swallowed retries. Second, review_minutes tracks wall-clock time humans spend verifying output, captured automatically via UI interactions to avoid manual logging errors. Third, a boolean accepted field records whether the output shipped, treating any redo as a new row to make rework visible.

Analyzing the Data for Actionable Insights

Once data is collected, two SQL queries provide clarity. The first calculates cost_per_accepted weekly by dividing total spend (including rejected attempts and review time) by the number of accepted deliverables. The second breaks down costs by agent role to identify expensive workflows. High rework rates suggest procedural issues, while high review times with low rework indicate boundary problems, such as reviewers auditing irreversible actions rather than just content quality.

Key Takeaways

  • Token costs are often a minor fraction of total agent expenses compared to human review time.
  • Automated logging of review minutes is essential, as manual tracking fails quickly.
  • Restricting agent capabilities at the system layer reduces review time more effectively than prompt engineering.
  • Self-hosting only beats cloud costs if hardware utilization is high; idle boxes are pure waste.

The Bottom Line

Stop negotiating token discounts and start measuring the cost of a finished job. If you don’t know your median review minutes, you’re flying blind on your biggest operational expense.