Microsoft has launched Developer Agent Experience (DevAX), a framework designed to move AI agent integration from anecdotal folklore to empirical science. The core premise is blunt: every current best practice for coding agents is a guess until you measure its impact on outcome quality and task cost. DevAX provides the methodology to identify what actually helps agents build with your technology and what simply drains budgets without improving correctness.

The Cost of Assumptions

The framework highlights how intuitive changes can backfire spectacularly. One case study reveals that passing CLI arguments as a single JSON payload—a move many developers assume is 'agent-friendly'—resulted in a 4–11× cost increase per task with zero improvement in correctness. Conversely, structured JSON output for CLI results provided a measurable lift, raising configuration checks from 72 to 85 out of 85. These findings challenge the prevailing narrative that simply adding more structure or 'smart' inputs automatically yields better agent performance.

Model Selection Is Not Linear

DevAX data dismantles the idea that the smartest model is always the right choice. Upgrading from Claude Sonnet 4.6 to GPT-6 Astra yielded only a +1 point improvement in correctness but incurred a 5.6× increase in cost per run. In another scenario, switching to a newer model with 33% cheaper tokens resulted in a 3.7× higher total cost per run due to inefficiencies. The framework argues that model selection must be optimized for specific task profiles, not just benchmark scores.

Practical Implementation and Tools

To facilitate this shift, Microsoft released the AX Practitioner Playbook, available as an agent skill via npx skills add microsoft/scope/docs/ax-playbook/ax-practitioner. The playbook guides developers through defining work scenarios, establishing baselines, and measuring gaps. A worked example involving a SharePoint Framework (SPFx) upgrade demonstrated that fixing documentation—specifically warning agents against manual package.json edits—raised configuration checks from 30/80 to 83/85 without shipping new code. This proves that often, the agent’s failure is a documentation failure, not a model limitation.

Key Takeaways

  • JSON CLI arguments increase costs by 4–11× without improving correctness.
  • Structured JSON output for CLI results significantly lifts configuration check pass rates.
  • Newer models are not always better; GPT-6 Astra cost 5.6× more for a +1 point gain.
  • Documentation fixes can yield higher ROI than model upgrades or MCP server integrations.
  • The AX Practitioner Playbook is available as an installable agent skill for immediate use.

The Bottom Line

Stop guessing. If you aren't measuring the cost-per-task and correctness lift of your agent interventions, you're just burning tokens on vibes.