Microsoft Developer Relations has officially launched Scope, an open-source platform built on Kubernetes designed to evaluate how AI coding agents interact with software products. Announced on October 19, 2026, Scope aims to solve the emerging challenge of Developer Agent Experience (DevAx), which goes beyond traditional Developer Experience (DevEx) by focusing on how agents discover APIs, select SDKs, and follow documentation. The platform is available under the MIT License, signaling a shift toward community-driven standards for agent evaluation.

The Problem With Traditional DevEx

Standard development practices like user testing and interviews fail to capture the nuances of agent behavior. An experience that is intuitive for a human developer may cause an AI agent to choose the wrong API or get stuck due to ambiguous instructions. Microsoft argues that single continuous integration tests are insufficient because results vary based on models, agent runtimes, skills, plugins, and Model Context Protocol (MCP) servers. Scope addresses this by creating a continuous learning loop where teams can define representative tasks, inspect agent interactions, and iteratively improve product guidance.

Real-World Impact With Azure Cosmos DB

The Azure Cosmos DB team utilized Scope to compare bare models against their Agent Kit across eight end-to-end scenarios in five languages. This evaluation process identified 20 Agent Kit issues and 27 documentation issues. According to Jay Gordon, Senior Program Manager at Azure Cosmos DB, a targeted update resulting from these findings increased hierarchical partition-key adoption from 47% to 100% and reduced OFFSET/LIMIT usage from 47% to 0%. These metrics demonstrate how evidence-based evaluation can directly influence product improvements and agent success rates.

Technical Architecture and Integration

Scope is not a hosted service but software that teams deploy themselves, allowing for the evaluation of private workloads without publishing code or run data. The platform supports existing coding agents, including GitHub Copilot, Claude Code, and Visual Studio Code with Copilot. It uses reusable, versioned profiles to capture configurations such as the worker, model, and agent version. Evaluation criteria are represented as a directed acyclic graph (DAG), enabling complex dependencies like requiring a successful build before running a runtime check. A Judge service uses LLM-based evaluation to assess outputs against these criteria.

The Bottom Line

Microsoft is betting that open-source infrastructure will become the backbone for validating AI agent utility. By releasing Scope, they are handing developers the tools to turn agent behavior from a black box into actionable product feedback.

Key Takeaways

  • Scope is an MIT-licensed, Kubernetes-based platform for evaluating AI coding agents.
  • It introduces the concept of Developer Agent Experience (DevAx) alongside traditional DevEx.
  • Azure Cosmos DB saw hierarchical partition-key adoption rise from 47% to 100% after using Scope.
  • The tool supports GitHub Copilot, Claude Code, and VS Code, using LLMs as judges for evaluation.