Attest 1.1.0 has hit the scene as a testing framework for AI agents in Python and TypeScript, shifting the focus from what an agent says to what it actually does. The tool integrates with standard runners like pytest and Vitest, allowing developers to verify tool call sequences and external state changes. This approach addresses a critical gap in current agent development where output-only tests fail to catch logical errors in execution paths.
The Rec-Replay-Gate Workflow
The framework operates on a three-step cycle: record, replay, and gate. Using attest capture, developers record an incident trace and model exchanges, applying redaction to draft contracts. The replay phase runs the agentβs real code against recorded model responses via a local proxy, eliminating live model variability. Finally, the gate phase blocks merges if tests fail, ensuring that only verified behavior enters the codebase.
Deterministic Checks Over LLM Judging
Attest prioritizes deterministic checks for cost and reliability, reserving LLM-as-judge scoring for criteria that cannot be expressed through code. The system offers six evaluated layers, ranging from free schema validation and constraint checks to paid embedding similarity and judge models. For example, a single line of code can assert that a refund tool is called at most once per order ID, catching double-refund bugs that text-based tests miss.
Framework Adapters and Maturity
The tool supports a wide array of providers with varying stability levels. OpenAI and Anthropic adapters are marked as stable for both Python and TypeScript, while Gemini, Ollama, and LangChain adapters are currently in beta. Multi-agent testing and simulation mode are also in beta, allowing users to mock tools and check cross-agent data flow. Continuous evaluation features remain experimental, offering alerting on score drift through webhooks.
Key Takeaways
- Attest 1.1.0 enables strict replay testing that removes live model variability from CI pipelines.
- The framework distinguishes between free deterministic checks and paid LLM-judged layers to optimize costs.
- Stable adapters exist for major providers like OpenAI and Anthropic, with beta support for others.
- Multi-agent trace testing is available in beta, supporting complex delegation chains.
The Bottom Line
Attest 1.1.0 is a necessary evolution for agent development, finally providing the deterministic rigor needed to trust autonomous systems in production.