Developer s3onghyun has released agent-otel-kit, a local OpenTelemetry stack designed to bring unified observability to three of the most popular AI coding agents: Claude Code, OpenAI Codex, and Gemini CLI. The tool addresses a critical pain point for engineers running these agents locally: the lack of transparent, granular data on token usage, costs, and tool execution traces. By leveraging a standard OTLP collector, Prometheus, Loki, and Tempo, the kit provides real-time dashboards for monitoring agent behavior without sending sensitive data to external vendors.
The Stack and Setup
The architecture is built on a Docker Compose stack that includes an OpenTelemetry Collector to receive OTLP telemetry, Prometheus for metrics, Loki for logs, and Tempo for traces. A small Go CLI handles the configuration, pointing each agent to the local collector and verifying data flow. The setup is designed for speed, claiming a 30-second quickstart process. It supports installation via Homebrew, prebuilt archives for Linux and macOS, or Go install. Crucially, the tool respects privacy by default, stripping out prompt text, model responses, and file paths unless explicitly opted in via environment variables.
Handling Agent Quirks and Data Normalization
One of the kit's most impressive features is its handling of the disparate telemetry formats from different agents. Claude Code, Codex, and Gemini CLI export counters with delta temporality, which Prometheus typically misreads. The collector uses a delta_to_cumulative processor to normalize this data. Furthermore, the tool accounts for specific agent behaviors: Codex lacks a cost metric, and Gemini CLI logs prompts by default, which the setup script disables to prevent data leakage. The dashboards normalize labels to a consistent agent_name field, allowing for cross-agent comparison of tool calls, success rates, and token usage.
Key Takeaways
- Local-first privacy: All data stays on the user's machine; external data transmission is disabled by default.
- Unified view: Provides consistent dashboards for Claude Code 2.1.325, Codex 0.169.0, and Gemini CLI 0.68.0.
- Privacy by default: Prompt text, file paths, and shell commands are redacted unless AGENT_OTEL_KIT_CAPTURE_CONTENT is set.
- Easy integration: A single Go CLI configures the agents' settings files to point to the local OTLP collector.
The Bottom Line
As AI agents become more autonomous, blind execution is no longer acceptable. Agent-otel-kit provides the essential telemetry needed to debug, optimize, and audit these tools locally, making it a must-have for serious AI engineers.