Voice agents have long been bloated by the heavy dependencies of PyTorch, but a new project called Macvoice is proving you can run a full local loop on Apple Silicon for a fraction of the usual footprint. Developed by johnrobinsn, the tool compresses the entire voice pipeline into just 1.4 GB of physical memory, including GPU overhead, while delivering a first-word latency of 0.6 to 0.8 seconds when paired with a local Qwen3-4B model.

The Architecture: Ditching the Torch

The core innovation here is the complete removal of PyTorch from the inference path. Macvoice utilizes ONNX Runtime for CPU-based tasks like Silero VAD v6 for voice activity detection and Smart Turn v3.2 for end-of-turn detection. For the heavy lifting, it leverages MLX for speech-to-text using a quantized Parakeet TDT 0.6B v2 model, and a Rust-based Phonon TTS engine running on Metal for text-to-speech. This modular approach allows the system to start speaking approximately 350 milliseconds after the user stops talking, significantly reducing the perceived lag.

Handling Interruptions with Apple's Native Tech

A major pain point for local voice agents is self-interruption, where the agent hears its own output. Macvoice solves this by integrating Apple's native voice processing echo canceller, the same technology used in FaceTime and Chrome. By routing the microphone and speaker through a single AVAudioEngine, the system reduces the agent's own voice at the mic from -21 dBFS to -78 dBFS on an M5 MacBook Pro. This allows users to interrupt the agent naturally; the system detects speech for 250 milliseconds before stopping playback, preventing accidental cuts from coughs or background noise.

Flexible Brain Integration and Setup

The voice pipeline is agnostic to the LLM powering the conversation. It defaults to an OpenAI-compatible server on localhost:8080, but users can plug in Ollama, LM Studio, or even hosted APIs. For those wanting to go fully offline, the quick start guide recommends running a local Qwen3-4B instance via mlx-lm. Additionally, the system supports the Agent Client Protocol (ACP), allowing integration with coding agents like Claude Code. Tool permissions are declined by default for safety, requiring a specific flag to enable autonomous actions.

Key Takeaways

  • Low Footprint: The entire voice stack consumes only 1.4 GB of RAM on an M5 MacBook Pro, with model downloads totaling just 0.85 GB.
  • No PyTorch: By using MLX, ONNX Runtime, and Rust, the project bypasses the massive dependency bloat of standard Python ML stacks.
  • Native Echo Cancellation: Leverages Apple's AVAudioEngine to achieve -78 dBFS self-voice suppression, enabling reliable barge-in functionality.
  • Modular Brains: Supports any OpenAI-compatible endpoint, local MLX models, or ACP agents like Claude Code.

The Bottom Line

Macvoice is a masterclass in efficient local AI. By stripping away PyTorch and leveraging Apple's native audio hardware, it delivers a responsive, interruptible voice agent that actually feels like a feature of the OS rather than a bulky third-party app. This is the future of local agents: lean, fast, and integrated.

Benchmarking and Testing

The repository includes comprehensive benchmarking tools to compare the new MLX/ONNX stack against a PyTorch baseline using Whisper large-v3 and Kokoro. Unit tests cover critical logic like the endpointer and echo guards, while device tests allow users to verify real-world echo cancellation performance. The project is licensed under Apache-2.0, with model licenses varying, and requires only uv and an Apple Silicon Mac to get started.