Tomer Yudovic has released Poinky, a new open-source interface for Claude Code that integrates directly into the macOS notch. Unlike standard terminal-based agents, Poinky listens to your voice, analyzes your screen content, and responds by drawing explanations or pointing to specific UI elements. The tool effectively turns the Mac's system UI into a canvas for a conversational AI assistant.

How It Works

The interaction model is distinct from text-only prompts. Users hold Control and Option to speak, triggering on-device speech-to-text conversion. Once the query is processed by the LLM, Poinky uses a yellow pen animation to sketch step-by-step guides or spin 3D models on the screen. It can also perform actions, such as typing emails or filling forms, while requesting confirmation before sending data.

Technical Requirements and Setup

Poinky requires macOS 26 or later and runs exclusively on Apple Silicon, though Intel Macs are reportedly supported but untested. The installation process is handled via a single bash command that builds the application locally, avoiding the need for Apple's paid developer notarization. This local build approach also prevents macOS from displaying standard 'downloaded from the internet' security warnings. The tool relies on Claude Code or an Anthropic API key, with a ~350 MB natural voice model downloaded separately.

Privacy and Future Vision

Yudovic emphasizes a privacy-first architecture, stating that voice processing occurs entirely on the device and screenshots are only sent to Claude when explicitly requested. There are no user accounts or tracking mechanisms involved. Looking ahead, the roadmap includes persistent background agents that visually detach from the cursor to perform complex tasks in separate screen regions, providing live progress updates upon completion.

Key Takeaways

  • Poinky integrates with the macOS notch and uses voice commands for interaction.
  • The tool visualizes AI responses through on-screen drawings and cursor movements.
  • Installation requires macOS 26+ and builds locally to bypass notarization.
  • Voice processing is local; only screen data is sent to Anthropic's API.
  • The project is MIT licensed and currently in early release with zero comments on HN.

The Bottom Line

Poinky represents a tangible shift toward multimodal, ambient AI interfaces, moving beyond chat windows into direct OS integration.