The dream of a fully local, privacy-first memory layer for Claude Code remains just out of reach on consumer hardware. Developer Avinash Jetwani recently tested whether open-source decision models running on Ollaya could replace TypeSafe’s Jev API in the jevmem tool. The short answer is no. While the integration was seamless—requiring only a URL and key swap in the config—the performance gap is currently insurmountable for real-time coding workflows.

The Speed vs. Accuracy Tradeoff

Jetwani’s tests, conducted on an M4 Mac with 16 GB of RAM using Ollaya 0.9.0, revealed a stark divide between two local models. The 'winnow:e4b' model (8 GB) achieved near-perfect accuracy, correctly identifying what to save in 60 out of 66 held-out turns, matching Jev’s 65/66 closely. However, this accuracy came at a brutal cost: a median latency of 28.5 seconds per turn, compared to Jev’s lightning-fast 0.23 seconds. Conversely, the 'laya:typed-decisions' model (852 MB) was faster at 13.6 seconds but failed catastrophically on accuracy, scoring only 19/66.

Recall and Guardrails Fall Short

The local models also struggled with the retrieval and safety layers of the memory system. For recall tasks, neither model could respond within jevmem’s strict 1-second budget, forcing a fallback to basic word matching. This resulted in a recall rate of 48/78 for the local model, significantly trailing Jev’s 73/78. Furthermore, the guardrail mechanism, designed to catch rule breaks, was unreliable. The 'laya' model caught only 43 of 68 rule breaks while generating 13 false positives, whereas Jev caught 66 breaks with just 3 false positives.

Why Hardware Limits the Dream

The core issue isn't necessarily the model architecture, but the hardware constraints. The 'winnow:e4b' model requires a client timeout of 180 seconds, a non-starter for interactive development. Jetwani notes that while larger open models score near Jev on server-grade hardware (via JevBench), the consumer laptop environment simply cannot sustain the necessary throughput. The 'laya' model’s 1,024-token context window also proved too small to handle the 18-line memory files effectively, leading to information loss.

Key Takeaways

  • Local integration is easy: Ollaya models connect to jevmem with zero code changes, only configuration updates.
  • Latency is the killer: The most accurate local model is over 100x slower than the Jev API.
  • Accuracy varies wildly: Small models are fast but wrong; large models are accurate but too slow for real-time use.
  • Recall fails: Local models cannot meet the 1-second response budget for retrieval, forcing degraded fallback methods.

The Bottom Line

We are still living in the era of API-dependent intelligence for high-frequency developer tools. Until consumer hardware catches up to the inference demands of accurate, fast decision models, Jev remains the only viable option for jevmem.