A new project called Lumabri has surfaced on Hacker News, offering a method to run large AI models by distributing inference workloads across multiple local computers and servers. Written in pure C, Lumabri leverages the Colibri engine to shard model layers across a private household network, allowing users to execute models that exceed the memory capacity of a single machine. The tool prioritizes local control and privacy, requiring explicit approval from each participating computer before loading any model weights.

Distributed Layer Execution

Unlike traditional multi-GPU setups that often replicate model states, Lumabri uses a 'resident Segment path' where each worker computer loads only the weights for its assigned layers. Conversation state remains with the nodes executing those specific layers, while the requesting computer holds the source checkpoint without duplicating the full model into every worker's RAM. This architecture allows users to pool memory from multiple devices, though the documentation explicitly notes that adding computers increases usable capacity but does not guarantee higher token generation speeds for each chat.

Strict Local Control and Approval

The system operates on a 'household' model where a creating computer manages the cluster, and other devices must join via a specific key. Before any inference begins, every participating donor must explicitly approve the plan, which details the layer range, RAM budget, and disk headroom required. If a donor rejects the allocation, the process halts, ensuring that no computer is forced to contribute resources without consent. This granular control extends to the network layer, with fixed TCP ports (47301โ€“47315) used for household services to avoid silent port conflicts.

Current Limitations and Certification Status

Lumabri is currently in a state where it supports specific model families, including GLM, Kimi, OLMoE, Qwen, and DeepSeek V4. However, the project is candid about its limitations: it has not yet been certified for general physical-LAN reliability, native Windows execution, or GPU household execution. While the browser interface allows for local API access and chat, it relies on loopback or explicit HTTPS reverse proxies for security, and the tool warns that WSL testing does not equate to native Windows certification. The developers emphasize that this is a smoke-tested utility for local experimentation rather than a production-ready distributed inference platform.

Key Takeaways

  • Lumabri shards LLM layers across multiple local computers to fit models that exceed single-machine RAM.
  • The tool is written in pure C and uses the Colibri engine, with strict explicit approval required from each donor computer.
  • Adding more computers increases memory capacity but does not automatically improve inference speed (tokens per second).
  • The project is not yet certified for native Windows, GPU execution, or general physical-LAN reliability.

The Bottom Line

Lumabri is a fascinating, low-level experiment in distributed local inference that prioritizes user consent and memory efficiency over raw speed, but its lack of native Windows and GPU certification keeps it firmly in the domain of dedicated tinkerers rather than general users.