VIDRAFT has successfully pre-trained a language model from scratch on an NPU specifically designed for inference, marking a significant shift in AI infrastructure. The resulting model, VKORE-46M-NPU, is now available on Hugging Face with weights, model card, and comparison data, proving that real backpropagation is possible without a GPU. This achievement challenges the long-held assumption that NPUs are incapable of handling the numerical stability and gradient synchronization required for training.

The Technical Hurdle

Inference NPUs are built for the forward path, utilizing static compiled graphs and treating weights as constants. Training requires the opposite: a backward graph with saved activations, optimizer state, precise gradient handling, and gradient synchronization between chips. VIDRAFT bypassed these limitations by executing backpropagation directly on a 32-chip NPU array. The company implemented VKORE, a software layer described as CUDA for NPU, which abstracts the hardware to enable full-cycle development, including LoRA fine-tuning and quantized execution.

Evidence of Success

To validate the approach, VIDRAFT trained the identical model under identical conditions (initialization, data, and scheduling) on both the NPU and a reference GPU. The loss trajectories correlated at 0.99, indicating that the NPU’s backpropagation and optimizer updates performed the same numerical work as the GPU. Additionally, the tokens-per-second performance converged to a comparable level, demonstrating that the NPU could handle the computational demands of training without significant degradation in speed.

Why This Matters for Builders

High-end GPU servers are expensive, power-hungry, and often subject to long wait times. By enabling training on cheaper, low-power NPUs, VIDRAFT offers a path for companies to develop custom models on-premises. This reduces dependency on specific GPU ecosystems and keeps sensitive data within the company’s infrastructure. While the VKORE-46M-NPU is a character-level model trained on TinyStories, it serves as a proof of concept for scaling to larger datasets and models in the future.

Key Takeaways

  • Real backpropagation was executed on a 32-chip inference-only NPU, disproving the belief that such hardware cannot handle training gradients.
  • Loss trajectories between the NPU and a reference GPU correlated at 0.99, validating numerical equivalence in training updates.
  • VKORE acts as a hardware-agnostic software layer, enabling pretraining, LoRA tuning, and inference acceleration on a single NPU setup.
  • The VKORE-46M-NPU model is a proof-of-concept for on-premises training, offering a cost-effective alternative to GPU clusters for sensitive data.

The Bottom Line

This is a pivotal moment for hardware independence in AI. By proving that inference-only NPUs can handle the rigorous demands of backpropagation, VIDRAFT has cracked open the door for cheaper, decentralized training infrastructure. While the current model is a proof-of-concept, the implication that we no longer strictly need GPUs to train models from scratch is a game-changer for data sovereignty and cost control.