Skymizer has introduced the HTX301, a reference chip on its HyperThought platform designed to execute ultra-large language models on a single PCIe card. The hardware boasts 384GB of on-card memory and targets 700B-parameter LLM inference with a total board power of approximately 240W. This architecture aims to eliminate the need for complex GPU clusters, NVLink interconnects, and liquid cooling systems typically required for models of this magnitude.

Architecture and Performance Claims

The HTX301 utilizes a decode-first silicon design with the LISA instruction set architecture, disaggregating prefill and decode workloads to optimize efficiency. By scaling from 4B to 700B parameters on the same architecture, Skymizer argues that enterprises can provision for specific model sizes without the waste of over-provisioning for worst-case scenarios. The card is built on a mature T28nm process node, signaling a focus on cost-effective, deterministic performance rather than raw compute density found in cutting-edge nodes.

On-Premise Sovereignty and Enterprise Focus

William Wei, Skymizer's CMO, stated that "the era of needing superscalar GPU clusters for ultra-large LLMs is over," positioning the HTX301 as a solution for data sovereignty. The device ensures that inference never leaves the customer's premises, keeping weights, prompts, and data entirely in-house. This approach specifically targets enterprise and agentic AI workloads where predictable cost and privacy are paramount, avoiding the "cluster tax" associated with multi-node orchestration and complex infrastructure.

Early Access and Availability

Currently in early access, the HTX301 is described as a reference platform, with specifications subject to change before general availability. Interested parties can request evaluation details and specifications through Skymizer's registration portal. The company emphasizes that the HTX301 is not just a hardware drop-in but a rethinking of LLM deployment, prioritizing on-prem simplicity and power efficiency over the traditional cluster-heavy model.

Key Takeaways

  • HTX301 supports 700B-parameter models on a single PCIe card with 384GB memory.
  • Power consumption is claimed to be around 240W, significantly lower than GPU clusters.
  • Uses LISA ISA with decode-first architecture to scale from 4B to 700B parameters.
  • Targets on-prem enterprise use cases with strict data sovereignty requirements.
  • Currently in early access; specs are subject to change before general availability.

The Bottom Line

Skymizer's HTX301 represents a pragmatic challenge to the prevailing dogma that larger models inherently require larger, more complex GPU clusters. If the performance claims hold up in real-world enterprise environments, this could significantly lower the barrier to entry for on-premises AI inference by prioritizing efficiency and sovereignty over raw compute density.