Cerebras Systems has published a granular technical breakdown of how its Wafer-Scale Engine 3 (WSE-3T) infrastructure handles the deployment of frontier models exceeding one trillion parameters. Moving beyond marketing hype, the engineering blog post uses the open-source Kimi K3 model, which boasts approximately 3 trillion parameters, to illustrate the memory partitioning and pipelining strategies required to achieve inference speeds of 1,000 to 2,000 tokens per second. This disclosure offers a rare look into the specific architectural constraints and solutions involved in running massive LLMs on non-GPU silicon.
The Memory Partitioning Strategy
The core challenge with trillion-parameter models is fitting weights into memory while maintaining low latency. A single WSE-3T wafer contains 44 GB of on-chip SRAM. For Kimi K3, which consists of 69 linear-attention layers and 24 dense layers, the raw parameter payload at 4-bit quantization requires significant distribution. The Cerebras compiler maps one dense layer per wafer to reserve remaining SRAM for the Key-Value (KV) cache, while fitting two linear-attention layers per wafer since they require only fixed-size state storage. This specific mapping results in a total requirement of 59 wafers to fully host the model weights and necessary cache structures.
Pipeline Architecture and Latency
Once the model is distributed across the wafer cluster, Cerebras employs a pipeline execution model where activationsβnot weightsβtravel between wafers. The system utilizes RDMA over Ethernet to pass intermediate results between stages, achieving a latency of roughly 2 microseconds per hop. Because the model weights remain static in on-chip SRAM, the system avoids the massive memory bandwidth bottlenecks typical of GPU-based inference, where weights often need to be fetched from HBM. This static weight approach allows the pipeline to function like an assembly line, with each wafer processing a specific segment of the model's layers.
Concurrency and Throughput Mechanics
A common misconception is that pipeline parallelism results in idle hardware while waiting for a single token to traverse all stages. Cerebras counters this by maintaining a full pipeline with multiple concurrent requests. While one wafer processes a token for Request A, the preceding wafer handles Request B, and the subsequent wafer handles Request C. The maximum number of concurrent requests is strictly capped by the available on-chip memory for the KV cache in dense layers. For a 256k context window, the remaining SRAM after allocating space for dense layer weights determines the ceiling for simultaneous sessions, allowing the system to stream requests continuously rather than batching them in a GPU-like fashion.
Scaling Beyond 3 Trillion Parameters
The architecture described is not limited to 3T parameter models. Cerebras notes that the same approach extends to larger models by calculating the memory needed for weights and request state, distributing the model across wafers, and connecting them into a pipeline. Each additional wafer contributes both compute and on-chip memory, allowing the infrastructure to accommodate models that exceed the capacity of any single chip. This modular scalability ensures that as frontier models grow in size, the wafer-scale architecture can expand linearly to meet the new memory and compute demands without fundamental redesign.
Key Takeaways
- Cerebras requires 59 WSE-3T wafers to deploy a 3T parameter model using 4-bit quantization.
- Dense layers are isolated on single wafers to preserve SRAM for KV cache, while linear layers are paired.
- Inter-wafer communication uses RDMA over Ethernet with ~2 microsecond latency per hop.
- Concurrency is limited by on-chip KV cache capacity, not by compute throughput or batch size delays.
The Bottom Line
Cerebras is effectively rewriting the rules of inference economics by proving that wafer-scale integration can bypass the memory bandwidth ceiling that plagues GPU clusters. While the upfront capital expenditure for a 59-wafer cluster is substantial, the elimination of weight-fetching bottlenecks and the ability to stream requests continuously suggest a superior path for serving massive, high-context models at scale.