For developers building AI infrastructure in their basements and home offices, getting a single GPU to cooperate is just the beginning. The real headaches start when you try to scale across multiple cards—and that's exactly what developer jdagostino explores in the second installment of his 'AI at Home' blog series.

Why Multi-GPU Setups Go Sideways

The post, titled 'Multi-GPU Drifting,' appears on a GitHub Pages-hosted personal blog and was shared to Hacker News where it garnered minimal engagement—just two points and zero comments. But don't let the quiet reception fool you; this is exactly the kind of hands-on infrastructure pain that builders obsess over in private Slack channels and late-night Discord threads. The 'drifting' in the title likely refers to workload imbalance—when one GPU idles while another bottlenecks, or when gradient synchronization introduces lag that kills throughput. Getting two or more consumer-grade GPUs to play nice together requires wrestling with PCIe lane allocation, NVLink (if you're lucky enough to have compatible cards), and framework-level tensor parallelism or pipeline parallelism configurations.

The Hardware Reality Check

Consumer hardware wasn't designed for these workloads. A typical home lab might pair mismatched GPUs—a couple of RTX 4090s or 3090s, maybe an older card pulled from a gaming rig—with a consumer motherboard that has limited PCIe bandwidth. The result is often worse than running on a single faster GPU. Getting multi-GPU setups right typically requires careful benchmarking to find the crossover point where parallelization overhead stops eating your gains. For inference workloads especially, batching strategies and model sharding become critical—and that's before you even get into the software stack complexity of frameworks like vLLM or llama.cpp.

The Developer Experience Gap

What's frustrating about home lab AI infrastructure is how little documentation exists for non-enterprise setups. Most tutorials assume either a single GPU or access to cloud instances with properly configured multi-GPU VMs. The middle ground—actual physical hardware in a residential setting—gets glossed over.

Key Takeaways

  • Multi-GPU scaling at home requires understanding PCIe topology and memory bandwidth constraints, not just model parallelism theory
  • Consumer-grade motherboards often become the bottleneck before GPU VRAM does
  • Workload distribution strategies ('drifting') need empirical benchmarking per workload—there's no universal config that works
  • The gap between academic multi-GPU tutorials and real home lab implementations remains significant

The Bottom Line

Stories like this remind me why I keep hammering on the practical side of dev tooling. The gap between 'it works in theory' and 'it runs reliably in my spare bedroom' is where most infrastructure work actually happens—and that's exactly the kind of knowledge that doesn't show up in benchmark papers.