Building a local RAG stack usually means spending an afternoon wrestling with systemd units, firewall rules, and SSH keys. Conn3x just cut that down to a single apply command. Their new example repository uses Terraform-style state management to provision a DigitalOcean droplet, install Ollama and Qdrant, and lock down the network in one go.

The Architecture: Loopback Only

The setup deploys an Ubuntu 24.04 droplet (s-2vcpu-4gb) in the London region by default. The critical design choice here is isolation: neither Ollama nor Qdrant listens on a public interface. Both services bind strictly to 127.0.0.1. Access is granted exclusively through an SSH tunnel established via conn3x output -raw tunnel. This means your data never touches the public internet, and neither does your model API, which lacks native authentication. The infrastructure code applies five distinct circuits. circuit-baseline handles time synchronization and key-only SSH. circuit-hardening tightens kernel parameters and SSH config. circuit-firewall deploys ufw to deny all incoming traffic except SSH. Finally, the Ollama and Qdrant circuits install the software and configure their respective ports (11434 and 6333). The Qdrant API key is generated randomly via a random_password resource, eliminating the human error of weak secrets.

Security and Cost Breakdown

The team tested the setup thoroughly. From an external IP, ports 11434, 6333, and 6334 remained unresponsive. A request to Qdrant without the API key returned a 401 error. The SSH configuration explicitly sets AllowTcpForwarding = "local", which permits ssh -L tunnels but blocks ssh -R reverse tunnels. This prevents the box from being used as a proxy to publish internal ports to the internet. The entire stack costs approximately $24 per month at DigitalOcean’s October 2026 pricing, with the cloud firewall itself being free. The initial apply takes about two and a half minutes. This delay is due to downloading the 1.5 GB Ollama release (which bundles CUDA libraries even for CPU use) and two models: nomic-embed-text and qwen2.5:0.5b. The total model size is around 670 MB. The setup was verified to survive a reboot, with all services and firewall rules persisting correctly. Teardown is equally automated: conn3x destroy removes the droplet, firewall, and SSH keys in under a minute.

Key Takeaways

  • One state file manages the droplet, firewall, and software, ensuring they are created and destroyed together.
  • Ollama has no built-in auth, so binding it to localhost and tunneling via SSH is essential for security.
  • The example uses nomic-embed-text for vectors and qwen2.5:0.5b for generation, both running comfortably on 4GB RAM.
  • conn3x v0.6.7 or later is required to run this automation.

The Bottom Line

This is a pragmatic solution for developers who want the convenience of cloud compute without the privacy trade-off of hosted AI APIs. By treating the entire AI stack as infrastructure code, Conn3x removes the manual toil of hardening a server for private LLM use. If you are already using Terraform or similar tools, this pattern is worth stealing for your own private RAG deployments.