CodeNinja Atoms has released an open reference architecture for a self-improving AI system designed to manage canal water allocation in Pakistan. The project, detailed in a new engineering summary by Muhammad Umar, focuses on a provincial irrigation department that struggles with fragmented data across telemetry systems, paper ledgers, and field diaries. The core promise is a sovereign infrastructure that operates entirely on hardware owned by the local operator, avoiding third-party model services and ensuring data never leaves the department's network.
Sovereign Infrastructure and Data Sovereignty
The architecture explicitly rejects edge serving at canal heads and cloud-based model inference. Instead, all processing occurs in the department's circle server room using two specific hardware tiers: a 48 GB PCIe class site server for embedding, OCR, and forecasting models, and a frontier-class node equipped with eight 141 GB HBM cards. The system is designed to run GLM 5.3 in FP8 precision, which fits within the aggregate memory of 1,128 GB. This setup addresses export control concerns, as the 141 GB class sits above the US advanced computing threshold, requiring a specific license determination before procurement.
The Three-Layer Design Pattern
The system follows a strict three-layer pattern: systems of record at the bottom, a governed object model in the middle, and applications at the top. Crucially, the platform acts only as a projection over existing records, such as Apache Kafka 4.3.x telemetry streams and TimescaleDB time-series data. It never writes to the legal records or operates physical gates; those remain manual tasks guided by telephone instructions. Data enters through two adapter families: a direct historian read for telemetry and an OCR pipeline for paper sources, which includes a 2 percent failure gate for human verification to prevent corrupting entitlement baselines.
Human-in-the-Loop Validation
Six services carry the workload, but the final decision always rests with named engineers. Three agents draft causes for shortfalls, indent requests, and dispute accounts, but controlling engineers must confirm or reject these drafts before any action is taken. The system uses Hyper Ontology v0.9 to link sixteen objects, allowing for complex traversals like identifying outlets short of share for two consecutive rotations. This approach ensures that the AI provides context and provenance rather than autonomous control, keeping the memory of corrections within the department even as staff rotates.
Cost and Implementation Strategy
The project prices the conditional shape at approximately $636,000 USD for three years of ownership, compared to roughly $773,000 for renting equivalent AWS resources on a three-year commitment. However, renting is deemed impractical because AWS has no region in Pakistan, which would force telemetry and ledger data across borders. The rollout is divided into two phases, with the first focusing on one main canal for a single cropping season. The gate for phase one is not a date but a technical reconciliation: the ontology must successfully match one real week of telemetry against ledgers before agents are allowed to draft any instructions.
Key Takeaways
- The architecture uses local hardware (8x 141 GB HBM cards) to run GLM 5.3, avoiding cloud dependency and data export issues.
- Human engineers act as the final gatekeepers for all AI-drafted actions, ensuring no autonomous gate operations occur.
- Data integrity is maintained through a 2 percent human verification gate on OCR inputs from paper ledgers.
- The total three-year ownership cost is estimated at $636,000, which is about 82% of the cost of a three-year cloud commitment and less than a third of on-demand cloud pricing.
The Bottom Line
This is a rare example of AI infrastructure that prioritizes regulatory and physical reality over hype. By keeping the model local and the human in the loop, CodeNinja has built a system that respects the constraints of government procurement and physical safety.