A new open-source project called Rubi is tackling one of the most persistent economic problems in AI-assisted coding: the quadratic cost of context. Developed by YURI BEDRAN and recently posted to Hacker News, Rubi acts as a knowledge graph and context manager for AI coding agents like Claude Code, Codex, and Cursor. The tool addresses the reality that AI assistants are billed for every token they read, and in long conversations, they re-read the entire history on every message. By analyzing 284 real Claude Code sessions, the creators found that 71% of costs were spent re-reading context, with 97% of expenses coming from sessions that grew past 150k tokens.

The Quadratic Cost of Memory

The core insight behind Rubi is that language models do not have persistent memory between messages; they rely on the context window. Every time a developer sends a request, the entire conversation history, including previous file reads and command outputs, is resent to the model. This means the cost of a session is not linear but grows with the square of the session length. While Anthropic’s prompt caching reduces the cost of re-reading the beginning of the context to about 10% of the normal input price, it does not eliminate the volume. In the analyzed data, the cache hit rate was 99%, yet the sheer volume of tokens being processed still dominated the bill. Rubi argues that Claude was not doing more work; it was carrying more weight.

How Rubi Works: Compaction and Knowledge Graphs

Rubi attacks the cost problem from two sides: session management and knowledge retrieval. First, it enforces early compaction. Instead of waiting for the context to hit the model’s limit, Rubi compacts sessions when they reach 150k tokens, saving a 'handoff' document before compaction and restoring it immediately after. This strategy alone was estimated to reduce total costs by approximately 37% when replayed on real session data. Second, Rubi builds a temporal knowledge graph of the project, mapping files, decisions, bugs, and concepts. This graph is written by Claude itself at the moment it holds that knowledge in context, ensuring accuracy without requiring a separate embedding model. The search mechanism uses a hybrid of BM25, Personalized PageRank, and Reciprocal Rank Fusion to recall relevant knowledge within a strict token budget.

Benchmark Results and Limitations

The project includes two distinct studies. The first, a retrospective analysis of 284 sessions, showed that a 150k token compaction threshold was the 'sweet spot,' offering roughly 37% savings. The second study, a controlled A/B benchmark with 90 sessions on public repositories like Flask and Hono, measured the impact of reading from a mature knowledge graph. In this test, sessions using Rubi with a mature graph required 37% fewer steps and re-read 37% less context, finishing 17% faster. However, the direct billed cost reduction in these short, cold-start sessions was only 6%, because the fixed cost of writing the cache remained high. The authors project that in real-world, long-running sessions where cache reads dominate costs, the savings would approach 26%.

Key Takeaways

  • Rubi reduces the quadratic cost of long AI sessions by enforcing early compaction at 150k tokens, simulating a ~37% cost saving on real data.
  • The tool uses a temporal knowledge graph stored in plain markdown (.rubi/) to help agents navigate projects without re-reading irrelevant files.
  • In controlled benchmarks, using a mature graph reduced context re-reads by 37% and wall-clock time by 17%, though immediate bill savings were lower in short sessions.
  • Writing memory too frequently (after every answer) increased costs by 15%, leading the developers to default to less frequent harvesting.
  • The system is compatible with multiple agents (Claude, Codex, Cursor, etc.) via a standard MCP server and runs locally with zero dependencies.

The Bottom Line

Rubi proves that the biggest lever for AI coding costs isn't a cheaper model, but smarter context management. By treating memory as a structured graph rather than a linear transcript, it offers a pragmatic path to sustainable agent usage.