As large language models push context windows to new limits, a fundamental problem remains: the 'lost in the middle' phenomenon. Developers working on multi-day projects often find that rules established on day one vanish from the model's effective attention by day three. Addressing this, a developer named aalishan001 has released 'cockpit,' a local-first command system designed to run inside Claude Code. The project, published as the MIT-licensed 'opencockpit' repository, shifts the burden of memory from the model's context window to deterministic scripts and structured files.

The Failure of Naive Retrieval

The core argument for cockpit rests on two significant findings in LLM research. First, the 'Lost in the Middle' study (Liu et al., TACL) demonstrated that models perform best when relevant information is placed at the very beginning or end of the input, with performance degrading significantly in the middle. Second, Chroma's 'Context Rot' report tested 18 models and found consistent performance degradation as input length grew, even on simple tasks. In a long-running chat, critical decisions made early on sit in the middle of the growing context, making them prone to being ignored. Naive retrieval methods, such as similarity search, often fail because they return stale information that looks similar to the current query, confusing the model with outdated rules.

Architecture: Small Files and Git History

Cockpit mitigates these issues by rejecting the chat thread as the primary memory source. Instead, it employs four key habits visible in the repository. First, current state is maintained in small, single JSON files (e.g., data/runway/state.json) that are overwritten in place, ensuring the model always reads the most recent truth. Second, history is offloaded entirely to Git; every command that modifies files triggers a commit and push, allowing developers to audit changes via git log rather than scrolling through an endless chat. Third, append-only logs for ideas or events are folded into current state by scripts, preventing 'ledger bloat' where the model has to parse hundreds of historical entries to find the active status. Finally, skills are modular; Claude Code loads only the specific instruction file needed for a command, keeping the active context window lean.

Deterministic Checks Over Prompting

The system enforces rigor through code, not just prompt engineering. A PostToolUse hook in .claude/settings.json runs a Python validator (scripts/validate.py) after every edit, taking approximately 0.4 seconds. This ensures that arithmetic, dates, and hashes are handled by scripts, not the model's probabilistic reasoning. The developer emphasizes that 'the model makes the judgment calls. The harness does everything that must be exact.' For public-facing content, the system includes disclosure lints and voice checks that block publications if banned phrases or formatting errors are detected. This deterministic approach reduces the cognitive load on the LLM, allowing it to focus on creative and logical tasks rather than remembering static rules.

Key Takeaways

  • Context drift is a measurable failure mode in long chats, validated by 'Lost in the Middle' and 'Context Rot' research.
  • 'Cockpit' uses small, overwritten JSON state files and Git history to bypass context window limitations.
  • Deterministic scripts handle exact tasks (math, validation) to prevent LLM hallucination on factual details.
  • The system is open-source (MIT) and available at github.com/aalishan001/opencockpit.

The Bottom Line

This project is a pragmatic admission that we cannot yet trust LLMs to manage their own long-term memory in complex workflows. By wrapping the model in a deterministic harness, developers can finally get reliable results from multi-day AI-assisted coding sessions.

Try It Yourself

The repository is designed as a template. Users can clone the repo, install requirements, and run a test suite before engaging with Claude Code. It ships with a fictional example user and empty data, allowing developers to experiment with the /cockpit command and domain-specific skills like RUNWAY (for website deployment) and EXTROVERT (for content creation) before integrating their own projects.