A developer discovered a systemic flaw in how LLM integration libraries handle bug fixes across provider-specific code. By auditing the git history of Prism, a Laravel package for LLM providers, the author found that a crash fix for DeepSeek and OpenRouter tool maps was never propagated to Groq, Mistral, xAI, and Z. This 'sibling bug' patternβ€”where a fix is applied to one adapter but forgotten in othersβ€”led to the creation of half-fixed, a CLI tool designed to automate this historical audit.

The Sibling Problem in LLM Adapters

LLM libraries like Prism and LiteLLM rely on parallel implementations for each provider. In Prism, each provider has a ToolMap.php file that translates tool definitions into the specific JSON schema required by that API. Commit 9447c71 fixed a bug where empty parameter lists were sent as empty arrays rather than objects, causing API rejections. While DeepSeek and OpenRouter were patched, the identical broken code persisted in four other providers. The author noted that this wasn't malice, but a common oversight where developers fix the immediate issue without checking parallel folders.

Half-Fixed Scans for Unpropagated Patches

The new tool, half-fixed, groups files by path structure (e.g., src/Providers/*/Maps/ToolMap.php) and analyzes git history for commits that touched some siblings but not others. It checks if the 'old' buggy code remains in the untouched siblings. For LiteLLM, which supports over 100 providers, the scan identified 66 high-confidence findings. The author manually verified these, confirming 13 real gaps, which resulted in 11 distinct bugs. Notably, the tool operates entirely on local git data with no network calls or AI inference during the scan, processing 5,000 commits in 69 seconds.

Confirmed Bugs in LiteLLM

Two specific bugs were confirmed and submitted as pull requests to LiteLLM. PR #44925 addressed a configuration mismatch where scope_user_search_to_org did not respect config.yaml settings, allowing internal users to search all proxy users despite the setting being disabled. PR #44936 fixed a streaming error in the vLLM passthrough, which failed to check the stream value, causing 500 errors for non-streaming requests. These examples highlight how subtle configuration and logic drift can occur between provider adapters.

Limitations and False Positives

The audit revealed that most findings are not actual bugs. In mature projects like Laravel, Django, and Rails, differences between siblings are often intentional, such as using different gems or APIs for deprecated features. The author emphasizes that half-fixed does not auto-fix; it generates a shortlist for human review. The tool currently struggles with projects where sibling files are named differently or refactored heavily, such as Knex, where it fails to match files like mysql-querycompiler.js and pg-querycompiler.js due to naming conventions.

Key Takeaways

  • A systematic 'sibling bug' pattern exists in multi-provider LLM libraries, where fixes are applied to one adapter but missed in others.
  • The tool half-fixed identifies these unpropagated patches by analyzing git history and file path structures.
  • Manual verification is critical: only 13 out of 66 findings in LiteLLM were real bugs, with many being intentional design choices.
  • The tool is open-source (MIT) and available via npx, with integrations for Claude Code and Codex to assist in triaging findings.

The Bottom Line

While automated history mining is powerful, the low hit rate of 13 real bugs out of 66 findings proves that human judgment remains the final gatekeeper in code quality. Tools like half-fixed are best viewed as triage assistants rather than autonomous fixers, highlighting that consistency across parallel codebases is a discipline problem, not just a technical one.