A customer tells a Hindi support bot to change the quantity on a 'kurti' order. The bot replies perfectly in Hindi, confirming the update. But it actually modified the 'kurta' order instead. The customer gets the wrong item, and the fluent reply hid the error. This is the exact failure mode BoliBench was built to catch.

Judging State, Not Sentences

BoliBench, created by Priyanshu Jha for the Hacktoberfest Open-Source AI Challenge, is an open-source behavioral benchmark for tool-using conversational agents. Unlike traditional evaluations that rely on LLMs grading text replies, BoliBench ignores what the agent says. It runs the agent against a sandboxed order system, records every actual tool call, and verifies the final sandbox state. Did the right order get cancelled? Did the quantity change correctly? The reply text is stored for debugging but never used as evidence of success.

Real-World Language Complexity

The benchmark features 12 hand-written cases in Hindi, Hinglish, and Awadhi-influenced Hindi, reflecting how people actually speak in North India. It tests ambiguous references like 'the red one,' mid-sentence corrections, and look-alike items like 'kurta' and 'kurti.' It also includes forbidden actions, such as bulk cancels, and injected tool failures to test recovery logic. The goal is to move beyond fluency and test whether the agent understands the operational context of the request.

Model Performance and Regression Detection

Initial runs showed stark differences. An example agent using qwen3:8b scored 3/12 on its first version, failing primarily on behavioral logic rather than language understanding. It invented order IDs and acted without waiting for user confirmation. A second version improved to 6/12 by fixing prompt logic, but BoliBench also caught a regression where the agent became overly cautious. In contrast, gemma3:4b scored only 2/12 locally, largely due to strict JSON parsing failures in tool calls, not necessarily a lack of understanding. In one case, Gemma’s rejected tool call was actually correct, while qwen’s accepted call was wrong.

Practical Integration and Future Work

BoliBench is Apache-2.0 licensed and supports integration with frameworks like Mastra and services like Temporal, ElevenLabs, and Render. The project emphasizes open innovation, allowing teams to run regression tests locally on open-weight models without exposing customer data to third-party APIs. While the current suite is synthetic, the next step is native speaker review to validate linguistic accuracy. For developers shipping Hindi or Hinglish agents, BoliBench offers a deterministic way to verify that their agents are doing what they say they are doing.

Key Takeaways

  • BoliBench judges agents on sandbox state changes, not reply text.
  • Fluent Hindi replies can hide critical operational errors.
  • Local runs with open-weight models enable cheap, frequent regression testing.
  • Current cases need native speaker validation to ensure linguistic accuracy.

The Bottom Line

Fluency is a trap for LLM agents; if your evaluation doesn't check the database, you're just grading the hallucination.