A new preprint titled "Verify, Then Vote" demonstrates that combining five small, local language models with a source-verification step significantly outperforms simple majority voting. While Claude Sonnet 5.5 proved to be both faster and cheaper when acting as the sole judge, the researchers argue that their modular approach offers superior transparency and reproducibility. The study, published on November 6, 2026, highlights a critical trade-off in LLM orchestration: raw performance versus auditability.

The Local Ensemble Setup

The experiment ran on an Apple M3 Pro with 36 GB of RAM, utilizing Ollama to host five distinct models: Llama 3.2 3B, Mistral 7B, Gemma 2 2B, Phi-3 mini, and Qwen 2.5 3B. Each model independently generated search queries, retrieved passages from Wikipedia or research abstracts, and provided answers with confidence scores and citations. This entire process took approximately eight hours for the full dataset, establishing a baseline for low-cost, privacy-preserving inference.

Verification Logic and Accuracy Gains

Instead of relying on a black-box vote, the system used Claude Sonnet 5.5 as a separate checker to verify if cited passages supported, contradicted, or were insufficient for each answer. A formal rule then selected the final answer, prioritizing those with verified support. On 526 mixed-accuracy questions, this method raised accuracy from 71.5% (simple vote) to 86.5%, fixing 91 errors while introducing only 12 new ones. The authors proved via Lean 4 that any incorrect swap by the rule must have been caused by a source check that erroneously backed the wrong answer.

The Claude-Only Alternative

The researchers explicitly tested using Claude Sonnet 5.5 as the sole judge for selecting the best answer from the ensemble's outputs. This approach achieved 93.9% accuracy on the same subset, surpassing the 86.5% of the verify-then-vote rule. Furthermore, it was more cost-effective, costing approximately $0.014 per question compared to $0.021 for the verification pipeline. This confirms that for pure accuracy and cost, a single powerful model acting as a judge is currently superior to the complex local ensemble workflow.

Transparency and Reproducibility

The primary advantage of the "Verify, Then Vote" framework is not immediate performance, but the ability to audit decisions without new model calls. Because answering, checking, and choosing are decoupled and saved, researchers can re-run decision rules on existing data. A Python script dropping one model at a time changed the final answer in 9.4% of cases, all within 23 seconds and zero API costs. This allows for rapid experimentation with decision logic, something impossible when a monolithic judge like Claude makes the final call in a single pass.

Key Takeaways

  • Local ensembles of five small models can achieve 85.0% accuracy with verification, up from 77.1% with simple voting.
  • Claude Sonnet 5.5 as a sole judge achieved higher accuracy (93.9%) and lower cost ($0.014/question) than the verification pipeline.
  • The verification method failed an initial pre-registered harm test due to a 5.5% error rate on correct answers, but a revised rule passed with a 2.2% worst-case rate.
  • Separating answer generation, source checking, and final selection enables zero-cost re-evaluation of decision rules.
  • The study was conducted on an Apple M3 Pro with 36 GB RAM, demonstrating feasibility for high-end consumer hardware.

The Bottom Line

While Claude Sonnet 5.5 is objectively better at judging answers, the "Verify, Then Vote" approach wins on engineering principles. If you need to know exactly why your AI chose an answer and want to tweak that logic without burning more tokens, the extra cost is worth the audit trail.