If you’re building code search infrastructure, the question isn’t whether to use specialized embeddings, but which ones survive contact with real-world data. Papr’s new papr-embed-code-1 models, available in 0.6B and 4B parameters, claim state-of-the-art performance on code retrieval benchmarks like CoIR. However, a recent deep dive by Qdrant reveals that while these models are trained on much of CoIR’s data, they also demonstrate measurable gains on code they never saw during training, specifically beating their base Qwen3-Embedding models by approximately 0.03 nDCG@10 on unseen tasks.
Performance on Unseen vs. Seen Data
The Qdrant team tested the models against tasks Papr didn’t train on, including HumanEval, MBPP, FreshStack, and WikiSQL. The results showed consistent improvements: the 4B model gained an average of 0.034 nDCG@10, while the 0.6B model improved by 0.026. Notably, the gains varied by task; on WikiSQL, the 4B model achieved a significant 0.070 increase, whereas on HumanEval, the base model was already so strong (0.98) that there was little room for improvement. Conversely, performance on prose tasks dropped slightly, with the 4B model losing 0.016 nDCG@10 compared to the base model, suggesting that if your corpus is primarily documentation or prose, sticking with the base Qwen3 model is still the safer bet.
Quantization Changes the Infrastructure Equation
For developers managing large codebases, memory footprint is often the bottleneck. The 4B model’s vectors are 2,560 dimensions, requiring 10 KB per vector in float32. However, Qdrant’s testing shows that these vectors compress exceptionally well. Using 1-bit TurboQuant, each vector shrinks to just 328 bytes—a 31x reduction—while retaining 99.7% of the float32 score without rescoring. With 2x oversampling rescoring, it recovers full performance. This means the 4B model, when compressed to 1 bit, actually outperforms the uncompressed 0.6B model while using 12x less RAM. This is a critical insight for scaling: you can afford the higher-quality 4B embeddings if you implement aggressive quantization strategies.
Hybrid Search and BM25 Considerations
A common instinct in code search is to combine dense vectors with keyword search like BM25, assuming exact identifier matches are crucial. Qdrant tested this using Qdrant 1.20’s code tokenizer, which splits camelCase and snake_case identifiers. The results were surprising: BM25 alone performed poorly, scoring just 0.01 nDCG@10 on APPS tasks because queries and code solutions rarely share words. Fusing BM25 with dense search at equal weight actually degraded performance by 0.02 to 0.17. Only when BM25 was heavily down-weighted (RRF weight of 0.1) did it reach break-even. For most code search use cases, dense embeddings alone are superior, and adding BM25 adds complexity without meaningful gain unless users are specifically searching for exact identifiers.
Key Takeaways
- Papr-embed-code-1 beats base Qwen3 models on unseen code tasks by ~0.03 nDCG@10 but underperforms on prose.
- 1-bit TurboQuant compression makes the 4B model more memory-efficient and higher quality than the uncompressed 0.6B model.
- BM25 hybrid search generally hurts performance in code retrieval unless heavily down-weighted and used for exact identifier matching.
- Increasing hnsw_ef to 256 can allow 1-bit quantized vectors to outperform float32 at lower ef settings while reducing latency.
The Bottom Line
For large-scale code retrieval, skip the 0.6B model and go straight to the 4B with 1-bit quantization. It offers better accuracy, massive RAM savings, and faster query times when tuned correctly, making it the new default for serious code search infrastructure.