TurboVec: The Vector Index That Fits 10M Docs in 4GB and Searches Faster

TurboVec is an open-source project that solves a real problem: traditional vector indices consume RAM like there's no tomorrow. Built on Google Research's TurboQuant algorithm, it's written in Rust with Python bindings — meaning you use it as a normal Python extension but with compiled-language performance.

The trick: TurboQuant is a "data-oblivious" quantizer. No preliminary training phase required. You add vectors, they get indexed. Done. No parameter tuning, no rebuilds as your corpus grows. Pure online ingest.

On concrete benchmarks: TurboVec beats FAISS IndexPQFastScan by 12–20% on ARM (the stuff running on phones and edge devices) and at least matches it on x86. How? Hand-written kernels in NEON (ARM) and AVX-512BW (x86). Translation: it actually knows how to talk to hardware instead of pretending.

Another feature that isn't just marketing: search-time filtering. You pass an allowlist of IDs or a bitmask, and the kernel respects it directly during the search — no over-fetching, no recall hit. Hybrid RAG? SQL + semantic search together? Exactly what you need.

Here's the kicker: it's pure local. Zero managed services, data stays on your machine. Want a fully air-gapped RAG stack with open-source embeddings? TurboVec is literally what you were looking for.

Framework integrations? It has drop-in replacements for LangChain, LlamaIndex, Haystack, and Agno. Change the import, your pipeline doesn't even notice. That's engineering for humans.