Hybrid Search: BM25, Vector & Reranking 2026

Hybrid search combines sparse keyword retrieval — BM25 — with dense vector search to handle the query types that break each approach in isolation. The fusion mechanism, Reciprocal Rank Fusion (RRF), operates on ranks not scores, which solves the score-incompatibility problem that makes naïve weighted averaging fail in production RAG pipelines.

The core tension is architectural. BM25 excels at exact-match queries — product codes, entity names, precise technical terms — but has no representation of semantic meaning. Dense retrieval excels at paraphrase and conceptual queries but struggles when the query contains a rare term that appears verbatim in only one or two documents. Neither wins across all query types. Empirical benchmarks on the WANDS e-commerce dataset confirm this: baseline BM25 and pure KNN are statistically indistinguishable at NDCG 0.6983 vs 0.6953, while a well-tuned hybrid reaches 0.7497 — a 7.4% NDCG lift over either alone.

This guide covers every layer of the hybrid retrieval stack: BM25 mechanics and its parameters, dense vector search and the ANN index types that power it, RRF fusion and why it outperforms linear score combination, per-vendor implementation details across Pinecone, Weaviate, Qdrant, and Elasticsearch, and the cross-encoder reranking layer that adds a further accuracy lift for RAG retrieval pipelines. All benchmarks are sourced to primary publications and annotated where they are vendor-stated rather than independently replicated.

The failure modes of sparse and dense retrieval are complementary, which is why hybrid search works: each method fills the other's blind spot.

BM25 is a lexical algorithm. It scores documents by matching query terms exactly against an inverted index. This makes it fast and highly effective when queries contain rare, distinctive terms — product SKUs, person names, error codes, regulatory references. But it has no notion of meaning: a query for “automobile repair” will miss documents that use “car maintenance” exclusively, even if they are semantically identical.

Dense vector retrieval maps both query and document into a shared embedding space, allowing semantic similarity matching via approximate nearest-neighbor (ANN) search. It handles paraphrase and conceptual queries naturally. But it can underweight exact rare-term matches — a query containing a specific product code or technical identifier may not retrieve the one document that contains that exact string, if the embedding model has never seen similar terminology in training.

As Qdrant's engineering team observed: “Neither of the algorithms performs best in all cases. In some cases, keyword-based search will be the winner and vice-versa.” The empirical answer is fusion — but the fusion mechanism matters more than most teams expect.

BM25 (Okapi BM25) is a probabilistic retrieval model developed by Stephen E. Robertson and Karen Spärck Jones at City University London in the 1980s–90s. It is the dominant sparse retrieval algorithm in production search systems today, underpinning Elasticsearch, OpenSearch, Solr, and the BM25 sparse vectors in Pinecone and Qdrant.

The two free parameters

BM25 has two tunable parameters that control its scoring behavior:

• k1 ∈ [1.2, 2.0]— term-frequency saturation. Controls how quickly a term's score contribution saturates as it appears more times in a document. Lucene (Elasticsearch, OpenSearch) defaults to k1 = 1.2. Higher k1 means a term that appears 10 times scores notably higher than one that appears 5 times; lower k1 flattens that curve.
• b = 0.75 — document-length normalization. Penalizes long documents to prevent them from dominating by sheer word count. b = 1.0 applies full normalization; b = 0 disables it entirely. These are typical starting values — they are tunable per-corpus.

BM25 requires no neural inference at query time — it operates entirely on an inverted index lookup. This makes it extremely fast and CPU-compatible, which is why it remains the first-stage retrieval mechanism in many high-throughput production systems even when dense retrieval is layered on top.

Learned sparse: SPLADE

SPLADE (Sparse Lexical and Expansion model) is a learned sparse encoder that maps text to BERT-vocabulary-sized sparse vectors (30,522 dimensions for bert-base-uncased). It outperforms BM25 on most BEIR benchmarks by learning implicit query expansion — but it requires GPU inference, unlike classic BM25 which needs only an inverted index. The architectural choice between BM25 and SPLADE depends on your latency and infrastructure constraints, not retrieval quality alone.

Dense retrieval maps text into high-dimensional embedding space using a neural encoder — typically a sentence transformer or a specialized embedding model from providers like Cohere, Voyage AI, or OpenAI. Both query and documents are encoded into vectors, and retrieval becomes a similarity search: find the top-k documents whose vectors are closest to the query vector under cosine similarity or dot product.

Because exact nearest-neighbor search across millions of vectors is computationally intractable at query latency requirements, production systems use Approximate Nearest Neighbor (ANN) indexes. The dominant index type is HNSW (Hierarchical Navigable Small World), a graph-based index that trades a small recall loss for dramatically faster search. Qdrant, Weaviate, and Pinecone all use HNSW as their primary ANN structure.

The quality of dense retrieval is highly dependent on the embedding model. An embedding model trained on general web text may represent domain-specific terminology poorly — a careful embedding model selection against your specific corpus is a prerequisite before any hybrid fusion work, not an afterthought.

Matryoshka embeddings and cascaded retrieval

Matryoshka Representation Learning (MRL) produces embeddings where the first ddimensions of a full-size vector form a useful lower-dimensional embedding. Qdrant's Query API supports Matryoshka cascades — retrieve with 64-dim vectors first (fast/cheap), oversample to 128-dim for a second pass, then full-dimension for the final shortlist. This reduces ANN index traversal cost at scale while preserving recall.

RRF was formally introduced by Gordon V. Cormack, Charles L. A. Clarke, and Stefan Buettcher at the University of Waterloo in their 2009 SIGIR paper “Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods.” The algorithm is deliberately simple: for each document, sum the reciprocal of its rank position across all result lists, dampened by a constant k.

The formula

score(d) = Σ(q) 1 / (k + rank(result(q), d))

Where k is a ranking constant (default 60 in Elasticsearch and most implementations) and rank(result(q), d) is the 1-indexed rank of document d in result list q. Critically, raw scores are ignored entirely — only rank positions contribute. This eliminates the BM25-vs-cosine incompatibility problem without any normalization step.

The rank constant k = 60 means a document at rank 1 contributes 1/61 ≈ 0.0164 per result list, while one at rank 100 contributes 1/160 ≈ 0.00625 — a 2.6× difference. Higher k dampens the influence of top-ranked documents and gives more weight to documents that appear consistently across many lists, even at moderate rank positions. The Elasticsearch documentation cites the Cormack paper directly: “RRF requires no tuning, and the different relevance indicators do not have to be related to each other to achieve high-quality results.”

The benchmark data from Doug Turnbull's March 2025 study on the WANDS furniture dataset makes the pattern visible: basic RRF (0.7068 NDCG mean) immediately outperforms both BM25 alone (0.6983) and pure KNN (0.6953). The bigger gains come from domain-aware tuning on top of the fused base — adding an all-terms clause lifts to 0.7191, and boosting the product name field reaches 0.7497 NDCG mean (0.8418 median). The takeaway is that RRF is a reliable no-tuning baseline, but field-level boosting of high-signal attributes can extract meaningfully more accuracy in e-commerce and structured-document settings.

Note the single-dataset caveat: WANDS covers furniture e-commerce queries. Performance characteristics on long-form technical documents, multilingual corpora, or question-answering tasks will differ. RRF's rank-based nature makes it robust across domains, but field-boosting strategies are inherently corpus-specific.

Each major vector database exposes hybrid search differently. The table below synthesizes the current implementation details across Pinecone, Weaviate, Qdrant, and Elasticsearch — information that currently requires reading four separate vendor documentation sets to compile.

The most consequential difference is the Weaviate v1.24 default change. Teams upgrading from an earlier Weaviate version without explicitly setting fusionType will silently switch from RRF to Relative Score Fusion. Both approaches work well, but they produce different result orderings — so a version upgrade could subtly shift search quality without a visible error. Always pin fusionType explicitly if you have baseline evaluations to protect.

For teams building on Elasticsearch without an Enterprise plan, the client-side RRF pattern is well-documented and production-viable. The Pinecone and Qdrant vector database comparison covers infrastructure decisions at a higher level — this guide focuses specifically on the fusion and reranking layer on top of whichever database you choose.

A hybrid retrieval pipeline produces a fused candidate list — say, the top 100 documents by RRF score. A cross-encoder reranker then re-scores that shortlist by computing a joint relevance score for each (query, document) pair. Unlike bi-encoder embedding models that encode query and document independently, a cross-encoder sees both simultaneously — which allows it to model fine-grained semantic relationships that embedding-space proximity misses.

The critical architectural constraint: cross-encoders are not suitable for first-stage retrieval over millions of documents at query time. The joint computation scales as O(n) with candidate count. The correct pipeline is always first-stage ANN retrieval (top-100 to top-1000 candidates) → second-stage cross-encoder reranking. Qdrant supports this pattern natively via ColBERT late-interaction reranking in the Query API.

Voyage rerank-2.5's instruction-following capability deserves separate attention. Prior rerankers were query-and-document scorers — you got the model's general notion of relevance. Instruction-following lets you specify domain context: “prefer results that include regulatory references,” “prioritize documents from technical specifications rather than marketing materials,” or “weight results about enterprise pricing above consumer pricing.” This is qualitatively different from score-based reranking and opens personalization and context-aware retrieval paths that were not available with earlier cross-encoders.

The Voyage API accepts up to 1,000 documents per rerank call, with a total-token limit of 600K tokens for rerank-2.5. At 32K context per document-pair, this is sufficient for most enterprise document chunks. The rerank-2.5 pricing is unchanged from rerank-2 — the context-window doubling came at no additional cost per the August 2025 announcement.

Cohere Rerank 3.5 remains the production baseline for teams already using the Cohere API. As Notion co-founder and CTO Simon Last noted on its release: “Cohere is a key part of what makes Notion AI work. Their reranker gives us both the speed and quality we need, and it's consistently improving.” For teams starting fresh in 2026, Voyage rerank-2.5's instruction-following and larger context window make it the stronger default — with the caveat that the published benchmark comparisons are vendor-run.

A production hybrid search pipeline for RAG and vector database applications has three distinct stages. Understanding which concerns belong to which stage determines both quality and cost.

Stage 1 — Dual retrieval

Query both sparse (BM25 or SPLADE) and dense (ANN) indexes in parallel. Each returns a ranked candidate list — typically top-50 to top-500 per retrieval mode depending on your reranking budget. Whether this happens server-side (Qdrant v1.10+ Query API, Weaviate hybrid, Pinecone single-index) or client-side (Elasticsearch Basic tier with ranx) is an operational concern, not a quality one — correctly implemented, both produce the same fused list.

Stage 2 — RRF fusion

Apply RRF to the two ranked lists. The formula is simple enough to implement from scratch — you need nothing more than a dictionary of document IDs to accumulating RRF scores. For most teams, using the vendor's native implementation (Qdrant, Weaviate, Elasticsearch Enterprise) is preferable to custom code. The default k = 60 is a reliable starting point; adjust toward k = 30–40 if your use case favors top-1 precision over top-10 recall.

Stage 3 — Cross-encoder reranking

Take the top-N from the RRF-fused list (N typically 50–200) and pass each (query, document) pair to a cross-encoder. Return the top-k reranked results to the application layer. This stage is where Voyage rerank-2.5's instruction-following capability becomes actionable — the instruction is prepended to the query before each cross-encoder call, not baked into the retrieval index.

Where the AI transformation work actually lives

Most of the engineering investment in a hybrid pipeline is not in the fusion algorithm — RRF is three lines of code. The real work is in embedding model evaluation against your specific corpus, chunking strategy (document length determines whether the 32K context window of rerank-2.5 is a meaningful constraint), metadata filtering to ensure ANN candidates are within scope before reranking, and evaluation harness setup (without an NDCG or MRR baseline on your own queries, you cannot measure whether any iteration actually improved quality).