Hybrid Search: Combining BM25 and Vector Search in RAG

TL;DR — The most common source of disappointment I see in enterprise RAG projects is this: "our model is smart, but it can't find the right document." The problem is usually not the LLM — it's the retrieval layer. Pure vector (dense) search captures meaning but misses exact strings like product codes, error codes, IBANs, and clause numbers. Pure BM25 (lexical) search catches exact terms but doesn't understand synonyms or indirect phrasing. In this piece I walk through how to combine the two with Reciprocal Rank Fusion (RRF), how Anthropic's "contextual retrieval" approach reduces failed retrievals by prepending context to chunks before embedding, how to add a cross-encoder reranker as a final stage, and how all of this comes together in a concrete Turkish bank/e-commerce knowledge base example. I close with the KVKK (Turkish data protection law) considerations you need when your retrieval corpus contains personal data.

Why I keep coming back to this topic

In nearly every organization I consult with, the same scene repeats itself: a team enthusiastically sets up a vector database, embeds their documents, wires it to an LLM, and everything looks great on demo day. Then it goes to production, real users start asking real questions, and the system either returns an irrelevant document or nothing at all for a query like "What does Circular No. SGK-2024/117 say?" At that point I usually make the same diagnosis: retrieval is walking on one leg. The team trusted that the embedding model was "smart," but there's no guarantee an embedding model will correctly distinguish the exact digits "117," or a rare alphanumeric code like "SGK-2024/117," from similar-looking neighbors.

Let me confess something here: I was once in the "embeddings solve everything" camp myself. In one of the first production RAG systems I built in mid-2023, I used dense retrieval exclusively. It worked well for natural-language questions like "what is the return policy," but the moment a customer asked "does SKU 8452-B have a warranty," the system got completely lost. That's because embedding models are inherently weak at turning a string like "8452-B" into a meaningfully distinct vector — for this kind of code, statistical (lexical) matching is a far more reliable signal than semantic proximity. That experience pushed me toward hybrid architecture, and I've been teaching this in every workshop and consulting engagement since.

Dense (vector) search: what it does, what it misses

Dense retrieval converts a piece of text (a query or a chunk) into a high-dimensional numeric vector — typically somewhere between 384 and 3072 dimensions. Embedding models that perform this transformation (OpenAI's text-embedding-3 family, Cohere embed, Voyage AI, or open-source families like BGE and E5) try to capture the "meaning" of the text. As a result, lexically different but semantically close phrases like "araç kiralama" and "otomobil kiralamak" (car rental vs. renting a vehicle) end up close together in vector space. Nearest neighbors are found via cosine similarity or dot product, usually accelerated through approximate nearest neighbor (ANN) indexes such as HNSW or IVF.

The strength of this approach is that a user can reach the right document even without knowing the exact wording. "When will the money reach my account" can match a document that says "havale valör süresi" (transfer value date) because the two are semantically close even though they share no words. But this is exactly where the weakness begins: embedding models generalize according to the distribution of their training data, and they offer no reliable discriminative power for rare, domain-specific strings — product codes, legal clause numbers, IBAN fragments, error codes, abbreviations. For an embedding model, "ABC-123" and "ABC-124" often collapse into nearly the same vector because the model has generalized both as "a product code" without knowing that the single-character difference is critical. Yet in a customer service system, that one character can mean an entirely different product with entirely different warranty terms.

There's another, less-discussed weakness in dense retrieval: negation and precision. "Price excluding VAT" and "price including VAT" can land on very close vectors because of high lexical overlap, even though their meanings are polar opposites. This is a known, not-yet-fully-solved problem in embedding models.

BM25 / sparse retrieval: what it does, what it misses

BM25 (Best Matching 25) is the statistical ranking function underlying Elasticsearch, OpenSearch, Postgres's tsvector feature, and nearly every Lucene-based search engine. The logic is simple but powerful: it produces a score based on how frequently a term appears in a document (term frequency), how rare that term is across the entire collection (inverse document frequency), and a normalization based on document length.

The BM25 score is roughly computed as:

score(D, Q) = Σ IDF(qi) * [f(qi, D) * (k1 + 1)] / [f(qi, D) + k1 * (1 - b + b * |D| / avgdl)]

Here f(qi, D) is the frequency of a query term in the document, |D| is document length, avgdl is the average document length across the collection, and k1 and b are tunable hyperparameters (typical values: k1 ≈ 1.2-2.0, b ≈ 0.75). The IDF term gives high weight to rare words (like "SGK-2024/117") and low weight to common ones ("and," "a," "for").

BM25's strength is exactly where dense retrieval is weak: it's nearly flawless for exact-match, rare, domain-specific expressions. When a user types "IBAN TR33 0006 1005 1978 6457 8413 26" or "error code E-4042," BM25 catches this directly because it's searching for word matches, not inferring meaning.

But its weakness is just as clear: it doesn't understand synonyms, superordinate concepts, or indirect phrasing. When a user asks "when will the money reach my account," and the document instead says "transfer value date," BM25 finds zero lexical overlap between the two phrases and either scores it low or doesn't match it at all. It's also fragile to spelling variation (case sensitivity, and in Turkish specifically the notorious dotted/dotless "İ/i, ı/I" confusion) and to root/suffix variation without a proper tokenizer and stemmer. In an agglutinative language like Turkish, this is a serious engineering burden — you can't teach BM25 that "bankalar," "bankada," and "bankacılık" share the same root without a separate morphological analysis layer (like Zemberek).

Why the two together outperform either alone

The summary so far is simple: dense search captures meaning but misses precision; BM25 captures precision but misses meaning. These two failure modes are largely non-overlapping — meaning that when one fails, the other is likely to succeed. Statistically, when you combine two models whose errors are weakly correlated, the combined system's error rate drops below either model's individual error rate. This is exactly the core principle behind ensemble learning.

I usually put it this way in workshops: "Dense search is your smart but forgetful assistant; BM25 is your meticulous but unimaginative archivist. Put them in the same room, and one covers the other's blind spot." The numbers I see in practice bear this out — various academic studies (including research on the BEIR benchmark) and industry reports consistently show hybrid search outperforming either dense-only or BM25-only search on metrics like recall@k and nDCG. The exact magnitude depends on the dataset, but the direction is very clear: hybrid almost never performs worse than either single method, and it frequently performs meaningfully better.

There's an important nuance worth stressing here: hybrid search is not "averaging," it's "fusion." You need a mechanism that takes two different ranked lists and re-ranks the candidate documents using both signals. That's where Reciprocal Rank Fusion comes in.

Reciprocal Rank Fusion (RRF): the most practical way to merge two lists

RRF is a simple but surprisingly effective method for combining ranked lists from different search systems without touching the raw scores at all — it uses only the rank (position) information. Why look at rank instead of raw scores? Because BM25 scores (typically 0 to 40, unbounded) and cosine similarity scores (0 to 1) live on completely different scales and distributions. Summing or weight-averaging them directly is misleading due to score normalization mismatches. RRF sidesteps this problem entirely because it works on position in the ranking, not the score itself.

The formula is:

RRF_score(d) = Σ (1 / (k + rank_i(d)))

Here d is a document, rank_i(d) is that document's position in ranking list i (say, the BM25 list or the vector list), and k is a smoothing constant typically set around 60 (the value recommended in the original paper, and it works well in most practical scenarios). If a document ranks highly in both lists, the sum of the two terms is high and it rises to the top of the final ranking. If a document is high in only one list and absent from the other, only that single term contributes — it can still make the final list, but with a lower combined score.

Let me give a concrete example. Suppose for the query "credit card statement dispute period," the BM25 list ranks an FAQ entry containing the words "dispute" and "statement" at position 1, but that same document ranks 8th in the vector search because it's semantically a bit distant. Meanwhile, a different document titled "account summary change request" ranks 2nd in vector search (because it's semantically close) but only 15th in BM25 (because lexical overlap is low). With RRF (k=60):

As you can see, both documents rise to the top with close final scores — if you'd looked at only one list, one of them could have been lost entirely. RRF's biggest practical advantage is that it can merge lists from different systems (even three or four different retrieval methods) instantly, with no training or calibration required. Elasticsearch 8.8+ ships an rrf retriever, Weaviate offers it as a hybrid search mode, and Qdrant and Vespa provide similar mechanisms out of the box; it's also easy to implement in your own pipeline in about 15-20 lines of code.

Contextual retrieval: stop stripping chunks of their context

Now let's get to the part I think is the least understood but highest-impact: contextual retrieval. This technique, laid out by Anthropic in a research note published in late 2024, aims to meaningfully reduce the "failed retrieval" rate in RAG systems, and I now recommend it as a standard part of every chunking strategy.

Let me describe the problem. When you split a document (say, a 50-page corporate policy document) into chunks and embed each one separately, every chunk becomes disconnected from its own context. Consider a chunk containing the sentence "This rate increased by 3% compared to the previous quarter." That sentence, on its own, doesn't say which product, which quarter, or which company it's about. When you embed the chunk in isolation, the embedding model has no way to know this context either, and the resulting vector lands in a fairly "generic" region of the space that doesn't really convey much. If a user asks "what was ACME's third-quarter growth rate," this chunk may not come out sufficiently close to the query semantically, because it contains none of the words "ACME," "growth," or "third quarter" — it just says "this rate."

Contextual retrieval's solution is simple but effective: before embedding each chunk (and before adding it to the BM25 index), a short context sentence describing that chunk's place within the document is automatically generated by an LLM and prepended to the chunk. So the example chunk above becomes:

Context: This chunk is taken from the "Revenue Analysis" section of 
ACME Inc.'s Q3 2024 financial report and discusses the growth rate 
of the company's main product line.

Original text: This rate increased by 3% compared to the previous quarter.

This addition is done with a short, cheap LLM call (in Anthropic's own approach, the entire document is loaded into a prompt cache and a cheap completion is generated per chunk), and the cost is kept manageable thanks to prompt caching. The result: both the embedding and the BM25 index now use a much richer, self-contained representation for chunks that would otherwise have been "lost."

According to results Anthropic published, adding contextual embeddings alone reduces the failed-retrieval rate by roughly 35%; adding contextual BM25 as well (i.e., feeding the same contextualized text into the BM25 index) brings that reduction to roughly 49%; and combining this with a reranker pushes the total improvement to roughly 67%. These numbers are of course dataset-dependent, but the direction is unmistakable: freeing chunks from their isolated context produces gains on both the dense and sparse sides, because the problem was never the retrieval method itself — it was the missing context inside the chunk.

In enterprise projects, I apply this in a fairly practical way: after chunking a document, instead of writing a separate prompt for each chunk, I feed the whole document as context to the model and ask it to summarize, in 1-2 sentences, where this particular chunk came from and what topic it covers. With models that support prompt caching (Claude, the GPT family), this keeps the per-document cost manageable — especially worth remembering that for a large corporate document repository with thousands of chunks, this is a one-time cost, not something that recurs at query time.

The final layer: cross-encoder reranking

Hybrid search plus RRF gives you a good candidate list, but that list is still a "coarse" ranking, because both BM25 and dense embeddings compute the relationship between a query and a document independently of each other (the bi-encoder paradigm). A cross-encoder reranker, by contrast, feeds the query and the candidate together into the same transformer, which lets it model the relationship between them at a much finer grain. The cost is computation — each query-document pair requires a separate forward pass, so you don't run the reranker over the whole collection, only over the top 20-50 candidates that hybrid search has already surfaced.

A typical production pipeline works like this:

1. The user's query goes to both the BM25 index and the vector index.
2. The top 50-100 candidates are retrieved from each.
3. RRF merges these two lists into a single ranked candidate list.
4. The top 20-30 candidates from this list are sent to a cross-encoder reranker (e.g., Cohere Rerank, BGE-reranker, Voyage rerank, or a model you've fine-tuned yourself).
5. The top 3-5 chunks, according to the reranker's final scores, are passed to the LLM as context.

This final step makes the biggest difference in "almost right but not quite" situations — it pushes down documents RRF ranked highly but that don't actually satisfy the query's real intent, and pulls the genuinely relevant one to the top. My observation is that adding a reranker delivers the largest benefit in enterprise datasets with many documents that are highly similar to one another — for example, an e-commerce catalog with hundreds of similar product descriptions. In a small knowledge base with few, clearly distinct documents, the reranker's contribution may be marginal, and it might do little more than add latency and cost.

Practical tuning: the alpha weight and when not to use hybrid search

Some vector databases (Weaviate, Pinecone's hybrid mode, Qdrant) allow a direct weighted sum via an alpha parameter instead of, or in addition to, RRF:

final_score = alpha * dense_score_normalized + (1 - alpha) * bm25_score_normalized

alpha = 1 means purely dense, alpha = 0 means purely BM25. The downside of this approach compared to RRF is that you need to normalize the scores (min-max or z-score normalization), and this normalization can be fragile across datasets. I generally recommend starting with RRF because it requires no tuning and provides a solid baseline; then, once you have a labeled evaluation set, sweep alpha across values like 0.3, 0.5, 0.7 and measure which works best for your query distribution.

The single biggest factor affecting alpha is your users' query style. If your users mostly ask natural-language questions ("how many years do I need to pay premiums before I can retire"), weighting toward dense makes sense. If your users frequently use exact expressions like codes, ID numbers, product numbers, or reference numbers (a call center assistant or technical support bot, for instance), you need to weight toward BM25. Some teams take this a step further and route queries dynamically — passing the query through a simple classifier first (e.g., a regex check for "does this query contain a code/number") and choosing alpha based on query type. I call this "query routing," and I'm seeing it show up more and more in mature RAG systems.

I should also say clearly: hybrid search isn't always necessary — this is a balance I stress often when consulting. If your dataset is small (a few hundred documents), your query variety is low, and rare codes/identifiers are almost never present in the content, dense retrieval alone may be sufficient, and the added operational complexity of hybrid architecture (maintaining two indexes, keeping two systems in sync, RRF/reranker latency) may not be worth it. Similarly, if your content is entirely structured tabular data (say, just a price list), a direct SQL query or structured search is probably a better fit than retrieval altogether — RAG is not a hammer for every nail.

A concrete Turkish enterprise example: a bank's customer service knowledge base

I don't want to leave this abstract, so let's walk through an example. Say we're building a customer service assistant for a mid-sized bank. The knowledge base contains: product brochures ("Gold Term Deposit," "Individual Pension System contribution rates"), operational procedures ("card cancellation process," "statement dispute period is 60 days"), regulatory references (BRSA communiqués, SGK circulars), and product codes (credit card product codes, campaign codes like "KK-PLATIN-2024").

Customer queries cluster into two extremes. One group is entirely natural language: "What do I gain if I transfer my retirement pension to this bank," "I lost my credit card, what should I do." The other group is very specific, code/number-heavy: "What are the terms of the KK-PLATIN-2024 campaign," "What does error code 4021 mean," "Can I make a transaction with just the last 4 digits of my IBAN."

In this scenario, pure dense retrieval frequently fails on the second group because a code like "KK-PLATIN-2024" doesn't separate out meaningfully in embedding space — the model generalizes it as "a credit card campaign code" without being able to distinguish which campaign it actually is. Pure BM25, on the other hand, struggles with the first group because "if I transfer my retirement salary" is likely expressed in the document with completely different words, like "salary promotion" or "SGK pension transfer."

The architecture we built for this example works as follows:

After building this architecture, the biggest difference showed up for queries containing campaign codes and regulatory clause numbers — previously, in the pure-dense system, a significant share of these queries either returned an irrelevant brochure or produced an "I don't have information on this" response. Contextual retrieval also proved especially valuable for long regulatory texts, because a standalone clause chunk ("the rate specified in the second paragraph is applied as 2%") gave no indication of which circular or which product it belonged to; once context was added, these chunks became far more accurately captured on both the BM25 and dense sides.

Evaluation: decide with measurement, not with gut feeling

The most common mistake I warn against once hybrid search is built is this: "it looks good" is not enough — you have to measure it. My recommended minimum evaluation setup looks like this:

• Labeled query-document pairs: at least 100-150 queries, ideally sourced from real user logs (if available) or prepared jointly with the business unit, each labeled with which chunk(s) count as the "correct answer."
• Recall@k: how many of the correct chunks are captured within the top k results. In RAG, k=5 or k=10 is usually the critical threshold, since that's roughly how much context you're passing to the LLM.
• nDCG (normalized Discounted Cumulative Gain): measures not just whether the correct document is in the list, but how high it ranks — particularly useful for seeing the reranker's impact.
• MRR (Mean Reciprocal Rank): summarizes, on average, at what position the correct answer appears — practical for reporting as a single number.

When you compute these metrics separately across three or four configurations (BM25 only, dense only, hybrid + RRF, hybrid + RRF + reranker) and lay them side by side, you get a clear picture of which layer is genuinely contributing and which is just adding cost and latency. I never recommend adding a reranker to a client's architecture without first showing this A/B table, because in some datasets the reranker's contribution is marginal and may not be worth the latency cost.

I also recommend regularly correlating user feedback in production (thumbs-down, "this answer wasn't helpful" buttons, etc.) with retrieval logs to catch "silent failures" — very often, what looks like the LLM giving a "wrong answer" actually originates from wrong or incomplete retrieval upstream.

Common mistakes

A few of the mistakes I see most often in the field:

• Not tuning the tokenizer/analyzer for Turkish. Some teams set up BM25 with the default English analyzer and then wonder why it performs poorly on Turkish queries. Turkish's agglutinative structure requires either stemming (with tools like Zemberek) or at minimum an edge n-gram / character n-gram approach.
• Locking chunk size to a single fixed number. Rather than searching for one universal chunk size (200 tokens, 500 tokens), it's worth experimenting with different chunking strategies per document type (FAQ entry, table, long paragraph).
• Skipping contextual retrieval and relying entirely on the reranker. A reranker can't rank a candidate that never reached it — if a chunk was never captured at the retrieval stage to begin with, the reranker provides zero benefit. You need to improve recall (getting the right candidate into the list) before you improve precision (ranking it correctly).
• Leaving the RRF constant k untouched but never testing alpha. RRF is a solid default, but that doesn't mean it's optimal for every dataset; it's worth testing at least a few configurations.
• Setting up an evaluation set once and never revisiting it. User behavior and content change over time; teams that don't refresh their evaluation set quarterly don't notice the system silently degrading.
• Running the reranker over the entire collection. Cross-encoders are computationally expensive; running thousands of documents directly through a reranker drives latency up substantially. Narrow the candidate set to 20-50 via hybrid search first, and only run the reranker on that small set.

Turkey and KVKK: when personal data lives in the retrieval layer

In enterprise RAG projects, the knowledge base frequently contains personal data — customer complaint records, employee personnel files, medical reports, credit application history, and the like. When building a hybrid search architecture in this context, there are several KVKK (Turkey's Personal Data Protection Law) considerations I always address as a separate topic in consulting engagements:

• Data minimization applies at the chunk level too. When generating context for chunks (contextual retrieval), be careful not to leak unnecessary personal data into that context. For example, if an LLM automatically generates a summary like "this chunk is from [Name Surname]'s complaint numbered [National ID Number]" for a customer complaint chunk, it has effectively copied personal data one more time into both the embedding vector and the text index — which can run counter to the data minimization principle.
• Vector databases also fall under "data processing." Since embeddings are derived from personal data, these vectors themselves may be considered personal data under KVKK (particularly if there's a risk of reversal or near-text inference). If your vector database provider hosts data abroad, their data processing agreement and cross-border data transfer terms need to be evaluated under Article 9 of KVKK.
• Access control needs to be enforced at the retrieval layer too. BM25 and vector indexes generally don't support per-user authorization by default. To ensure a customer service bot only retrieves records within its own authorized scope, you need to add filters (metadata-based access control) to the index query — otherwise hybrid search could surface another customer's personal data to an unauthorized user as "relevant."
• Logging and retention periods. Retrieval logs (which queries returned which chunks) are usually kept for debugging and evaluation, but these logs can also contain personal data. Under KVKK's purpose limitation and storage limitation principles, how long these logs are retained and who can access them needs to be clearly defined, with anonymization or masking applied where possible.
• Anonymization can degrade embeddings. Some teams mask personal data (e.g., "Dear [CUSTOMER NAME]") before embedding — a good practice, but you need to make sure the masking doesn't destroy the chunk's semantic integrity; overly aggressive masking can also degrade retrieval quality. Finding this balance requires the legal and technical teams to work together.

Retrieval as an ongoing engineering discipline

Building hybrid search is not a "build it once and you're done" project — I always warn clients about this. As your knowledge base grows, as your users' query distribution shifts, as new products/codes/abbreviations get added, you need to periodically revisit your BM25-dense balance, your chunking strategy, and your contextual retrieval prompts. The healthiest approach is to treat the retrieval layer as its own "service," with its own evaluation set, its own version history, and its own monitoring dashboard. That way, even when you change something on the LLM side, you can be confident that retrieval quality has stayed constant or improved independently — and when someone in production asks "why did answer quality drop," you can rule the retrieval layer in or out as a variable much faster.