Hybrid Search in RAG: Boosting Accuracy with BM25 + Vector + Reranking

TL;DR — In RAG (retrieval-augmented generation) systems, the problem I see most often is relying on a single search method. Pure vector search captures meaning but stumbles on exact term matching; BM25 finds the keyword but misses the synonym. The solution is hybrid search: combining BM25 (sparse) with vector (dense) search via Reciprocal Rank Fusion (RRF), then adding a cross-encoder reranker on top. The numbers are clear: on the WANDS e-commerce benchmark, tuned hybrid scores 0.7497 NDCG (BM25 alone 0.6983, pure vector 0.6953); hybrid + Cohere Rerank delivers a 39% improvement in Recall@5 over pure-dense. In this article I explain the architecture, how RRF works, the reranker's cost/latency trade-off, practical tips for Turkish, and KVKK-aware in-house deployment. I recommend hybrid search as the "minimum viable baseline" for every RAG project.

When I advise organizations on RAG projects, the disappointment I encounter most begins with this sentence: "We set up a vector database, we got embeddings, but the system still retrieves the wrong documents." I have heard it so often that I now know the answer before the question is even asked. The problem almost always stems from blindly trusting a single retrieval method.

To understand the issue, we need to separate two different search philosophies.

Sparse search — BM25. Classic keyword search. It looks at whether the term you're searching for appears exactly in a document, how frequently it appears, and how rare that term is in the collection. It works brilliantly in cases requiring exact matches such as product codes, legal article numbers, proper nouns, abbreviations, and error codes. When searching for "model number ABC-1234," BM25 scores a direct hit. But it cannot build the semantic bridge between "notebook" and "laptop"; if the word doesn't appear verbatim, it can't see it.

Dense search — vector embedding. It embeds text into a semantic space and looks at meaning similarity. The query "how to get car insurance" can find a document titled "vehicle policy application process" even if none of those exact words appear. Synonyms, paraphrases, and conceptual proximity are strong here. But it remains weak on exact term matching, especially for rare special terms and codes; the embedding can drift toward something "close but wrong."

"

BM25 knows the word, vector knows the meaning. Real-world queries want both at once.

That single sentence contains the entire logic of hybrid search: the two methods' weaknesses complement each other. Vector catches the semantic match BM25 misses; BM25 catches the exact term vector misses. When you combine them, you reach an accuracy neither could reach alone.

I am a person who believes in numbers, not in the sentence "it felt better." Fortunately, the benefit of hybrid search has been measured.

On the e-commerce search benchmark called WANDS, the results are very instructive. Let's look at the NDCG values (a metric measuring ranking quality):

Method	NDCG
Pure vector (dense)	0.6953
BM25 alone	0.6983
Tuned hybrid (BM25 + vector)	0.7497

Notice: pure vector and BM25 alone are almost neck and neck. So the myth that "vector is always better" collapses here. The real jump comes when you combine the two: tuned hybrid delivers about a 7.4% improvement over the single methods. In a field like search, that is not a difference to dismiss.

Let's look at a tougher scenario, financial documents where text and tables are intertwined. Here the combination of two-stage hybrid retrieval + neural reranking reaches Recall@5 = 0.816. That is, the vast majority of relevant documents are captured within the top five results; this is critical for the quality of the context given to the LLM.

But the number that impressed me most is the value the reranker adds. When a Cohere Rerank layer is added to hybrid search:

• A 39% improvement in Recall@5 compared to pure-dense search,
• An additional 17.4% improvement compared to hybrid RRF.

Let me emphasize this once more: adding a reranker nearly increases your recall by one and a half times compared to where you started with pure vector. Let me also add that RRF alone, even without score normalization, gives about 91% recall@10. So the fusion step already provides a serious gain very cheaply.

I always explain hybrid RAG in three clear stages. If you fix these three boxes in your mind, everything else falls into place.

Stage 1 — Retrieval. The query goes to two engines at once: BM25 and dense vector search. Each, with its own logic, returns, say, the top 50-100 documents. The goal at this stage is high recall; that is, not missing any relevant document. Precision is not our priority yet, because the later stages will fix it. We cast a wide net.

Stage 2 — Fusion (RRF). We merge the two lists into a single ranking. Here Reciprocal Rank Fusion (RRF) comes into play; I'll explain its formula shortly. The beauty of RRF is that it uses only the rank information, without bothering to bring the two engines' scores to the same scale (normalization). Directly summing a BM25 score and a cosine similarity score is summing apples and pears; RRF elegantly sidesteps this problem.

Stage 3 — Reranking. We feed the top, say, 50 documents from fusion into a cross-encoder reranker. This model evaluates the query and each document together (as a pair) and produces a true relevance score. Then we re-sort by this score, prune to the top 5-10 documents, and place only those into the LLM's context window. This final pruning both increases accuracy and reduces token cost.

Let me summarize this architecture in a table:

Stage	Goal	Priority	Typical output
1. Retrieval (BM25 + dense)	Build candidate pool	High recall	50-100 documents
2. Fusion (RRF)	Merge two lists	Rank consistency	Single ranked list
3. Reranking	Measure true relevance	High precision	Top 5-10 documents

Reciprocal Rank Fusion sounds academic but is actually a one-line idea. For each document, we look at its rank in each search list and sum the following contribution:

score(document) = sum ( 1 / (k + rank) )

Here "rank" is the document's position number in the relevant list (1, 2, 3...), and "k" is a smoothing constant; in practice 60 is generally used. A document gets this contribution from every list it appears in, and its total score forms.

Why does it work so well? There are a few reasons:

• It requires no score normalization. BM25 scores can be between 0 and 30, cosine similarity between 0 and 1; summing them is meaningless. Because RRF uses only the rank, not the raw score, this problem disappears.
• It weights top ranks. With k=60, the document at rank 1 gets a 1/61 contribution, rank 2 gets 1/62; the difference is more pronounced at the top. So a document that both engines placed at the top naturally rises.
• It's robust. Even if one engine produces a strange score, the system doesn't collapse; RRF is immune to outlier scores.

If you increase the k constant, the difference between ranks shrinks (more democratic); if you decrease it, the top ranks become more dominant. 60 is a good start; but I recommend experimenting and tuning with your own data. Part of the difference between "tuned hybrid" and "raw hybrid" comes precisely from here.

The cross-encoder reranker is the part of the architecture that adds the most value but also demands the most thought. Why is it so good? Because its operating principle is fundamentally different.

Vector search works with a "bi-encoder" logic: the query and document are converted to vectors separately, then compared. It's fast because document vectors can be precomputed. But because the query and document are encoded without "seeing" each other, fine nuances are lost. A cross-encoder, on the other hand, feeds the query and document to the model together, at the same time; it sees the word-by-word interaction. That's why it is far more accurate, but far slower; a separate model run is needed for each query-document pair.

Here is exactly where the balance lies. You can't apply the cross-encoder to the entire collection; scoring millions of documents one by one is both impossible and unnecessary. So you apply it only to the small candidate pool from fusion (like 50 documents). This way you get both the cross-encoder's accuracy and keep latency under control.

My practical recommendations:

• Keep the candidate pool reasonable. Reranking 50 documents instead of 100 nearly halves latency while usually lowering accuracy very little. Find this sweet spot with your own data.
• Be selective if budget/latency is critical. A reranker isn't mandatory for every query; for simple queries the RRF result may suffice. But remember, the 17.4% improvement the reranker adds is well worth this cost in most enterprise scenarios.
• Managed service or open source? Managed services like Cohere Rerank let you start fast; but since data leaves the organization, you must evaluate it from a KVKK perspective. Running open-source cross-encoders on your own servers is safer for organizations that want data sovereignty.

I don't impose all three stages on every project. You have to think at scale. But let me say clearly: I see hybrid search as the "minimum viable baseline" for almost every RAG deployment. Rather than starting with pure vector and then agonizing over "why isn't it working," adding BM25 from the start is far smarter.

Cases where hybrid + reranker is especially essential:

• Terminology-heavy fields: Law, medicine, finance, engineering. Here special terms, codes, and article numbers make BM25 indispensable.
• Mixed content: Documents containing text + tables + code. The 0.816 Recall@5 in the financial document example was obtained precisely in this kind of complexity.
• Scenarios requiring high accuracy: Fields where the wrong document has serious consequences, such as customer support, compliance, and medical information.

Conversely, if you have a small and homogeneous knowledge base, queries are simple, and latency is very critical, perhaps just hybrid (without a reranker) suffices. In engineering everything is a trade-off; but from what I've seen, most people regret moving to hybrid late, not early.

Now we come to the part you won't find in most English sources: Turkish. Turkish is an agglutinative language, and this directly affects both the BM25 and the embedding side.

On the BM25 side, tokenization and stemming are critical. If the words "ev" (house), "evler" (houses), "evimizde" (in our house), "evlerimizden" (from our houses) are counted as separate tokens with English logic, BM25 cannot match them with each other and you lose your exact-term-matching advantage. For Turkish you must use an analyzer that performs stemming or at least lemmatization. In engines like Elasticsearch/OpenSearch, the Turkish analyzer and tools like Zemberek do the job here. If you neglect this, all of BM25's power stays dull in Turkish.

Embedding model selection. An embedding model trained only on English captures meaning weakly in Turkish. Choose a multilingual embedding model or one trained specifically for Turkish. Don't put it into production without comparing a few models on a small validation set; in my experience, model choice affects the result far more in Turkish than in English.

Character normalization. Turkish's specific "ı/İ" and "i/I" upper-lower case conversion is a classic trap. A misconfigured lowercasing cannot match "İstanbul" with "istanbul." Use Turkish-sensitive normalization, or both BM25 and matching silently break.

In short, hybrid search is even more critical in Turkish: while stemming problems strain BM25, a good Turkish embedding carries the vector side; when you combine the two, you get a robustness neither could provide alone.

When building an in-house document RAG, the most frequently skipped topic is security and KVKK. Leaking personal data while chasing accuracy creates a problem far bigger than the one you solved. A few principles from the field.

Access control must be embedded in the retrieval stage. Your RAG system must not retrieve documents the querying user is not authorized to see. That is, you must add a filter (metadata filtering) to BM25 and vector search based on the user's access permissions. Otherwise an employee might see a quote from another department's confidential document in the answer to their question. Don't leave access control to the LLM; apply it at the retrieval layer.

PII masking. If the documents you index contain personal data (national ID, phone, address, health information), consider masking them for KVKK purposes or at least managing them with purpose limitation and access restriction. When sending the reranker to a managed external service, check whether there is exposed PII in the text going to that service; if data crosses the border, you need a notice and a legal basis.

Logging and traceability. Logging which query retrieved which documents is valuable for both debugging and KVKK accountability. But the logs themselves may contain personal data; define their retention periods and access.

Layer	Accuracy concern	KVKK concern
Retrieval	High recall	Access permission filter
Fusion (RRF)	Rank consistency	(neutral)
Reranking	High precision	Sending PII to external service
Context pruning	Token efficiency	Minimum data principle

When talking about hybrid search, most people jump straight to BM25 and vector; but perhaps the decision that most affects search quality is how you split your documents (chunking). What you can retrieve is limited by how you indexed it in the first place. With a bad chunking strategy, even the most advanced reranker can't save you.

The mistake I see often is blindly cutting documents at a fixed length (for example, every 500 words). This can split sentences in the middle, tear a table from its heading, and separate a clause from its context. As a result, both the vector embedding blurs and the meaningful unit for BM25 is lost.

My recommendations:

• Structure-respecting splitting. Where possible, split the document at natural boundaries such as headings, paragraphs, and clauses. If there's Markdown or HTML structure, use it; keep a heading and the paragraph beneath it in the same chunk.
• Leave overlap. A small overlap (a few sentences) between consecutive chunks prevents information at the boundary from being lost.
• Special treatment for tables. In financial or technical documents, don't crush the table into flat text; process it in a way that preserves the table's context (under which heading, which period). The high recall numbers in the text+table scenario are obtained precisely with this care.
• Tune chunk size to the task. Small chunks for short, precise answers; larger chunks work better for conceptual, broad questions. There is no single magic number; experiment with your own data.

Taking chunking seriously and fixing it once often yields a bigger gain than trying a new model. See it as a prerequisite.

In the hybrid architecture, most energy is spent after retrieval (fusion, rerank); but there is a lot to gain before retrieval too. The raw query the user types is often not in the ideal form the search needs.

A few practical techniques:

• Query expansion. By adding synonyms or related terms to the user's query, you can widen BM25's catch area. In Turkish this is especially useful, because the Turkish and foreign equivalent of the same concept (for example "bilgisayar" and "PC") can appear mixed in documents.
• Query rewriting. Especially in chat-based RAG, the user asks a context-dependent question like "and what about the price?" Converting this question into a self-contained query using the previous conversation dramatically improves retrieval quality.
• Multi-query retrieval. Producing a few variants from a single query, searching with each, and merging the results is naturally compatible with RRF; you're already merging rankings.

These steps are cheap but effective. The better the query entering the retrieval stage, the easier all subsequent stages become. Garbage in, garbage out; this holds all too well for RAG.

After understanding the theory, the most common question is: "So what do I build this with?" Let me suggest a concrete starting stack, but let me say up front: there is no single correct stack, what matters is choosing tools that support the three stages.

• Retrieval engine. Elasticsearch or OpenSearch is a natural start for hybrid because it offers both BM25 and vector search in one product. Turkish analyzer support is also mature. Alternatively, you can combine a separate vector database (like Qdrant, Weaviate, Milvus) with a separate text search engine.
• Fusion. Most modern search engines now offer RRF built in; if not, you can write it yourself in a few lines of code. The formula is simple, remember.
• Reranker. A managed service (like Cohere Rerank) for a fast start; running an open-source cross-encoder on your own infrastructure for organizations that want data sovereignty. If you have KVKK concerns, seriously consider the latter.

My recommendation is not to try to build the perfect architecture from day one. First build a baseline with BM25 only, measure. Then add vector, measure. Then RRF, then reranker. See the value each layer adds on your own data; this way you both avoid unnecessary complexity and justify your investment.

Over the years I have seen the same mistakes made again and again. To spare you from them, I leave a short checklist:

• Summing scores naked. Don't directly sum a BM25 score and a cosine similarity; the scales are different. Use RRF or proper normalization.
• English tokenization in Turkish. Indexing Turkish with the default analyzer silently cripples your BM25. A Turkish analyzer and stemming are essential.
• Applying the reranker to everything. Apply the cross-encoder only to the candidate pool, not the entire collection. Otherwise latency explodes.
• Going to production without measuring. If there's no golden set, you only feel that you've improved, you can't prove it.
• Leaving access control to the LLM. Apply the permission filter at the retrieval layer; the "we'll tell the model not to show it" approach is unacceptable from a KVKK standpoint.
• Leaving chunking for later. If your indexing decision is wrong, no subsequent layer can fully compensate.

I recommend reading this list once at the start of every new RAG project; because most of these mistakes can be prevented cheaply, while fixing them later comes at a high cost.

There is a topic as important as accuracy but constantly neglected: the latency budget. While each architectural layer gains you accuracy, it also adds some time. When a user in a chat interface expects the answer in two seconds, you can't stretch a three-stage pipeline to four seconds; otherwise even the most accurate answer loses because it arrives late.

In every RAG project I draw up a "latency budget" from the start. I set the total target time, then allocate it across the stages: retrieval, fusion, reranking, and LLM generation. Usually the two biggest items are the reranker and the LLM itself. Reducing the reranker's candidate pool from 100 to 50 often provides serious relief in latency while barely degrading accuracy; this is exactly the place to tune.

A practical balance table:

Setup	Relative accuracy	Relative latency	When?
BM25 only	Baseline	Lowest	Simple, term-focused queries
Vector only	Baseline	Low	Semantic, small knowledge base
Hybrid (RRF)	High	Medium	Most production scenarios
Hybrid + reranker	Highest	High	Fields requiring high accuracy

Fill this table with your own measurements; the real numbers vary by system. But the decision logic is always the same: is the accuracy gain worth the added latency? Aim for the best answer you can give without keeping the user waiting.

The first objection from executives is usually cost: "Two separate searches, plus a reranker; won't this cost us dearly?" Let's talk realistically.

The BM25 side is nearly free; it's already built into most search engines and cheap computationally. The vector side requires embedding generation (a one-time indexing cost) and vector storage; this scales with collection size but is manageable. The real variable cost is in the reranker: if you use a managed service, you pay a call fee proportional to the number of candidates in each query.

But also do this math: what is the cost of a RAG that retrieves the wrong document? A customer support bot that gives the wrong answer means a lost customer. A wrong citation in the compliance field means legal risk. The 17.4% recall improvement the reranker adds often easily amortizes itself in this context. My approach is this: see the reranker not as an expense but as insurance. But not blindly; manage the cost wisely by keeping the candidate pool reasonable and skipping it for simple queries when needed. A well-tuned hybrid system is the sweet spot between "do everything the most expensive way" and "don't do it at all."

Finally, the most neglected topic that makes the biggest difference: measurement. Most RAG projects that go to production with the sentence "the system seems better" can't understand six months later what broke and why. Establish an evaluation discipline from the start.

My concrete recommendation: create a small but representative "golden set"; that is, a validation set consisting of real user queries and the correct documents corresponding to them. With this set, regularly measure the following metrics:

• Recall@k: The rate at which relevant documents are captured within the first k results. Perhaps the most critical metric for RAG; because the LLM can only use the document that was retrieved.
• NDCG: Measures not only capture but also the quality of the ranking. Look at this if it matters that the good document is at the top.
• Latency: Monitor your p50 and p95 latency. When adding a reranker, you should see how these numbers change.

Re-measure these metrics with every architectural change (BM25 only, vector only, hybrid, hybrid+reranker). This way you answer the question "did hybrid really gain us X%?" with data, not feelings. My style of advancing in production is always this: start with a small golden set, measure every change, keep it only if the number improves. The on-paper benefit of hybrid search and reranking is impressive; but only your own measurement tells you how much it gains on your data. When you start your next RAG project, let the first thing you build be not the vector database but this evaluation loop.

Let me add one more point: measure the metrics not just once but at regular intervals. As new documents are added to your knowledge base, as user question patterns change, and as you update models, yesterday's optimum may not be today's optimum. On live systems I recommend running a small golden set through an automated test every week; when there's a silent drop in recall or NDCG, let you know before the user complains. Measurement is not a starting ritual but a continuous habit; only with this discipline can you preserve the true power of hybrid search.