Hybrid Search
Hybrid search combines two fundamentally different retrieval paradigms: keyword-based search (lexical) and semantic vector search (dense). Each approach has complementary strengths and weaknesses, and combining them produces a retrieval system that is more robust than either alone.
Keyword search excels at exact matches, handling rare terms, code snippets, product codes, and proper nouns. Semantic search captures meaning and handles synonyms, paraphrasing, and conceptual queries. Hybrid search merges both result sets into a single ranked list, giving users the best of both worlds.
info
The two retrieval paradigms use fundamentally different representations of text. Understanding their differences is key to designing an effective hybrid search system.
| Property | Dense (Vector Search) | Sparse (Keyword Search) |
|---|---|---|
| Representation | Fixed-size dense vector (e.g., 768 floats) | High-dimensional sparse vector (vocabulary size) |
| Semantic matching | Excellent — captures meaning, synonyms, paraphrases | Poor — requires exact or near-exact term overlap |
| Exact matching | Poor — can miss rare terms, IDs, codes | Excellent — exact term matching is the core mechanism |
| Out-of-vocabulary | Subword tokenization handles novel words | Unseen terms have zero representation |
| Query understanding | Handles ambiguous or conversational queries | Literal interpretation of query terms |
| Storage | Larger per vector (768+ floats) | Smaller per document (sparse) |
| Index size | O(nd) — grows linearly with dimensions | O(n * avg_terms) — grows with vocabulary |
note
BM25 (Best Matching 25) is the most widely used keyword retrieval algorithm. It extends TF-IDF with document length normalization and saturation, producing relevance scores that are more robust across varying document lengths.
| 1 | # BM25 implementation with rank_bm25 |
| 2 | from rank_bm25 import BM25Okapi |
| 3 | |
| 4 | corpus = [ |
| 5 | "RAG combines retrieval with generation", |
| 6 | "Vector databases store embeddings for search", |
| 7 | "BM25 is a keyword-based retrieval algorithm", |
| 8 | "Hybrid search merges dense and sparse results" |
| 9 | ] |
| 10 | |
| 11 | tokenized_corpus = [doc.lower().split() for doc in corpus] |
| 12 | bm25 = BM25Okapi(tokenized_corpus) |
| 13 | |
| 14 | # Search |
| 15 | query = "keyword search algorithm".lower().split() |
| 16 | scores = bm25.get_scores(query) |
| 17 | top_k = sorted(range(len(scores)), |
| 18 | key=lambda i: scores[i], reverse=True)[:3] |
| 19 | |
| 20 | print([corpus[i] for i in top_k]) |
| 21 | # ['BM25 is a keyword-based retrieval algorithm', |
| 22 | # 'Hybrid search merges dense and sparse results', |
| 23 | # 'RAG combines retrieval with generation'] |
best practice
RRF is the most common method for merging dense and sparse search results. It combines multiple ranked lists into a single ranking using the reciprocal of each document's rank position. RRF is simple, effective, and does not require score normalization between different retrieval methods.
| 1 | def reciprocal_rank_fusion( |
| 2 | result_lists: list[list[str]], |
| 3 | k: int = 60 |
| 4 | ) -> list[tuple[str, float]]: |
| 5 | """Combine multiple ranked lists using RRF.""" |
| 6 | scores = {} |
| 7 | |
| 8 | for rank_list in result_lists: |
| 9 | for rank, doc_id in enumerate(rank_list): |
| 10 | if doc_id not in scores: |
| 11 | scores[doc_id] = 0.0 |
| 12 | scores[doc_id] += 1.0 / (k + rank + 1) |
| 13 | |
| 14 | # Sort by RRF score descending |
| 15 | ranked = sorted(scores.items(), |
| 16 | key=lambda x: x[1], |
| 17 | reverse=True) |
| 18 | return ranked |
| 19 | |
| 20 | # Usage |
| 21 | dense_results = ["doc3", "doc1", "doc5", "doc2"] |
| 22 | sparse_results = ["doc1", "doc4", "doc3", "doc6"] |
| 23 | |
| 24 | fused = reciprocal_rank_fusion([dense_results, sparse_results]) |
| 25 | print(fused) |
| 26 | # [('doc1', 0.032), ('doc3', 0.031), ('doc4', 0.016), ...] |
Choosing the RRF Constant k
The k constant prevents high rankings from dominating. Lower k values (e.g., 10) give more weight to top-ranked documents; higher k values (e.g., 100) distribute weight more evenly across the ranking. The original BM25 authors recommend k=60 as a sensible default.
| 1 | # Effect of k on score distribution |
| 2 | # k=10: top-ranked doc gets 1/11 ≈ 0.09 |
| 3 | # k=60: top-ranked doc gets 1/61 ≈ 0.016 |
| 4 | # k=100: top-ranked doc gets 1/101 ≈ 0.01 |
| 5 | |
| 6 | # For high-precision needs, use lower k |
| 7 | # For high-recall needs, use higher k |
| 8 | # Tune on your validation set |
pro tip
Beyond simple fusion methods like RRF, learning-to-rank (LTR) uses machine learning to combine signals from multiple retrieval sources. A trained model learns optimal weighting of feature scores based on historical relevance judgments.
| 1 | # Feature engineering for LTR |
| 2 | def extract_features(query: str, doc: str, |
| 3 | dense_score: float, sparse_score: float): |
| 4 | return { |
| 5 | "dense_score": dense_score, |
| 6 | "sparse_score": sparse_score, |
| 7 | "query_length": len(query.split()), |
| 8 | "doc_length": len(doc.split()), |
| 9 | "exact_match_ratio": count_exact_matches(query, doc), |
| 10 | "overlap_ratio": len(set(query) & set(doc)) / len(set(query)), |
| 11 | "dense_normalized": normalize_score(dense_score), |
| 12 | "sparse_normalized": normalize_score(sparse_score), |
| 13 | } |
| 14 | |
| 15 | # Use a cross-encoder for final re-ranking |
| 16 | from sentence_transformers import CrossEncoder |
| 17 | |
| 18 | reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2") |
| 19 | |
| 20 | def rerank(query: str, candidates: list[str]) -> list[str]: |
| 21 | pairs = [(query, doc) for doc in candidates] |
| 22 | scores = reranker.predict(pairs) |
| 23 | ranked = sorted(zip(candidates, scores), |
| 24 | key=lambda x: x[1], reverse=True) |
| 25 | return [doc for doc, _ in ranked] |
warning
Query transformation improves hybrid search by reformulating the user's raw query before sending it to the retrieval systems. Common techniques include query expansion, decomposition, and rewriting using an LLM.
| 1 | from openai import OpenAI |
| 2 | |
| 3 | client = OpenAI() |
| 4 | |
| 5 | def transform_query(raw_query: str) -> dict: |
| 6 | """Generate multiple query representations for hybrid search.""" |
| 7 | |
| 8 | # 1. LLM rewriting for better semantic retrieval |
| 9 | rewrite_response = client.chat.completions.create( |
| 10 | model="gpt-4o-mini", |
| 11 | messages=[ |
| 12 | {"role": "system", "content": |
| 13 | "Rewrite the query for optimal search retrieval."}, |
| 14 | {"role": "user", "content": raw_query} |
| 15 | ] |
| 16 | ) |
| 17 | rewritten_query = rewrite_response.choices[0].message.content |
| 18 | |
| 19 | # 2. Extract keywords for BM25 |
| 20 | keywords_response = client.chat.completions.create( |
| 21 | model="gpt-4o-mini", |
| 22 | messages=[ |
| 23 | {"role": "system", "content": |
| 24 | "Extract 3-5 key search terms from the query."}, |
| 25 | {"role": "user", "content": raw_query} |
| 26 | ] |
| 27 | ) |
| 28 | keywords = keywords_response.choices[0].message.content |
| 29 | |
| 30 | return { |
| 31 | "original": raw_query, |
| 32 | "rewritten": rewritten_query, |
| 33 | "keywords": keywords |
| 34 | } |
| 35 | |
| 36 | # Use transformed queries |
| 37 | transformations = transform_query("How does RAG handle hallucinations?") |
| 38 | dense_results = vector_search(transformations["rewritten"], k=10) |
| 39 | sparse_results = bm25_search(transformations["keywords"], k=10) |
| 40 | fused = reciprocal_rank_fusion([dense_results, sparse_results]) |
best practice
Here is a complete hybrid search implementation using Qdrant, which supports both dense and sparse vectors natively.
| 1 | from qdrant_client import QdrantClient |
| 2 | from qdrant_client.models import ( |
| 3 | VectorParams, SparseVectorParams, |
| 4 | SparseIndexParams, Filter, PointStruct |
| 5 | ) |
| 6 | from qdrant_client.http import models |
| 7 | |
| 8 | client = QdrantClient(":memory:") # or remote URL |
| 9 | |
| 10 | # Create collection with both dense and sparse vectors |
| 11 | client.create_collection( |
| 12 | collection_name="hybrid_demo", |
| 13 | vectors_config={ |
| 14 | "dense": VectorParams(size=768, distance="Cosine") |
| 15 | }, |
| 16 | sparse_vectors_config={ |
| 17 | "bm25": SparseVectorParams( |
| 18 | index=SparseIndexParams() |
| 19 | ) |
| 20 | } |
| 21 | ) |
| 22 | |
| 23 | # Index documents with both representations |
| 24 | for doc_id, (text, dense_vec, sparse_vec) in enumerate(docs): |
| 25 | client.upsert("hybrid_demo", points=[ |
| 26 | PointStruct( |
| 27 | id=doc_id, |
| 28 | vector={ |
| 29 | "dense": dense_vec, |
| 30 | "bm25": sparse_vec |
| 31 | }, |
| 32 | payload={"text": text} |
| 33 | ) |
| 34 | ]) |
| 35 | |
| 36 | # Hybrid search with RRF fusion |
| 37 | results = client.query_points( |
| 38 | collection_name="hybrid_demo", |
| 39 | prefetch=[ |
| 40 | models.Prefetch( |
| 41 | query=dense_query_vec, |
| 42 | using="dense", |
| 43 | limit=20 |
| 44 | ), |
| 45 | models.Prefetch( |
| 46 | query=sparse_query_vec, |
| 47 | using="bm25", |
| 48 | limit=20 |
| 49 | ) |
| 50 | ], |
| 51 | query=models.FusionQuery( |
| 52 | fusion=models.Fusion.RRF # Reciprocal Rank Fusion |
| 53 | ), |
| 54 | limit=10 |
| 55 | ) |
pro tip
✗ Problem
Imbalanced fusion weightsIf one component dominates (e.g., dense scores are much higher), the weaker component's results never appear in the top-k. Normalize scores or use rank-based fusion.
✗ Problem
Ignoring latency impactRunning two searches doubles latency. Use the faster system first, then re-rank with the slower one, or run in parallel and fuse.
✗ Problem
Not tuning per componentEach component (BM25 params, vector index, RRF k) should be tuned independently. Joint tuning of all parameters is prohibitively expensive.