Embedding-Based Selection: Bağlamdan İlişkisiz Parçaları Atmanın En Pratik Yolu

Pattern — Retrieved'in Retrieved'i#

[User Query]
     ↓
[Vector DB search] → top-20 chunks (RAW)
     ↓
[Embedding-based selection] → top-5 most relevant
     ↓
[LLM call with compact context]

Implementation#

import numpy as np
from openai import OpenAI

client = OpenAI()

def embed(text: str) -> np.ndarray:
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=text,
    )
    return np.array(response.data[0].embedding)

def cosine_similarity(a: np.ndarray, b: np.ndarray) -> float:
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

def select_relevant_chunks(
    query: str,
    chunks: list[str],
    top_k: int = 5,
    threshold: float = 0.6,
) -> list[str]:
    query_emb = embed(query)
    scored = []
    for chunk in chunks:
        chunk_emb = embed(chunk)
        score = cosine_similarity(query_emb, chunk_emb)
        scored.append((chunk, score))

    # Top-K + threshold filter
    scored.sort(key=lambda x: x[1], reverse=True)
    filtered = [c for c, s in scored if s >= threshold][:top_k]
    return filtered

# Usage
all_chunks = vector_db_search(query, top_k=20)  # geniş retrieve
selected = select_relevant_chunks(query, all_chunks, top_k=5, threshold=0.6)

context = "\n\n".join(selected)
response = llm(query, context)

Threshold Seçimi — Tasarruf vs Recall#

• Çok düşük (0.3): hepsi geçer, tasarruf az
• Çok yüksek (0.8): çok azı geçer, kritik bilgiyi kaçırırsın

Doğru threshold'u bulma#

def find_optimal_threshold(eval_set):
    """50 test sorusunda farklı threshold'ları test et."""
    thresholds = [0.4, 0.5, 0.6, 0.7, 0.8]
    results = {}
    for t in thresholds:
        accuracies = []
        for query, expected_chunks in eval_set:
            selected = select_relevant_chunks(query, all_chunks, threshold=t)
            recall = len(set(selected) & set(expected_chunks)) / len(expected_chunks)
            accuracies.append(recall)
        results[t] = np.mean(accuracies)
    return results

# Output
# 0.4: recall 0.95, avg_chunks 18
# 0.5: recall 0.92, avg_chunks 12
# 0.6: recall 0.86, avg_chunks 7   ← sweet spot
# 0.7: recall 0.71, avg_chunks 4   ← çok agresif
# 0.8: recall 0.45, avg_chunks 2

Reranking vs Selection — Fark#

Birlikte kullan#

# 1. Geniş retrieve (top-20)
chunks = vector_db.search(query, k=20)

# 2. Rerank (Cohere Rerank veya Voyage rerank)
reranked = cohere_rerank(query, chunks, top_n=10)

# 3. Embedding selection (top-K with threshold)
selected = select_relevant_chunks(query, reranked, top_k=5, threshold=0.6)

# 4. LLM call

LLM-as-Judge ile Doğrulama#

def is_chunk_relevant(query: str, chunk: str) -> bool:
    """Bir küçük LLM (Haiku) ile chunk-relevance check."""
    response = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=10,
        messages=[{
            "role": "user",
            "content": f"Soru: {query}\nMetin: {chunk}\n\nBu metin sorunun cevabına katkı yapar mı? Sadece 'YES' veya 'NO' yaz."
        }],
    )
    return "YES" in response.content[0].text.upper()

# Validation
for chunk in selected:
    if not is_chunk_relevant(query, chunk):
        print(f"⚠️ Selected ama relevant değil: {chunk[:80]}")

for chunk in all_chunks:
    if chunk not in selected and is_chunk_relevant(query, chunk):
        print(f"⚠️ Atıldı ama relevant idi: {chunk[:80]}")

Gerçek Etki — Bir Vaka#

Önce (selection yok)#

• top-10 chunks retrieve
• Ortalama 8K context token
• $/istek:$0.024 (Sonnet 4.6)
• 100K istek/ay = $2.400

Sonra (embedding selection)#

• top-20 retrieve, top-5 select (threshold 0.6)
• Ortalama 3.2K context token
• $/istek:$0.011
• 100K istek/ay = $1.100