Full Telemetry Tools Comparison: Langfuse vs Helicone vs LangSmith vs Phoenix vs OTel

1️⃣ Langfuse — Açık Kaynak, En Esnek#

Feature seti#

• ✅ Tracing (full request/response capture)
• ✅ Cost tracking (provider-aware pricing built-in)
• ✅ Prompt management (versioning, A/B test)
• ✅ Eval framework (LLM-as-judge, custom scorers)
• ✅ Dataset & playground
• ✅ Sessions & users (cost attribution)
• ✅ Self-host (Docker Compose)
• ❌ Built-in alerting (Grafana/Slack webhook gerekli)

Fiyatlandırma#

Entegrasyon#

from langfuse.openai import openai  # drop-in replace
# veya
from langfuse import Langfuse
langfuse = Langfuse()

@langfuse.trace()
def my_function(...):
    ...

Türkiye için#

2️⃣ Helicone — Proxy Tabanlı, Sıfır Kod#

Feature seti#

• ✅ Tracing (otomatik, proxy üzerinden)
• ✅ Cost tracking
• ✅ Caching (built-in semantic cache)
• ✅ Rate limiting
• ✅ Custom properties (cost attribution)
• ⚠️ Eval framework sınırlı
• ⚠️ Prompt management basit
• ✅ Self-host (Docker)

Fiyatlandırma#

Entegrasyon#

client = OpenAI(
    api_key="sk-...",
    base_url="https://oai.helicone.ai/v1",
    default_headers={
        "Helicone-Auth": f"Bearer {HELICONE_KEY}",
        "Helicone-User-Id": "user_42",
        "Helicone-Property-Feature": "smart_search",
    },
)

Avantaj/Dezavantaj#

3️⃣ LangSmith — LangChain'in Resmi Observability'si#

Feature seti#

• ✅ Tracing (LangChain chain'leri otomatik yakalanır)
• ✅ Cost tracking
• ✅ Datasets & eval (LangChain ecosystem)
• ✅ Prompt hub
• ✅ Annotation queue (human-in-the-loop eval)
• ❌ Self-host (henüz)
• ⚠️ LangChain dışı entegrasyon limited

Fiyatlandırma#

Avantaj/Dezavantaj#

4️⃣ Phoenix (Arize AI) — ML Observability Geçmişli#

Feature seti#

• ✅ Tracing (OTel-native)
• ✅ Cost tracking
• ✅ Eval framework (LLM-as-judge, embeddings drift)
• ✅ Datasets
• ✅ Self-host (Docker Compose)
• ❌ Prompt management zayıf
• ⚠️ Cloud version Arize'ın paid tier'ı

Fiyatlandırma#

• Phoenix open-source: $0, self-host
• Arize AX cloud: $0/free (limited), custom paid

Avantaj/Dezavantaj#

5️⃣ OpenTelemetry GenAI — Vendor-Neutral Standard#

Feature seti#

• ✅ Span semantics (gen_ai.* attributes)
• ✅ Vendor-neutral (Jaeger, Tempo, Datadog, Honeycomb, vs.)
• ✅ Cost tracking (manual span attribute)
• ❌ Built-in dashboard yok
• ❌ Eval framework yok
• ✅ Tüm major backend'lerle entegre

Entegrasyon#

from opentelemetry.instrumentation.openai import OpenAIInstrumentor
OpenAIInstrumentor().instrument()

# OpenAI çağrıları otomatik Jaeger/Tempo/Honeycomb'a span gönderir

Avantaj/Dezavantaj#

Karar Matrisi#

Bu kursun seçimi#

KVKK Uyumu — Hangi Araç Hangi Düzeyde?#

Türkiye kurumsal için pratik öneri#

1. Geliştirme/test: Langfuse cloud free tier (test data, prod data değil)
2. Production: Langfuse self-host (VPC içinde, Docker Compose) veya Vertex AI built-in observability (Google ekosistem)