Skip to content

Observability: Logging, Tracing, LangSmith / Langfuse

Production LLM gözlemlenebilirliği: structured logs, distributed tracing, anomaly detection. LangSmith, Langfuse, Helicone karşılaştırması.

Şükrü Yusuf KAYA
11 min read
Advanced

LLM Observability

Ne Loglanmalı?#

Her LLM çağrısı için:
json
{
  "request_id": "req_abc123",
  "trace_id": "trace_xyz",
  "user_id": "user_456",
  "session_id": "sess_789",
  "timestamp": "2026-05-10T14:23:11Z",
  "prompt_name": "customer_support",
  "prompt_version": "v3.1",
  "prompt_hash": "ab12cd34",
  "model": "claude-sonnet-4-6",
  "input": "...",
  "output": "...",
  "input_tokens": 850,
  "output_tokens": 120,
  "cost_usd": 0.0042,
  "latency_ms": 1834,
  "ttft_ms": 380,
  "cache_hit": true,
  "tools_used": ["search", "calculator"],
  "feedback": null,
  "error": null,
  "metadata": {
    "intent": "order_status",
    "confidence": 0.94,
    "escalation_triggered": false
  }
}
LLM trace JSON

Tools Karşılaştırması#

Production-grade. LangChain/LangGraph entegrasyonu hazır.
✅ Trace UI mükemmel ✅ Eval, dataset, A/B integrated ❌ Vendor lock-in (LangChain ekosistemine bağlı) ❌ Pricing (production'da hızlı toplanır)

Critical Alarmlar#

yaml
alerts:
  - name: "Cost Spike"
    condition: "hourly_cost > 1.5x daily_avg"
    action: page_oncall
 
  - name: "Latency Degradation"
    condition: "p95_latency > 5000ms for 5min"
    action: slack #incidents
 
  - name: "Error Rate"
    condition: "error_rate > 5% for 10min"
    action: page_oncall + auto_failover
 
  - name: "Hallucination Suspect"
    condition: "ungrounded_responses > 10/hour"
    action: slack #ml-quality
 
  - name: "Prompt Injection Attempt"
    condition: "injection_classifier_positive > 5/min"
    action: slack #security
Production alarm config

Yorumlar & Soru-Cevap

(0)
Yorum yazmak için giriş yap.
Yorumlar yükleniyor...

Related Content

1. Temeller — Yapay Zekâ ve LLM'lere Giriş

Bu Eğitim Hakkında ve Verimli Çalışma Yöntemi

Start Learning
1. Temeller — Yapay Zekâ ve LLM'lere Giriş

Yapay Zekâ → Üretken AI → LLM: Bağlamsal Harita

Start Learning
1. Temeller — Yapay Zekâ ve LLM'lere Giriş

LLM'ler Aslında Nasıl Düşünür? (Token, Embedding, Attention)

Start Learning

Connected pillar topics

Pillar topics this article maps to

Pillar Topic

LLMOps: Production-Grade LLM Operations

LLMOps is the engineering discipline that covers the development, deployment, monitoring, evaluation and cost management of LLM-powered applications — extending classic MLOps with prompt versioning, eval-driven CI and observability tailored for non-deterministic systems.