Multi-Tenant SaaS'ta Cost Attribution: Aynı API Key ile 1000 Müşterinin Maliyetini Doğru Atfetmek

Problem: Hangi Müşteri Ne Harcatıyor?#

• 500 müşterin var
• Tek bir OpenAI hesabı
• Aylık toplam LLM faturası: $25.000
• "Acme Corp'un AI özelliği ne kadar maliyetli?" sorusunu bilemiyorsun

1. Pricing — bazı müşterilere "AI feature unlimited" vaat ettin. Gerçekten unlimited olunca, kâr ya da zarar mı?
2. Chargeback — kurumsal müşterilere "AI feature usage'ın $X" demek
3. Abuse detection — bir müşteri sistemini sömürüyorsa görmek

Çözüm#

Attribution Mimarisi — 4 Katman#

[HTTP Request] → [Middleware: tenant context]
                       ↓
[Service Layer] → [LiteLLM call with metadata]
                       ↓
[LiteLLM] → [Provider API + cost calc]
                       ↓
[Callback] → [Langfuse / ClickHouse with full attribution]

Adım 1 — Tenant Context Middleware#

FastAPI örneği#

from fastapi import FastAPI, Depends, Request
from contextvars import ContextVar

# Per-request context
current_tenant: ContextVar[str] = ContextVar("current_tenant", default="")
current_user: ContextVar[str] = ContextVar("current_user", default="")

app = FastAPI()

@app.middleware("http")
async def tenant_context_middleware(request: Request, call_next):
    # JWT'den tenant_id ve user_id çıkar
    auth = request.headers.get("Authorization", "")
    payload = decode_jwt(auth.replace("Bearer ", ""))

    current_tenant.set(payload.get("tenant_id", "anonymous"))
    current_user.set(payload.get("user_id", "anonymous"))

    response = await call_next(request)
    return response

Adım 2 — Servis Layer'da Metadata Injection#

from litellm import completion

def call_llm(
    messages: list,
    model: str = "claude-sonnet-4-6",
    feature: str = "default",
):
    """LLM call wrapper — otomatik attribution ekler."""
    return completion(
        model=model,
        messages=messages,
        metadata={
            "tenant_id": current_tenant.get(),
            "user_id": current_user.get(),
            "feature": feature,
            "trace_id": str(uuid.uuid4()),
            "timestamp": datetime.utcnow().isoformat(),
            # Ekstra business context
            "tenant_plan": get_tenant_plan(current_tenant.get()),
            "deploy_env": os.environ.get("ENV", "dev"),
        },
    )

Adım 3 — Callback'te Saklama#

def telemetry_callback(kwargs, response, start_time, end_time):
    metadata = kwargs.get("metadata", {})

    ch_client.insert("llm_telemetry.requests", [[
        datetime.utcnow(),
        metadata.get("trace_id"),
        metadata.get("tenant_id"),       # ← KEY için indexed
        metadata.get("user_id"),         # ← indexed
        metadata.get("feature"),         # ← indexed
        metadata.get("tenant_plan"),     # ← business segmentation
        response.usage.prompt_tokens,
        response.usage.completion_tokens,
        response._hidden_params["response_cost"],
        ...
    ]])

Adım 4 — Dashboard Segmentation#

"Top 10 maliyet üreten tenant"#

SELECT
    tenant_id,
    sum(cost_total_usd) AS monthly_cost,
    count() AS requests,
    avg(cost_total_usd) AS avg_per_request
FROM llm_telemetry.requests
WHERE ts >= toStartOfMonth(now())
GROUP BY tenant_id
ORDER BY monthly_cost DESC
LIMIT 10

"Plan'a göre ortalama maliyet"#

SELECT
    tenant_plan,
    count(DISTINCT tenant_id) AS tenant_count,
    sum(cost_total_usd) AS total_cost,
    sum(cost_total_usd) / count(DISTINCT tenant_id) AS avg_per_tenant
FROM llm_telemetry.requests
WHERE ts >= now() - INTERVAL 30 DAY
GROUP BY tenant_plan

Vaka: B2B Asistan SaaS#

Attribution'sız hayat#

• "Acme Corp şu özelliği çok kullanıyor sanırım" (tahmin)
• "AI feature kâr getiriyor mu?" (bilmiyorum)
• "Pro plan ile Enterprise plan margin'i nasıl?" (bilmiyorum)

Attribution'lı hayat#

TOP 5 müşteri:
  Acme Corp        : $3.200/ay (16% of total)
  Beta Industries  : $2.100/ay (10.5%)
  Gamma Holding    : $1.700/ay (8.5%)
  Delta Group      : $1.100/ay (5.5%)
  Epsilon LLC      : $890/ay   (4.4%)

Plan margin:
  Starter ($199/mo): avg $35 LLM cost   → 82% margin ✅
  Pro ($499/mo)   : avg $120 LLM cost   → 76% margin ✅
  Enterprise (custom): avg $450 LLM cost → varies — bireysel

LiteLLM Virtual Keys — Tenant-per-key#

Mimari#

Master API Key (gerçek OpenAI key)
    ↓
LiteLLM Proxy (yönetir, attribute eder, limit koyar)
    ├── Virtual Key 1 (acme-corp-key)
    ├── Virtual Key 2 (beta-industries-key)
    ├── Virtual Key 3 (gamma-holding-key)
    ...

• Kendi budget'ı (örn: Acme Corp aylık $5.000 limit)
• Kendi rate limit'i
• Kendi model whitelist'i (örn: Starter plan sadece GPT-5-mini)
• Otomatik attribution

Nasıl çalışır?#

# LiteLLM Proxy admin API ile virtual key yarat
import requests

resp = requests.post(
    "http://litellm-proxy/key/generate",
    headers={"Authorization": f"Bearer {LITELLM_MASTER_KEY}"},
    json={
        "key_alias": "acme-corp",
        "models": ["gpt-5-mini", "claude-haiku-4-5"],
        "max_budget": 5000,  # USD
        "budget_duration": "1mo",
        "tpm_limit": 100_000,
        "rpm_limit": 600,
        "metadata": {"tenant_id": "acme-corp", "plan": "pro"},
    },
)
new_key = resp.json()["key"]  # sk-...-acme-corp-spesifik