A/B + Shadow Traffic: Feature Flag + Canary 1%→5%→25% + Automated Rollback

1. Shadow Traffic Pattern#

Request → API Gateway
            ├── (primary 100%) → Production model v1
            └── (shadow 100%) → Candidate model v2  [response logged, not returned]

Compare:
  - Response similarity (semantic + exact)
  - Latency (P50, P95, P99)
  - Token usage
  - Error rate

2. Canary Rollout Reçetesi#

• Error rate > %2 (baseline 0.5%)
• P95 latency > 5s (baseline 2s)
• Thumbs-down rate > %10 (baseline 5%)
• Critical eval set regression > 3 puan

# Feature flag pseudo
if feature_flag("llm_model_v2", user_id, percent_rollout=5):
    response = call_model_v2(prompt)
else:
    response = call_model_v1(prompt)