Tüm roadmap'e dön
topiccore
Rate Limiting & Queue Management
Vendor rate limit + concurrent agent limit + queue with backpressure.
2 saat
Production agent sistemde 3 katman:
- Vendor rate limit (OpenAI/Anthropic) — RPM + TPM quota. Aşılırsa 429.
- Concurrent agent limit — eş zamanlı kaç agent çalışabilir (CPU/RAM/cost).
- Per-user rate limit — abuse prevention.
Pattern:
- Token-bucket algoritması (her user için)
- Redis-based distributed rate limiter
- Queue (BullMQ / Sidekiq / Temporal) backpressure ile
- 429 retry with exponential backoff
- Priority queue (paid user > free, real-time > async)