Direct answer
Not every 429 is a short-lived rate limit. Read the error body and headers before retrying, and separate traffic limits from exhausted quota or balance.
What this guide helps you solve
Resolve API throttling and quota failures without creating retry storms or duplicate work.
Symptoms
Requests return 429 under bursts, long contexts, or parallel workers, sometimes with Retry-After or rate-limit headers.
A single request may also fail consistently when balance, project budget, account quota, or model entitlement is exhausted.
Likely causes
- Requests per minute, tokens per minute, or concurrent requests exceed a service limit.
- Balance, prepaid quota, project budget, or account credit is unavailable.
- Independent workers retry simultaneously without a shared limiter.
- Large prompts and output budgets consume token capacity faster than request counts suggest.
Diagnostic procedure
- 01Establish the boundaryClassify the error body as throttling, quota, budget, balance, or access.
- 02Create a baselineRecord RPM, TPM, concurrency, input size, output budget, and retry count.
- 03Compare one variableHonor Retry-After and rate-limit reset headers when present.
- 04Record decisive evidenceStop workers and send one request to test baseline account availability.
date -u
curl -sS -D /tmp/rate-headers.txt -o /tmp/rate-body.json 'https://<your-api-host>/v1/models' -H "Authorization: Bearer ${API_KEY:?set API_KEY first}"
sed -n '1,30p' /tmp/rate-headers.txt
Remediation
- 01Correct the failing layerUse a shared queue or limiter across workers, not per-process counters alone.
- 02Restore required behaviorApply exponential backoff with jitter, maximum attempts, and a total deadline.
- 03Remove temporary workaroundsReduce context and concurrency or restore the required balance, budget, and model entitlement.
for attempt in 0..4:
response = request()
if response.status != 429: return response
if response.error is quota_or_billing: stop_and_alert()
wait = retry_after ?? min(30s, 2^attempt + random_jitter)
sleep(wait)
raise rate_limit_exhaustedVerification
- All workers share one effective concurrency and rate budget.
- Retries have jitter, an attempt cap, and a total time cap.
- Balance, project budget, and model access are confirmed separately.
- Traffic ramps gradually without another spike in latency or 429 rate.
Sensitive diagnostic data
- Never log full prompts while measuring token load.
- Do not treat billing failures as transient retries.
- Use idempotency or task state to prevent duplicate side effects.
- Escalate with sanitized limit headers, workload metrics, UTC time, and request IDs.
Official sources and verification scope
This guide is grounded in protocol specifications and official client documentation. Error text, retry headers, and configuration fields may change by service or client version; verify the sources and redact logs and request samples before sharing.
View documentation scope