Direct answer
Not every 429 is a short-lived rate limit. Read the error body and headers before retrying, and separate traffic limits from exhausted quota or balance.
What this guide helps you solve
Resolve API throttling and quota failures without creating retry storms or duplicate work.
Confirm the symptom and blast radius
Requests return 429 under bursts, long contexts, or parallel workers, sometimes with Retry-After or rate-limit headers.
A single request may also fail consistently when balance, project budget, account quota, or model entitlement is exhausted.
- Capture the status or exception, response body, request ID, UTC time, client version, and final host before changing configuration.
- Compare one minimal request with the failing workflow to determine whether the issue is global, model-specific, or feature-specific.
- Keep credentials, full prompts, customer data, and private file contents out of screenshots and support bundles.
Common causes and ownership boundaries
Treat the error as evidence from one layer, not as proof that every downstream component failed. These are the highest-value causes to test first.
- Requests per minute, tokens per minute, or concurrent requests exceed a service limit.
- Balance, prepaid quota, project budget, or account credit is unavailable.
- Independent workers retry simultaneously without a shared limiter.
- Large prompts and output budgets consume token capacity faster than request counts suggest.
Run a layered minimal diagnosis
Start with a request that has the fewest moving parts. Preserve the same account, API host, and target model while removing optional features.
Change one variable at a time and retain the raw status, headers, response body, and timing. This separates client serialization from gateway and upstream behavior.
- 01Establish the boundaryClassify the error body as throttling, quota, budget, balance, or access.
- 02Create a baselineRecord RPM, TPM, concurrency, input size, output budget, and retry count.
- 03Compare one variableHonor Retry-After and rate-limit reset headers when present.
- 04Record decisive evidenceStop workers and send one request to test baseline account availability.
date -u
curl -sS -D /tmp/rate-headers.txt -o /tmp/rate-body.json 'https://<your-api-host>/v1/models' -H "Authorization: Bearer ${API_KEY:?set API_KEY first}"
sed -n '1,30p' /tmp/rate-headers.txt
Apply the fix at the confirmed root cause
Make the smallest change that addresses the proven layer. Avoid hiding deterministic configuration failures behind broad retries or disabled validation.
- 01Correct the failing layerUse a shared queue or limiter across workers, not per-process counters alone.
- 02Restore required behaviorApply exponential backoff with jitter, maximum attempts, and a total deadline.
- 03Remove temporary workaroundsReduce context and concurrency or restore the required balance, budget, and model entitlement.
for attempt in 0..4:
response = request()
if response.status != 429: return response
if response.error is quota_or_billing: stop_and_alert()
wait = retry_after ?? min(30s, 2^attempt + random_jitter)
sleep(wait)
raise rate_limit_exhaustedVerify the repair, not just one successful call
Repeat the original path after the minimal check succeeds, then verify stable behavior under normal streaming, concurrency, and timeout conditions.
- All workers share one effective concurrency and rate budget.
- Retries have jitter, an attempt cap, and a total time cap.
- Balance, project budget, and model access are confirmed separately.
- Traffic ramps gradually without another spike in latency or 429 rate.
Security boundaries and escalation evidence
Collect enough evidence to reproduce and escalate the failure without exposing credentials or weakening transport, permission, and validation controls.
- Never log full prompts while measuring token load.
- Do not treat billing failures as transient retries.
- Use idempotency or task state to prevent duplicate side effects.
- Escalate with sanitized limit headers, workload metrics, UTC time, and request IDs.
Official sources and verification scope
This guide is grounded in protocol specifications and official client documentation. Error text, retry headers, and configuration fields may change by service or client version; verify the sources and redact logs and request samples before sharing.
Read the verification methodology