Direct answer
A timeout only identifies the deadline that expired. Measure each phase to avoid masking connection failures or slow generation with one oversized global timeout.
What this guide helps you solve
Fix API connect, read, first-byte, generation, proxy, and job timeouts without unsafe retries.
Confirm the symptom and blast radius
The client reports a timeout before connection, before first byte, during generation, or while reading a stream.
Short requests may work while long context, large output budgets, concurrency, or proxy hops exceed a shorter hidden deadline.
- Capture the status or exception, response body, request ID, UTC time, client version, and final host before changing configuration.
- Compare one minimal request with the failing workflow to determine whether the issue is global, model-specific, or feature-specific.
- Keep credentials, full prompts, customer data, and private file contents out of screenshots and support bundles.
Common causes and ownership boundaries
Treat the error as evidence from one layer, not as proof that every downstream component failed. These are the highest-value causes to test first.
- Slow DNS, TCP connection, TLS negotiation, or an unreachable network path.
- Model queueing or generation exceeds the client first-byte or read timeout.
- The SDK, reverse proxy, load balancer, and job runner use inconsistent deadlines.
- Connection-pool starvation delays a request before it reaches the network.
Run a layered minimal diagnosis
Start with a request that has the fewest moving parts. Preserve the same account, API host, and target model while removing optional features.
Change one variable at a time and retain the raw status, headers, response body, and timing. This separates client serialization from gateway and upstream behavior.
- 01Establish the boundaryMeasure DNS, TCP, TLS, first-byte, and total duration separately.
- 02Create a baselineUse one short non-streaming request to establish a baseline.
- 03Compare one variableCompare short and long input, output budgets, streaming, and controlled concurrency.
- 04Record decisive evidenceList every timeout in the SDK, proxy, gateway, and outer job; find the shortest one.
curl -sS -o /tmp/timeout-body.json --connect-timeout 10 --max-time 60 -w 'dns=%{time_namelookup} connect=%{time_connect} tls=%{time_appconnect} first=%{time_starttransfer} total=%{time_total} status=%{http_code}
' 'https://<your-api-host>/v1/models' -H "Authorization: Bearer ${API_KEY:?set API_KEY first}"
Apply the fix at the confirmed root cause
Make the smallest change that addresses the proven layer. Avoid hiding deterministic configuration failures behind broad retries or disabled validation.
- 01Correct the failing layerSet separate connect, read, and total deadlines based on observed service behavior.
- 02Restore required behaviorReduce context, output budget, and queue pressure before simply increasing timeouts.
- 03Remove temporary workaroundsAlign proxy and job deadlines so outer layers do not terminate healthy longer requests.
connect_timeout = 10s
read_or_idle_timeout = 90s
overall_job_deadline = 120s
retry_attempts = 2
# 数值仅为示意，应以真实模型延迟、代理限制和业务 SLO 校准。Verify the repair, not just one successful call
Repeat the original path after the minimal check succeeds, then verify stable behavior under normal streaming, concurrency, and timeout conditions.
- Connection failures exit quickly under a clear connect deadline.
- The read and total budget covers expected model latency without being unbounded.
- Connection pools remain available under controlled concurrency.
- Requests with side effects are not blindly replayed after an ambiguous timeout.
Security boundaries and escalation evidence
Collect enough evidence to reproduce and escalate the failure without exposing credentials or weakening transport, permission, and validation controls.
- Do not disable every timeout; stalled connections must remain bounded.
- Never include authorization data in timing traces.
- Use idempotency and task state before retrying tool or write operations.
- Escalate with phase timings, final host, model, payload size, and sanitized request IDs.
Official sources and verification scope
This guide is grounded in protocol specifications and official client documentation. Error text, retry headers, and configuration fields may change by service or client version; verify the sources and redact logs and request samples before sharing.
Read the verification methodology