Direct answer
A 502 response means a proxy or gateway could not obtain a valid upstream response. The response headers and body often reveal which layer generated it.
What this guide helps you solve
Resolve reverse-proxy and upstream connectivity failures that return HTTP 502.
Confirm the symptom and blast radius
Requests receive 502 from a CDN, ingress, reverse proxy, or application gateway before a valid model response is returned.
Model discovery may work while generation or streaming fails because different upstream paths, timeouts, or buffering rules apply.
- Capture the status or exception, response body, request ID, UTC time, client version, and final host before changing configuration.
- Compare one minimal request with the failing workflow to determine whether the issue is global, model-specific, or feature-specific.
- Keep credentials, full prompts, customer data, and private file contents out of screenshots and support bundles.
Common causes and ownership boundaries
Treat the error as evidence from one layer, not as proof that every downstream component failed. These are the highest-value causes to test first.
- The upstream host, port, protocol, DNS record, SNI, or Host header is wrong.
- The upstream closes a reused connection or exceeds a proxy timeout.
- TLS verification or certificate name validation fails between proxy and upstream.
- Streaming responses are buffered, compressed, truncated, or treated as invalid.
Run a layered minimal diagnosis
Start with a request that has the fewest moving parts. Preserve the same account, API host, and target model while removing optional features.
Change one variable at a time and retain the raw status, headers, response body, and timing. This separates client serialization from gateway and upstream behavior.
- 01Establish the boundaryUse headers, branded HTML, and request IDs to identify the layer that generated 502.
- 02Create a baselineTest a lightweight model-list endpoint and one minimal non-streaming request.
- 03Compare one variableTest streaming only after the non-streaming baseline is stable.
- 04Record decisive evidenceCorrelate gateway and upstream logs by UTC time, connection metadata, and request ID.
curl -sS -D /tmp/api-502.headers -o /tmp/api-502.body -w 'remote=%{remote_ip} status=%{http_code} total=%{time_total}
' 'https://<your-api-host>/v1/models' -H "Authorization: Bearer ${API_KEY:?set API_KEY first}"
sed -n '1,30p' /tmp/api-502.headers
Apply the fix at the confirmed root cause
Make the smallest change that addresses the proven layer. Avoid hiding deterministic configuration failures behind broad retries or disabled validation.
- 01Correct the failing layerAlign upstream protocol, host, port, DNS, SNI, and Host settings.
- 02Restore required behaviorCalibrate connection-pool lifetime, connect timeout, read timeout, and idle timeout.
- 03Remove temporary workaroundsDisable inappropriate SSE buffering while preserving TLS and certificate validation.
Verify the repair, not just one successful call
Repeat the original path after the minimal check succeeds, then verify stable behavior under normal streaming, concurrency, and timeout conditions.
- The configured upstream identity and transport settings match the real service.
- Connection pools and timeout budgets remain stable under controlled concurrency.
- SSE events pass through incrementally without buffering or truncation.
- Bounded retries, circuit breaking, request IDs, and upstream metrics prevent amplification.
Security boundaries and escalation evidence
Collect enough evidence to reproduce and escalate the failure without exposing credentials or weakening transport, permission, and validation controls.
- Do not bypass upstream TLS verification as a permanent fix.
- Keep private upstream addresses and headers out of public reports.
- Restrict proxy trust and forwarded-header handling.
- Escalate with sanitized response headers, timestamps, request IDs, and both-layer status.
Official sources and verification scope
This guide is grounded in protocol specifications and official client documentation. Error text, retry headers, and configuration fields may change by service or client version; verify the sources and redact logs and request samples before sharing.
Read the verification methodology