Direct answer
A 502 response means a proxy or gateway could not obtain a valid upstream response. The response headers and body often reveal which layer generated it.
What this guide helps you solve
Resolve reverse-proxy and upstream connectivity failures that return HTTP 502.
Symptoms
Requests receive 502 from a CDN, ingress, reverse proxy, or application gateway before a valid model response is returned.
Model discovery may work while generation or streaming fails because different upstream paths, timeouts, or buffering rules apply.
Likely causes
- The upstream host, port, protocol, DNS record, SNI, or Host header is wrong.
- The upstream closes a reused connection or exceeds a proxy timeout.
- TLS verification or certificate name validation fails between proxy and upstream.
- Streaming responses are buffered, compressed, truncated, or treated as invalid.
Diagnostic procedure
- 01Establish the boundaryUse headers, branded HTML, and request IDs to identify the layer that generated 502.
- 02Create a baselineTest a lightweight model-list endpoint and one minimal non-streaming request.
- 03Compare one variableTest streaming only after the non-streaming baseline is stable.
- 04Record decisive evidenceCorrelate gateway and upstream logs by UTC time, connection metadata, and request ID.
curl -sS -D /tmp/api-502.headers -o /tmp/api-502.body -w 'remote=%{remote_ip} status=%{http_code} total=%{time_total}
' 'https://<your-api-host>/v1/models' -H "Authorization: Bearer ${API_KEY:?set API_KEY first}"
sed -n '1,30p' /tmp/api-502.headers
Remediation
- 01Correct the failing layerAlign upstream protocol, host, port, DNS, SNI, and Host settings.
- 02Restore required behaviorCalibrate connection-pool lifetime, connect timeout, read timeout, and idle timeout.
- 03Remove temporary workaroundsDisable inappropriate SSE buffering while preserving TLS and certificate validation.
Verification
- The configured upstream identity and transport settings match the real service.
- Connection pools and timeout budgets remain stable under controlled concurrency.
- SSE events pass through incrementally without buffering or truncation.
- Bounded retries, circuit breaking, request IDs, and upstream metrics prevent amplification.
Sensitive diagnostic data
- Do not bypass upstream TLS verification as a permanent fix.
- Keep private upstream addresses and headers out of public reports.
- Restrict proxy trust and forwarded-header handling.
- Escalate with sanitized response headers, timestamps, request IDs, and both-layer status.
Official sources and verification scope
This guide is grounded in protocol specifications and official client documentation. Error text, retry headers, and configuration fields may change by service or client version; verify the sources and redact logs and request samples before sharing.
View documentation scope