Direct answer
A 500 response is generated inside the responding service, but the trigger may still be a specific model or optional request feature. Reduce before retrying.
What this guide helps you solve
Isolate transient upstream failures, feature-specific server bugs, and gateway 500 responses safely.
Symptoms
The service returns structured or plain HTTP 500 after accepting the connection, often intermittently or only for one feature.
A minimal text request or another model may succeed while tools, images, structured output, or long context triggers the failure.
Likely causes
- A transient internal failure in the gateway, model service, or dependency.
- An unsupported or malformed advanced feature reaches an unhandled server path.
- A specific model deployment is unhealthy while other routes remain available.
- Unbounded client retries amplify an existing partial outage.
Diagnostic procedure
- 01Establish the boundaryIdentify the responding host, content type, request ID, and UTC time.
- 02Create a baselineRemove tools, images, response formatting, streaming, and long context.
- 03Compare one variableCompare one known-good model on the same host and account.
- 04Record decisive evidenceStop batch traffic if the error rate rises and preserve one sanitized reproduction.
date -u
curl -sS -D /tmp/api-500.headers -o /tmp/api-500.body -w 'status=%{http_code} total=%{time_total}
' 'https://<your-api-host>/v1/models' -H "Authorization: Bearer ${API_KEY:?set API_KEY first}"
Remediation
- 01Correct the failing layerRetry only replay-safe requests with bounded exponential backoff.
- 02Restore required behaviorDisable or correct the isolated trigger until the service confirms support.
- 03Remove temporary workaroundsUse a verified fallback model only when its capability, cost, and data policy are acceptable.
Verification
- Only replay-safe requests are retried and attempts remain bounded.
- Advanced fields are restored one at a time to identify the trigger.
- The fallback path has explicit capability and cost boundaries.
- The original model succeeds repeatedly without the same server error.
Sensitive diagnostic data
- Never replay a tool action or external write blindly after an ambiguous failure.
- Redact request bodies before sharing a reproduction.
- Do not disable validation to force a request through.
- Escalate with request IDs, model, endpoint, timing, and a minimal sanitized payload.
Official sources and verification scope
This guide is grounded in protocol specifications and official client documentation. Error text, retry headers, and configuration fields may change by service or client version; verify the sources and redact logs and request samples before sharing.
View documentation scope