Direct answer
A 500 response is generated inside the responding service, but the trigger may still be a specific model or optional request feature. Reduce before retrying.
What this guide helps you solve
Isolate transient upstream failures, feature-specific server bugs, and gateway 500 responses safely.
Confirm the symptom and blast radius
The service returns structured or plain HTTP 500 after accepting the connection, often intermittently or only for one feature.
A minimal text request or another model may succeed while tools, images, structured output, or long context triggers the failure.
- Capture the status or exception, response body, request ID, UTC time, client version, and final host before changing configuration.
- Compare one minimal request with the failing workflow to determine whether the issue is global, model-specific, or feature-specific.
- Keep credentials, full prompts, customer data, and private file contents out of screenshots and support bundles.
Common causes and ownership boundaries
Treat the error as evidence from one layer, not as proof that every downstream component failed. These are the highest-value causes to test first.
- A transient internal failure in the gateway, model service, or dependency.
- An unsupported or malformed advanced feature reaches an unhandled server path.
- A specific model deployment is unhealthy while other routes remain available.
- Unbounded client retries amplify an existing partial outage.
Run a layered minimal diagnosis
Start with a request that has the fewest moving parts. Preserve the same account, API host, and target model while removing optional features.
Change one variable at a time and retain the raw status, headers, response body, and timing. This separates client serialization from gateway and upstream behavior.
- 01Establish the boundaryIdentify the responding host, content type, request ID, and UTC time.
- 02Create a baselineRemove tools, images, response formatting, streaming, and long context.
- 03Compare one variableCompare one known-good model on the same host and account.
- 04Record decisive evidenceStop batch traffic if the error rate rises and preserve one sanitized reproduction.
date -u
curl -sS -D /tmp/api-500.headers -o /tmp/api-500.body -w 'status=%{http_code} total=%{time_total}
' 'https://<your-api-host>/v1/models' -H "Authorization: Bearer ${API_KEY:?set API_KEY first}"
Apply the fix at the confirmed root cause
Make the smallest change that addresses the proven layer. Avoid hiding deterministic configuration failures behind broad retries or disabled validation.
- 01Correct the failing layerRetry only replay-safe requests with bounded exponential backoff.
- 02Restore required behaviorDisable or correct the isolated trigger until the service confirms support.
- 03Remove temporary workaroundsUse a verified fallback model only when its capability, cost, and data policy are acceptable.
Verify the repair, not just one successful call
Repeat the original path after the minimal check succeeds, then verify stable behavior under normal streaming, concurrency, and timeout conditions.
- Only replay-safe requests are retried and attempts remain bounded.
- Advanced fields are restored one at a time to identify the trigger.
- The fallback path has explicit capability and cost boundaries.
- The original model succeeds repeatedly without the same server error.
Security boundaries and escalation evidence
Collect enough evidence to reproduce and escalate the failure without exposing credentials or weakening transport, permission, and validation controls.
- Never replay a tool action or external write blindly after an ambiguous failure.
- Redact request bodies before sharing a reproduction.
- Do not disable validation to force a request through.
- Escalate with request IDs, model, endpoint, timing, and a minimal sanitized payload.
Official sources and verification scope
This guide is grounded in protocol specifications and official client documentation. Error text, retry headers, and configuration fields may change by service or client version; verify the sources and redact logs and request samples before sharing.
Read the verification methodology