Direct answer
A stream can fail after a successful HTTP handshake. Preserve event order and termination state so partial output is never treated as a complete result.
What this guide helps you solve
Fix truncated, buffered, timed-out, or incorrectly consumed SSE model streams.
Confirm the symptom and blast radius
Streaming starts and emits partial output, then stops without a valid completion event or finish reason.
The same request succeeds without streaming, or succeeds directly but fails through a proxy, CDN, SDK wrapper, or UI consumer.
- Capture the status or exception, response body, request ID, UTC time, client version, and final host before changing configuration.
- Compare one minimal request with the failing workflow to determine whether the issue is global, model-specific, or feature-specific.
- Keep credentials, full prompts, customer data, and private file contents out of screenshots and support bundles.
Common causes and ownership boundaries
Treat the error as evidence from one layer, not as proof that every downstream component failed. These are the highest-value causes to test first.
- A proxy buffers, compresses, or terminates SSE at an idle or response timeout.
- Event framing is altered, missing blank lines, or parsed with the wrong protocol assumptions.
- The client stops iterating, mishandles cancellation, or tears down listeners early.
- The upstream model or network closes the stream before a terminal event.
Run a layered minimal diagnosis
Start with a request that has the fewest moving parts. Preserve the same account, API host, and target model while removing optional features.
Change one variable at a time and retain the raw status, headers, response body, and timing. This separates client serialization from gateway and upstream behavior.
- 01Establish the boundaryRun the same model and input without streaming to establish a generation baseline.
- 02Create a baselineInspect raw Content-Type, event boundaries, data frames, and terminal signal.
- 03Compare one variableCompare direct and proxied paths for buffering, compression, and idle timeout.
- 04Record decisive evidenceTrace consumer iteration, cancellation, exceptions, and cleanup.
curl -N -sS --max-time 120 'https://<your-api-host>/v1/chat/completions' -H "Authorization: Bearer ${API_KEY:?set API_KEY first}" -H 'Content-Type: application/json' --data '{"model":"<YOUR_MODEL_ID>","stream":true,"messages":[{"role":"user","content":"reply with three short words"}]}'
Apply the fix at the confirmed root cause
Make the smallest change that addresses the proven layer. Avoid hiding deterministic configuration failures behind broad retries or disabled validation.
- 01Correct the failing layerConfigure proxies to pass SSE incrementally with an appropriate idle budget.
- 02Restore required behaviorParse the exact event format emitted by the selected API instead of mixing protocols.
- 03Remove temporary workaroundsHandle cancellation, partial output, exceptions, and listener cleanup explicitly.
started = false
completed = false
for event in stream:
started = true
validate_and_append(event)
if event_is_terminal(event): completed = true
if not completed:
mark_result_incomplete()
do_not_execute_follow_up_actions()Verify the repair, not just one successful call
Repeat the original path after the minimal check succeeds, then verify stable behavior under normal streaming, concurrency, and timeout conditions.
- Incremental events arrive in protocol order.
- Normal responses include a recognizable terminal event or finish reason.
- Cancellation releases upstream work, sockets, and listeners.
- Interrupted output is marked partial and never committed as a complete result.
Security boundaries and escalation evidence
Collect enough evidence to reproduce and escalate the failure without exposing credentials or weakening transport, permission, and validation controls.
- Do not log full streamed prompts or completions by default.
- Ensure cancellation does not leave privileged tools running.
- Apply output moderation and validation to partial as well as complete text.
- Escalate with sanitized event types, sequence, elapsed time, proxy path, and request ID.
Official sources and verification scope
This guide is grounded in protocol specifications and official client documentation. Error text, retry headers, and configuration fields may change by service or client version; verify the sources and redact logs and request samples before sharing.
Read the verification methodology