Direct answer
A stream can fail after a successful HTTP handshake. Preserve event order and termination state so partial output is never treated as a complete result.
What this guide helps you solve
Fix truncated, buffered, timed-out, or incorrectly consumed SSE model streams.
Symptoms
Streaming starts and emits partial output, then stops without a valid completion event or finish reason.
The same request succeeds without streaming, or succeeds directly but fails through a proxy, CDN, SDK wrapper, or UI consumer.
Likely causes
- A proxy buffers, compresses, or terminates SSE at an idle or response timeout.
- Event framing is altered, missing blank lines, or parsed with the wrong protocol assumptions.
- The client stops iterating, mishandles cancellation, or tears down listeners early.
- The upstream model or network closes the stream before a terminal event.
Diagnostic procedure
- 01Establish the boundaryRun the same model and input without streaming to establish a generation baseline.
- 02Create a baselineInspect raw Content-Type, event boundaries, data frames, and terminal signal.
- 03Compare one variableCompare direct and proxied paths for buffering, compression, and idle timeout.
- 04Record decisive evidenceTrace consumer iteration, cancellation, exceptions, and cleanup.
curl -N -sS --max-time 120 'https://<your-api-host>/v1/chat/completions' -H "Authorization: Bearer ${API_KEY:?set API_KEY first}" -H 'Content-Type: application/json' --data '{"model":"<YOUR_MODEL_ID>","stream":true,"messages":[{"role":"user","content":"reply with three short words"}]}'
Remediation
- 01Correct the failing layerConfigure proxies to pass SSE incrementally with an appropriate idle budget.
- 02Restore required behaviorParse the exact event format emitted by the selected API instead of mixing protocols.
- 03Remove temporary workaroundsHandle cancellation, partial output, exceptions, and listener cleanup explicitly.
started = false
completed = false
for event in stream:
started = true
validate_and_append(event)
if event_is_terminal(event): completed = true
if not completed:
mark_result_incomplete()
do_not_execute_follow_up_actions()Verification
- Incremental events arrive in protocol order.
- Normal responses include a recognizable terminal event or finish reason.
- Cancellation releases upstream work, sockets, and listeners.
- Interrupted output is marked partial and never committed as a complete result.
Sensitive diagnostic data
- Do not log full streamed prompts or completions by default.
- Ensure cancellation does not leave privileged tools running.
- Apply output moderation and validation to partial as well as complete text.
- Escalate with sanitized event types, sequence, elapsed time, proxy path, and request ID.
Official sources and verification scope
This guide is grounded in protocol specifications and official client documentation. Error text, retry headers, and configuration fields may change by service or client version; verify the sources and redact logs and request samples before sharing.
View documentation scope