Stream Agent Responses Without Corrupting State

Prerequisites

Before opening a stream, your persistence layer must distinguish an in-progress turn from a completed assistant message. At minimum, give each run and turn a stable ID plus a status such as streaming, completed, interrupted, or failed. If the existing message schema cannot represent those states, change it before streaming; the same compatibility concerns covered in agent state schema migrations apply here.

You also need named completion and failure events. A closed connection is not a substitute for either one.

Steps

1. Create an incomplete turn before forwarding any fragments

   Persist the turn ID and streaming status when the run starts, but do not create a final assistant message yet. Keep arriving text in a presentation buffer owned by the active turn.

   A streamed token is presentation data until the turn completes. It lets the reader see work in progress; it does not prove that the model or agent reached an answer. Persisting each fragment as final state erases that distinction. If the stream stops after a sentence that happens to end with a period, the database would otherwise contain something that looks complete even though the missing continuation might have changed its meaning.

   The OpenAI streaming responses guide identifies events by type and, as checked on 2026-08-26, distinguishes text deltas from lifecycle events including completion and error. Preserve that distinction in your own state model.

   An in-memory buffer is the simplest implementation, but a server restart loses it. If partial output must survive a restart, store it as a draft or append-only event record explicitly marked incomplete. That costs extra storage and cleanup work. It still must not appear in conversation history as a completed assistant turn.

2. Map provider events into separate application event types

   Put a small adapter between the provider stream and the rest of the application. Its output should be a discriminated event envelope, for example:

   • text.delta: presentation text for the active draft;
   • tool.call: a requested action and its structured arguments;
   • tool.result: the structured outcome of that action;
   • turn.completed: permission to commit the final assistant text;
   • turn.failed: a terminal failure with no completed assistant message.

   Tool calls and text deltas are different event types. Do not concatenate a tool name, argument JSON, tool result, or protocol frame into the text buffer. Text belongs in the visible draft. A tool call belongs in the execution path, where authorization, argument checking, execution, and result handling can occur without being mistaken for prose. Apply the checks from validating agent tool results before a result enters model context.

   This separation matters even when every transport message arrives as a string. The Cloudflare Agents WebSocket documentation notes, as checked on 2026-08-26, that connections can receive JSON text frames for identity, state, and protocol messages, and it exposes separate error and close handlers. A string frame is therefore only transport data until its event type has been decoded.

   The cost is an adapter per provider or stream format. The alternative—letting UI, tool execution, and persistence each interpret raw provider events—duplicates parsing rules and makes a provider change capable of corrupting all three paths.

3. Reduce events according to the turn status

   Process each decoded event through one state transition function. That function should enforce these rules:

   • text.delta may extend only the active draft of a streaming turn.
   • tool.call may update tool-run state but must not alter assistant text.
   • tool.result may complete that tool-run record but must not complete the assistant turn.
   • turn.completed may promote the assembled draft to final assistant content.
   • turn.failed may retain the draft as visibly incomplete or discard it according to your retention policy, but it may not promote it.

   Reject or ignore deltas received after a terminal state. If the transport can replay events, record an event ID or sequence cursor and make repeated delivery harmless. Otherwise a reconnect can duplicate text or execute the same tool call twice.

   This reducer adds lifecycle code that a simple append loop does not need. It is the right trade when the output affects durable conversation history or can trigger tools. For a throwaway terminal display with no persistence and no actions, a plain text append loop can be sufficient; it is the wrong answer once the stream writes state.

4. Treat a dropped connection as an incomplete turn

   Handle transport closure independently from agent completion. If the connection ends without turn.completed or turn.failed, move the client to reconnecting or interrupted. Never infer success from the last text delta, a clean socket close, or the absence of another event.

   For Server-Sent Events, the WHATWG Server-Sent Events specification defines reconnection behavior and the Last-Event-ID request header, as accessed on 2026-08-26. It also says an event left incomplete at end of stream is not dispatched. These transport rules can help resume delivery, but they do not declare the agent turn complete.

   For a WebSocket, route both error and close callbacks into the same incomplete-turn decision. The UI must leave its loading-only state and render one of two outcomes:

   • a terminal error when the server cannot continue the same turn; or
   • a resumable state when the server can continue from a known cursor.

   A spinner with no status or action is not failure handling. It silently freezes the interface and leaves the reader unable to tell whether the answer finished, failed, or is still running.

5. Define one explicit resume contract

   If resumption is supported, reconnect with the run ID, turn ID, and last accepted event cursor. The server must either replay events after that cursor or report that the turn can no longer be resumed. The client then deduplicates replayed events before updating the draft or tool-run state.

   If the backend cannot replay from a cursor, do not label a fresh request as a resume. Mark the first turn interrupted, keep its visible text labelled as partial if policy allows, and start a new turn for the retry. This prevents two attempts from being merged into one apparently continuous answer.

   Replay support costs durable event storage, cursor tracking, and cleanup. It suits long or tool-heavy runs where restarting would repeat meaningful work. A terminal failure with a retry action is simpler and suits short turns where the application does not retain stream events.

6. Commit final state only on the completion event

   When turn.completed arrives, assemble the accepted text deltas and commit the assistant message plus the turn’s completed status as one logical operation. Clear or archive the presentation draft only after that operation succeeds.

   Keep tool records separate even at commit time. They may be attached to the same run for audit and replay, but they are not assistant prose. This makes later agent run replay and debugging able to distinguish what the model displayed, which action the agent requested, what the tool returned, and whether the turn reached completion.

   If the final commit fails, the client should receive a terminal failure or remain resumable; it should not display a successful state merely because all expected text appeared. Persisting an event journal is compatible with this rule: individual events may be durable, but the assembled assistant message remains provisional until completion.

7. Test every interruption boundary

   Exercise the reducer and client with the stream cut before the first delta, midway through text, after a tool call but before its result, after a tool result but before more text, and after the last visible delta but before completion. Also test an explicit error, a duplicate event, and a resumed stream that replays the last accepted event.

   Each case should preserve the same invariants: tool data never appears in assistant text; an interrupted draft never becomes a completed message; a replay does not duplicate text or actions; and the client reaches a terminal error or resumable state instead of remaining in an indefinite loading state.

Expected result

Done means completed assistant messages exist only for turns that received an explicit completion event. Partial text may remain visible or durable, but it is labelled incomplete. Tool calls and results follow their own event path, and every premature transport ending produces either a terminal error or a resume action backed by a known turn and cursor.