Streaming responses

Enable streaming

Set stream: true. The response uses text/event-stream instead of a single JSON object. The OpenAI SDK parses SSE for you. With cURL, add -N to disable output buffering.

curl -N https://bulktoken.ai/v1/chat/completions \
  -H "Authorization: Bearer $BULKTOKEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "glm-5.3",
  "messages": [
    {
      "role": "user",
      "content": "Explain how a hash table works in three sentences."
    }
  ],
  "stream": true
}'

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["BULKTOKEN_API_KEY"],
    base_url="https://bulktoken.ai/v1",
    timeout=120.0,
)

stream = client.chat.completions.create(
    model="glm-5.3",
    messages=[{"role": "user", "content": "Explain how a hash table works in three sentences."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.BULKTOKEN_API_KEY,
  baseURL: "https://bulktoken.ai/v1",
  timeout: 120_000,
});

const stream = await client.chat.completions.create({
  model: "glm-5.3",
  messages: [{ role: "user", content: "Explain how a hash table works in three sentences." }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

Read chunks safely

Read answer text from choices[0].delta.content. A chunk can contain a role, tool-call fragments, reasoning metadata, or usage instead of answer text. Check that choices is not empty before indexing it.

Reasoning models can emit reasoning before the answer. Do not assume every chunk contains content, and do not mark the request complete when you receive the first text.

Illustrative SSE frames

data: {"choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Usage and completion

Where supported, stream_options: {"include_usage": true} requests usage in the stream. A usage-only chunk can have an empty choices array. Route support can vary, so keep reading even when there is no answer delta and use Usage Logs to verify charges.

Inspect the finish reason. length indicates that the output budget was reached; tool_calls means your application needs to handle a tool request. Preserve a distinct interrupted or failed state when the stream does not complete normally.

Handle an interrupted stream

There is no offset-based resume operation for Chat Completions. A network failure or client disconnect does not prove the server stopped processing. Check Usage Logs before deciding to submit a new request; resubmission may produce a different answer and incur another charge.

For interactive products, show that the answer was interrupted and let the user explicitly retry. Apply the bounded retry guidance in Errors and retries.