Skip to main content

Streaming

Get responses in real-time as they're generated.

Enable Streaming​

Get responses in real-time as they're generated. Set stream=True (Python) or stream: true (JavaScript):

# Enable streaming
stream = client.chat.completions.create(
  model="bizora-1.0",
  messages=[{"role": "human", "content": "What is section 179?"}],
  stream=True
)

# Process chunks as they arrive
for chunk in stream:
  if chunk.choices and chunk.choices[0].delta.content:
      print(chunk.choices[0].delta.content, end="", flush=True)

Multi-Turn Conversations with Streaming​

Include conversation history to provide context:

# Include conversation history
messages = [
  {"role": "human", "content": "What is section 179?"},
  {"role": "assistant", "content": "Section 179 allows businesses to deduct..."},
  {"role": "human", "content": "What are the requirements?"}
]

# Stream the response
stream = client.chat.completions.create(
  model="bizora-1.0",
  messages=messages,
  stream=True
)

for chunk in stream:
  if chunk.choices and chunk.choices[0].delta.content:
      print(chunk.choices[0].delta.content, end="", flush=True)

Response Format​

Each chunk contains incremental content:

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion.chunk",
  "created": 1677652288,
  "model": "bizora-1.0",
  "choices": [{
    "index": 0,
    "delta": {
      "content": "text chunk"
    },
    "finish_reason": null
  }]
}

The stream ends with:

data: [DONE]