Stream responses

Streaming sends the model's output token by token as it is generated, instead of waiting for the whole response. Through Flux you stream the same way you would against OpenAI: set "stream": true on a standard chat completions request to https://api.fluxrouter.ai/v1/chat/completions. The only Flux-specific part is the base URL and the flux-auto model.

Stream a chat completion#

Add "stream": true to the request body. The server responds with a text/event-stream of data: lines, each carrying a chunk in OpenAI's chat.completion.chunk shape, and finishes with a data: [DONE] line.

curl#

curl https://api.fluxrouter.ai/v1/chat/completions \
  -H "Authorization: Bearer $FLUX_API_KEY" \
  -H "Content-Type: application/json" \
  -N \
  -d '{
    "model": "flux-auto",
    "stream": true,
    "messages": [
      { "role": "user", "content": "Write a haiku about routing." }
    ]
  }'

The -N flag disables curl's output buffering so you see chunks as they arrive. Each event looks like:

data: {"id":"...","object":"chat.completion.chunk","choices":[{"delta":{"content":"Routes"},"index":0}]}
data: {"id":"...","object":"chat.completion.chunk","choices":[{"delta":{"content":" flow"},"index":0}]}
data: [DONE]

Python (OpenAI SDK)#

The OpenAI SDK turns the event stream into an iterator. Point it at the Flux base URL and pass stream=True.

from openai import OpenAI

client = OpenAI(
    api_key="sk-...",                          # your Flux key
    base_url="https://api.fluxrouter.ai/v1",   # the one line you change
)

stream = client.chat.completions.create(
    model="flux-auto",
    stream=True,
    messages=[{"role": "user", "content": "Write a haiku about routing."}],
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)
print()

Node (OpenAI SDK)#

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.FLUX_API_KEY,            // your Flux key
  baseURL: "https://api.fluxrouter.ai/v1",     // the one line you change
});

const stream = await client.chat.completions.create({
  model: "flux-auto",
  stream: true,
  messages: [{ role: "user", content: "Write a haiku about routing." }],
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

Stream with the Anthropic shape#

The Anthropic-compatible base also streams. Set stream=True (or stream: true) against https://api.fluxrouter.ai/anthropic and read Anthropic's event types (content_block_delta and friends) exactly as you would calling Anthropic directly.

from anthropic import Anthropic

client = Anthropic(
    api_key="sk-...",                                # your Flux key
    base_url="https://api.fluxrouter.ai/anthropic",  # the one line you change
)

with client.messages.stream(
    model="flux-auto",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Write a haiku about routing."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
print()

Thinking summaries on reasoning streams#

On streaming requests served by reasoning models, Flux can emit a short thinking summary before the answer starts: a one-line subject describing what the model is working on, delivered as a reasoning_summary field on the chunk delta (choices[0].delta.reasoning_summary). It may arrive more than once as the summary is refined; treat the latest one as current. It is display metadata for progress UIs; ignore it if you only want the answer, which streams in delta.content as usual.

Cost on streaming responses#

On non-streaming responses Flux returns an X-Flux-Cost-Usd header with the request cost. On streaming responses that header is absent, because the final cost is only known after the headers have been sent.

On a stream, the cost arrives in the usage object instead: the final chunk that carries usage includes usage.cost_usd and usage.currency. With the OpenAI SDK, ask for that chunk with stream_options={"include_usage": True}. Either way, every request is metered and billed the same. See Receipts for the full list of headers.

Notes#

• Streaming and the model that serves the request are independent: flux-auto still routes the request, and the other X-Flux-* headers (model, request id) are present on the streamed response.
• Token-level streaming behaviour depends on the model you route to, but the wire format is the standard OpenAI (or Anthropic) event stream in every case.