Build a chatbot on Flux

A chatbot is a loop: you keep a list of messages, append the user's turn, send the whole list to the model, and append the reply. Flux is OpenAI-compatible, so you build this exactly the way you would against OpenAI. The only Flux-specific parts are the base URL (https://api.fluxrouter.ai/v1), your Flux key (sk-...), and the model (flux-auto).

Why flux-auto fits a chatbot#

The turns in a chat are not equally hard. "Hi", "thanks" and "what time is it" are easy. "Explain this stack trace" or "rewrite this function to be O(n)" are not. With flux-auto, Flux picks among vetted models for each kind of task, cost-aware, so you do not have to detect difficulty yourself or maintain your own routing rules. flux-auto bills at the Standard rate, or the Fast rate when it sends a turn to the Fast lane and a Fast model answers, and never more than Standard; see How billing works and Pricing.

If you need one consistent model for every turn (for example, to fix a tone or a behaviour), pin it with a flux-pinned-* id instead of flux-auto. See Models.

Minimal chat loop#

This is a complete terminal chatbot. It keeps the message history, streams each reply token by token, and adds the assistant's reply back to the history so the model has context on the next turn.

from openai import OpenAI

client = OpenAI(
    api_key="sk-...",                          # your Flux key
    base_url="https://api.fluxrouter.ai/v1",   # the one line you change
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
]

while True:
    user_input = input("you: ")
    if user_input.strip() in {"exit", "quit"}:
        break

    messages.append({"role": "user", "content": user_input})

    stream = client.chat.completions.create(
        model="flux-auto",
        messages=messages,
        stream=True,
    )

    print("bot: ", end="", flush=True)
    reply = ""
    for chunk in stream:
        delta = chunk.choices[0].delta.content or ""
        print(delta, end="", flush=True)
        reply += delta
    print()

    messages.append({"role": "assistant", "content": reply})

The same loop in TypeScript:

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.FLUX_API_KEY,            // your Flux key
  baseURL: "https://api.fluxrouter.ai/v1",     // the one line you change
});

const messages = [
  { role: "system", content: "You are a helpful assistant." },
];

messages.push({ role: "user", content: userInput });

const stream = await client.chat.completions.create({
  model: "flux-auto",
  messages,
  stream: true,
});

let reply = "";
for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content ?? "";
  process.stdout.write(delta);
  reply += delta;
}

messages.push({ role: "assistant", content: reply });

Stream for a responsive UI#

For anything user-facing, set stream: true. Streaming sends tokens as they are generated instead of waiting for the full reply, so the user sees text appear immediately. This is the single biggest perceived-latency win for a chatbot. For the full streaming setup in both formats, including server-sent events for a web app, see Streaming responses.

Keep the conversation in context#

The model has no memory between requests. You provide the memory by sending the full messages array every turn, including past user and assistant messages. Append each reply to the array as shown above. If conversations get long, trim or summarise older turns to stay within the model's context window and to control token cost, since you pay for the input tokens you send each turn.

Practical tips#

• Set a system message to fix the bot's role and tone. It is sent on every turn, so keep it concise.
• Show which model answered. Responses carry X-Flux-Model and X-Flux-Routed headers, so you can display or log which model served a turn. See Receipts.
• Handle rate limits. On a 429, back off and retry rather than dropping the turn.

Next steps#

• Streaming responses
• Models
• How routing works