By Sean Donahoe · Published July 16, 2026 · accurate as of this date
How do I evaluate an LLM router before I trust it?
Point a trial key at it and watch one real request. A router earns trust by what it puts on the response: which model answered, what the call cost, whether it swapped a model on you, and how few lines it takes to leave. Seven criteria below, each with a test you run yourself, most of them in five minutes flat. The ones that count are the ones a router can't fake in a live API response.
Every router sells you the same three words. One key. Many models. Cheaper. The pricing page can't tell you which one is telling the truth, because a pricing page will claim anything you let it.
A response header won't.
So here's the whole move. You don't read their docs and score the marketing. You point a trial key at the thing and interrogate the API directly, then you judge it on what it actually hands back about a single real request. Everything below is something you read off that response, or something you make a human answer out loud. If a router won't show you which model served your call and what it cost, it's asking for trust it hasn't earned. Doesn't matter whose logo is on it.
This test works the same against any OpenAI-compatible router. Flux, OpenRouter, LiteLLM, Portkey, a proxy your team hand-rolled last quarter. Run it against all of them. Run it against us. That's the point. New to the category and not sure what one of these things even is, start with what an LLM gateway actually is.
The one probe you run against all of them
You need exactly one thing to start: a trial key and the base URL. Almost every router speaks the OpenAI chat-completions shape, so a single snippet points at any of them with a base-URL swap and a Bearer key.
# Set these two per router you're testing.
ROUTER_URL="https://api.fluxrouter.ai/v1" # swap for any router's base URL
ROUTER_KEY="$FLUX_API_KEY" # a trial key for that router
curl -sD - "$ROUTER_URL/chat/completions" \
-H "Authorization: Bearer $ROUTER_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "flux-auto", "messages": [{"role": "user", "content": "Reply with the word ok."}]}' \
-o /dev/null
That's it. One non-streaming call. curl -sD - prints the response headers, which is where a router's receipts live. Change model to whatever the router you're testing calls its auto or default option; against Flux the base URL is https://api.fluxrouter.ai/v1 with a Bearer sk-... key from the dashboard. To read only Flux's documented headers, append | grep -iE '^x-flux-(model|original-model|routed|request-id|cost-usd): ' and score off those.
On Flux, that filter returns the five documented headers. Here's one response, from a probe run July 16, 2026. Routing is adaptive, so your identical call will usually name a different model at a different cost. That variation is the whole point, not a mistake:
X-Flux-Model: deepseek-v4-flash
X-Flux-Original-Model: flux-auto
X-Flux-Routed: true
X-Flux-Request-Id: <id>
X-Flux-Cost-Usd: 0.000180
Those five X-Flux-* headers are the documented, stable set, and they're the ones to build against. Five facts about a single request, on the request itself: which model answered, what you asked for, whether it rerouted you, a support handle, and what the call cost. Now the checklist is just reading that output, one criterion at a time, and asking a couple of questions the API can't answer for you.
The checklist: 7 criteria, 7 tests you run yourself
1. Receipts: does it name the model that answered?
The question. After a call, can you see the specific model that served it?
Why it matters. If a router can silently swap models behind your back, you can't reproduce a bug, audit a regression, or trust a benchmark you ran last week. The number you wrote down was for a model you can no longer prove you were talking to.
The test. Read the response header that names the served model. On Flux that's X-Flux-Model, plus X-Flux-Original-Model showing what you asked for.
Green: named on every response. Red: "trust us," or it only turns up in a dashboard a few hours later.
2. Cost visibility: what did THIS call cost, right now?
The question. Is per-request cost on the response, or only in a monthly total?
Why it matters. You can't cut what you only meet once a month. If cost shows up as one lump on an invoice, you've got no idea which route, which prompt, which feature is bleeding you. (More on why that opacity is expensive in Nobody knows what their AI actually costs. I won't re-argue it here.)
The test. Look for a per-request cost on the response. On Flux, X-Flux-Cost-Usd carries the dollar figure on non-streaming responses. One nuance worth knowing so your dashboards don't lie to you: on the OpenAI path the cost also rides in the response usage block, so streaming callers read it there rather than off the header (setting include_usage is the portable, OpenAI-documented way to guarantee that usage chunk). On the Anthropic-SDK path the usage block carries token counts only, no dollar figure, so if you stream through that path you price it from tokens yourself.
Green: cost stapled to the response, or a documented per-request usage record you can pull. Red: aggregate-only, "check the dashboard."
3. Provider spread and fallback: how many ways can this prompt be served?
The question. If the provider behind your model has a bad hour, is there a second path, automatically?
Why it matters. Your reliability is capped at your narrowest point. One upstream, one bad day, and your product is down with it. (One provider, one bad day is the long version.)
The test. Ask the vendor straight: "how many providers can serve model X, and what happens when one of them starts returning 500s?" Then check catalog breadth yourself with a GET /v1/models call on your key. On Flux that endpoint returns the live, authoritative list of ids your key can use, and Flux routes each request across many models and multiple providers so a bad upstream becomes a routing decision instead of an outage.
curl -s "$ROUTER_URL/models" -H "Authorization: Bearer $ROUTER_KEY"
Green: multiple providers per capability, automatic fallback described as behavior. Red: one upstream per model, and failover is a thing you do by hand at 3am.
4. Lock-in and exit cost: how many lines to leave?
The question. Is it a standard OpenAI-compatible base-URL swap, or a proprietary SDK you'd have to rip out?
Why it matters. A router you can't leave in an afternoon is a router that never has to keep earning you. The exit cost is the whole leverage. If leaving is a rewrite, they know it, and they'll act like they know it. (The DIY tax on the other side of that decision is Engineers as plumbers.)
The test. Count the lines it takes to switch in. Base URL, key, model string. That's the same count it takes to switch out. On Flux it's a drop-in for the OpenAI SDK: change the base URL, key, and model, and your endpoints, request JSON, and response JSON don't move. On the Anthropic SDK you point at https://api.fluxrouter.ai/anthropic with the standard client, no bespoke request shape.
Green: drop-in base URL, standard SDKs, your prompts unchanged. Red: a proprietary client, a custom request body, real re-tooling to get out.
5. Determinism when you need it: can you pin?
The question. When you need one exact model every single time, for a repro, a compliance requirement, a golden test, can you force it? And does the router tell you it obeyed?
Why it matters. Adaptive routing is great right up until the moment you need a fixed answer and can't get one. Pinning without a signal that the pin held is just a nicer-sounding version of the same black box.
The test. Pin a specific model, send a request, and read the response to confirm the router honored the pin instead of substituting. On Flux you pin a backing model with a flux-pinned-* id when you want determinism instead of flux-auto, and X-Flux-Routed: false on the response is the router's own signal that it didn't reroute you, paired with X-Flux-Model telling you exactly what served.
Green: pinning plus a signal on the response that the pin was honored. Red: no pin at all, or pins that quietly fall back with nothing to flag it.
6. Cost model legibility: is the price you pay published?
The question. Is the price you actually pay written down somewhere you can read it, and does a month of traffic reconcile to it?
Why it matters. A router whose economics you can't articulate is a router whose incentives you can't check. You don't need to audit their margin. You need the number you pay to be legible, in writing, and to match the invoice when the month closes.
The test. Find the published price you pay, then run real traffic for a day and reconcile the per-request costs you read off the response against what you actually got charged. On Flux, the price you pay is on a published pricing page and lands as one invoice across providers, so those X-Flux-Cost-Usd figures reconcile to the bill.
Green: a published price you pay that reconciles to the invoice. Red: "it's complicated," or a number you can't find in writing.
7. Data handling: where does your prompt go?
The question. Is your prompt and response data logged, retained, or trained on, and can you get the answer in writing?
Why it matters. It's the one criterion that's a lawyer's problem later if you wave it off now. The prompt you send might be the most sensitive thing your company hands to a third party all day.
The test. Ask for the retention, training, and processing-location policy, and get it in writing. This is the one place I'm telling you to ask us the exact same question you'd ask anyone else, and hold the answer to the same standard. Get specifics, not comfort.
Green: a written, specific answer you can file. Red: hand-waving, vibes, "we take privacy seriously."
How to score it
Run the probe against every router on your shortlist. Fill this in from what came back, not from what the site promised.
| Criterion | What a good answer looks like | Weight |
|---|---|---|
| 1. Names the model that answered | On every response, in a header | Highest |
| 2. Per-request cost | On the response, or a documented per-request record | Highest |
| 3. Provider spread and fallback | Many providers per capability, auto fallback | High |
| 4. Exit cost | Base-URL swap, standard SDK, prompts unchanged | High |
| 5. Pinning with an honored-pin signal | Pin holds, and the response says so | High |
| 6. Legible, published price | Published, reconciles to the invoice | Medium |
| 7. Data handling in writing | Specific retention and training policy | Gate it, don't average it |
Here's the meta-point, and it's the only scoring rule that really matters. Weight the criteria a router can't fake in a live response the heaviest. Criteria 1, 2, and 5 all get answered by the API itself, not by a salesperson. A header can't spin. The rest need a human to tell you the truth, which means the rest are only as good as the human. So lean your score on the un-fakeable ones, and treat criterion 7 as a gate: fail it, and the other six don't get to save the deal.
Hold us to it, too
I'm not going to publish a test and then dodge it.
Run this exact checklist against Flux. On a live response today, Flux stamps five documented headers: X-Flux-Model (what served), X-Flux-Original-Model (what you asked for), X-Flux-Routed (whether it rerouted you), X-Flux-Request-Id (your support handle), and X-Flux-Cost-Usd (per-request cost on non-streaming calls). It's a drop-in for the OpenAI SDK, with an Anthropic path at /anthropic. You pin a flux-pinned-* id when you need determinism, and GET /v1/models gives you the live catalog your key can use. The price you pay is published, and it's one invoice across providers. On criterion 7, ask us the same written question you'd ask anyone, and hold the answer to the same bar.
We published the test that could fail us because a router that hides the receipts isn't worth buying. Ours or anyone's. If the five documented headers on a live response today disagree with anything up there, trust the response and come tell us.
Grab a key from the quickstart, read the transparency headers doc so you know what you're reading off the response, and the models page covers the id scheme. Then run the probe. On us and on everyone else on your list. And once the router earns your trust, the sibling question is which model to run on it, which is how to actually choose an LLM.