Skip to content
Skip to Content
Batchwell Docs
Sign in

Routing

Fallback models with `models` and provider preferences with the `provider` object.

Fallback models

models lists up to 5 model ids to try, in order, if model cannot serve the request:

JSON
{
  "model": "batchwell/deepseek-v4.1-flash-uncensored",
  "models": ["batchwell/qwen3.6-72b-uncensored", "batchwell/llama-4.2-70b-uncensored"],
  "messages": [{ "role": "user", "content": "..." }]
}

Fallback happens only before any output reaches you. Once the first token is delivered we stay on that model; a later failure ends the stream with an error event. The credit hold uses the dearest model in the list, and you are charged the price of the model that answered (see x-batchwell-model).

The provider object

FieldTypeDefaultEffect
orderstring[]—Provider ids to try first, in this order.
onlystring[]—Use only these providers.
ignorestring[]—Never use these providers.
allow_fallbacksbooleantrueWhen false, do not move beyond order.
require_parametersbooleanfalseOnly providers that support every parameter you sent. Parameters are always checked against each endpoint, so this mostly documents intent.
data_collection"allow" | "deny""allow"deny excludes endpoints whose provider may retain prompts.
max_price{ prompt, completion }—USD per 1M tokens. Models whose published price is above this are skipped.
JSON
{
  "model": "batchwell/qwen3.6-72b-uncensored",
  "provider": { "data_collection": "deny", "max_price": { "prompt": 1, "completion": 3 } },
  "messages": [{ "role": "user", "content": "..." }]
}

How an endpoint is chosen

An endpoint is eligible only if it is enabled, fits your context and output size, supports every parameter in your request, matches your data_collection choice and is currently healthy. Among eligible endpoints we use your order first, otherwise our priority and live health. If nothing is eligible you get 503 no_eligible_endpoint.

A request makes at most 4 attempts within one deadline. We retry the same endpoint only when the provider confirms the request never started; otherwise we move to the next endpoint, so a request is never run twice by accident.

Streaming

Server-sent events, the final usage chunk, the in-band error event and keep-alives.

Errors

Error shape and the codes Batchwell returns.

On this page

Fallback modelsThe provider objectHow an endpoint is chosen