Skip to content
Skip to Content
Batchwell Docs

Request lifecycle

The five steps every chat completion goes through, from the key check to settlement.

Every request to POST /v1/chat/completions goes through the same five steps, in this order. If a request is refused before a model runs, it costs nothing.

Before any credit is held, the request is checked:

  • A missing, unknown or revoked key gets 401. Revoking a key takes effect everywhere within seconds. See authentication.
  • A request for a suspended or frozen organization gets 403.
  • A rate-limited request gets 429 with a Retry-After header. See rate limits.

We hold credit for your estimated input plus the full output you allow (max_tokens), priced at the dearest model in model and models. If that is more than you have, the request is refused and nothing runs:

  • 402 insufficient_credit: not enough available credit, or the key's spend limit would be exceeded.
  • 402 org_spend_limit_reached: your organization's monthly spend limit is reached.

See credits and billing and errors.

Only an endpoint (a deployment of the model) that is healthy, fits your context and output size, supports every parameter you sent and matches your data_collection choice is eligible. A request makes at most 4 attempts within one deadline. We retry only when the request provably did not start; otherwise we move to the next endpoint or fallback model, so a request is never run twice by accident. If nothing is eligible you get 503 no_eligible_endpoint. See routing.

Until output reaches you, a failure moves on to the next attempt or your next fallback model in models. Once the first token is delivered we stay on that model. A later failure ends the stream with an error event, then the usage chunk for what was delivered, then [DONE].

Every answer carries ai_generated: true in its batchwell object. See chat completions.

We charge the input plus the output delivered, at the price of the model that answered, never more than the hold, and release the rest. The charge is in usage.cost_micro (micro-USD), and every response carries a generation id in x-batchwell-generation-id, which you can look up later. See generation lookup and credits and billing.