Request lifecycle
The five steps every chat completion goes through, from the key check to settlement.
Every request to POST /v1/chat/completions goes through the same five steps, in this order. If a request is refused
before a model runs, it costs nothing.
Before any credit is held, the request is checked:
- A missing, unknown or revoked key gets
401. Revoking a key takes effect everywhere within seconds. See authentication. - A request for a suspended or frozen organization gets
403. - A rate-limited request gets
429with aRetry-Afterheader. See rate limits.
We hold credit for your estimated input plus the full output you allow (max_tokens), priced at the dearest model in
model and models. If that is more than you have, the request is refused and nothing runs:
402 insufficient_credit: not enough available credit, or the key's spend limit would be exceeded.402 org_spend_limit_reached: your organization's monthly spend limit is reached.
See credits and billing and errors.
Only an endpoint (a deployment of the model) that is healthy, fits your context and output size, supports every
parameter you sent and matches your data_collection choice is eligible. A request makes at most 4 attempts within
one deadline. We retry only when the request provably did not start; otherwise we move to the next endpoint or
fallback model, so a request is never run twice by accident. If nothing is eligible you get 503 no_eligible_endpoint. See
routing.
Until output reaches you, a failure moves on to the next attempt or your next fallback model in models. Once the
first token is delivered we stay on that model. A later failure ends the stream with an
error event, then the usage chunk for what was delivered, then [DONE].
Every answer carries ai_generated: true in its batchwell object. See
chat completions.
We charge the input plus the output delivered, at the price of the model that answered, never more than the hold, and
release the rest. The charge is in usage. (micro-USD), and every response carries a generation id in
x-batchwell-generation-id, which you can look up later. See generation lookup and
credits and billing.