Choosing how to buy and run Amazon Bedrock inference capacity

Two applications using the same model can have very different bills and reliability depending on how they consume it. An interactive assistant that throttles at peak hours fails its users however good its answers are; an overnight document job running at interactive rates pays for speed nobody needed. Choosing a consumption mode means matching how capacity is bought to the shape of the traffic.

The options differ on four axes: how you pay, whether you commit in advance, how requests are prioritized when capacity is tight, and where requests are processed. The table below compares them; current rates and discounts are published on the Amazon Bedrock pricing page¹.

On-demand use is limited by per-model quotas on tokens and requests per minute, plus a daily token quota. At the start of each request, Bedrock reserves the input tokens plus the request's max_tokens value against the quota, then adjusts to the actual output when the request finishes⁶. Setting max_tokens far above typical response length therefore reduces how many requests can run at once, even though you are billed only for the tokens actually used.

For some models, each output token counts several times against the quota through a burndown rate, so a quota sized on billed tokens can throttle earlier than expected⁶. Throttled requests should be retried with exponential backoff and jitter; persistent throttling is a signal to request a quota increase, spread load across Regions or move part of the traffic to another mode. The Priority, Standard and Flex tiers all share the same on-demand quota².

Batch inference suits jobs such as classifying an archive, summarizing a quarter's documents or generating evaluation outputs. Each request becomes a JSONL record in S3 in the InvokeModel or Converse format; you submit a job and collect results from S3 when it completes, with EventBridge notifying you of state changes instead of polling³.

Because each record is processed independently, batch supports neither tool calling nor structured output, and it cannot target provisioned models³. Prompt caching is unavailable for batch jobs too⁷. Keep batch prompts self-contained and validate the outputs after the job, since there is no interactive retry.

Provisioned Throughput buys model units, each delivering a set input and output token rate for one model, billed hourly whether or not it is used; longer commitments lower the hourly price⁴. The Provisioned Throughput documentation states that a customized model needs it⁴, while the model lifecycle page also describes on-demand deployments for custom models⁸, so confirm which applies to your base model before planning capacity.

The Reserved tier works differently: you reserve input and output tokens-per-minute capacity separately, and traffic above the reservation overflows to the Standard tier instead of failing². Both options make sense only once on-demand metrics show sustained demand, and both should be sized from measured peaks, including cache-write tokens, rather than from forecasts.