A dashboard can report overspending perfectly while doing nothing to prevent it. Enforcement requires a decision before the expensive work begins, and that decision must remain valid when several requests arrive at once.
The race is easy to miss: two requests both read the same remaining allowance, both decide they can proceed, and together spend more than the tenant’s limit. A simple “check balance, then call model” sequence does not solve that problem.
Reserve capacity atomically
I model spendable allowance as the limit minus settled usage and active reservations. A request must reserve enough capacity through an atomic operation before it starts billable work.
For a bounded text-generation request, the reservation can account for estimated input cost and the configured maximum output. Other billable dimensions, such as image processing or additional tool steps, need their own explicit treatment.
The reservation is an application policy, not a guarantee about every provider’s billing behavior. Strict controls depend on bounded calls, known pricing, and a policy for uncertainty. If a call can create unbounded downstream work, no initial estimate makes it safe.
Keep admission and settlement separate
Admission decides whether a request may start. Settlement records what it actually consumed and releases any unused reservation. Both operations need stable identifiers so retries do not charge or release the same amount twice.
A simplified admission rule is:
available = limit - settled_usage - active_reservations
atomically:
if requested_reservation > available:
reject_or_degrade()
else:
create_reservation(request_id, requested_reservation)This pseudocode expresses the invariant. The production implementation still needs transactional updates, expiry handling, currency precision, and a durable request ledger.
Handle interrupted and uncertain calls
A disconnected client does not prove that the provider stopped computing. A timeout does not prove that the call was free. Immediately releasing every reservation on connection failure can undercount the work most likely to be retried.
I keep uncertain usage in a reconciliation state. Where provider usage records are available, they can settle it later. Otherwise the product needs a conservative accounting rule and visibility into how much usage is estimated.
Reservations also need recovery rules for crashed workers. Expiring a reservation should consider whether the external work might still be running; a lease timeout alone is not proof that a provider call ended.
Version prices and budget policy
A usage record should capture the model, billable quantities, pricing version, request identity, tenant, and task. Recalculating old usage using today’s price table makes historical reports unstable.
I avoid treating an unknown model as zero cost. Depending on the product, an unpriced model should be unavailable or receive a conservative reservation until its pricing is configured. Missing configuration should never silently expand spending permission.
The policy should also define whether a customer limit applies to provider cost, a commercial credit balance, or another usage unit. These are different concepts and should not share a misleading “balance” label.
Give the customer a predictable degraded path
A warning threshold can help a tenant act before reaching the limit. At the limit, the application might preserve human support, show a clear notice, or offer a lower-cost task that remains within policy.
Automatic fallback must pass the same admission check. Moving to another provider should not reset the tenant’s budget or create an independent allowance.
I test concurrent admissions, duplicate settlements, worker crashes, and unknown pricing. The important property is that the spending rule still holds under those conditions, rather than only during a single successful request.
Updated 25 September 2026.