A named token bucket that jobs draw from to control throughput.

A job bound to a budget dispatches only when its defined cost can be debited from the budget. When it cannot, it waits — it is not failed, and it does not leave the queue. Two strategies decide how tokens are managed:

# A rate limit: 100 dispatches per minute.
Zizq.Budget.new!(
  key: "emails",
  allocation: 100,
  strategy: :time_based,
  duration: :timer.minutes(1)
)

# A concurrency limit: at most 3 running at once.
Zizq.Budget.new!(
  key: "stripe",
  allocation: 3,
  strategy: :while_in_flight
)

:duration is in milliseconds.

:time_based

Tokens accrue on a continuous drip rather than in fixed windows, so an empty bucket is half full after half the duration has passed. There is no fixed window boundary around which sudden spikes occur.

A bucket starts full, which means 100 per minute really does permit two hundred dispatches in the first minute before settling to its long-run rate.

:burst

Caps how many tokens the bucket may hold at once, and so caps that opening spike without changing the long-run rate, while also capping the natural burst that would be permitted if the budget had sat idle for its total duration:

Zizq.Budget.new!(
  key: "emails",
  allocation: 100,
  strategy: :time_based,
  duration: :timer.minutes(1),
  burst: 5
)

A :burst of 1 paces dispatches evenly with no overshoot at all. It also applies to a bucket that has gone unused: idling for an hour does not bank an hour's worth of tokens.

A burst above the allocation is meaningful too — it keeps filling across consecutive periods, so 200 on an allocation of 100 accrues over two periods, but no further.

:while_in_flight

Has no clock. Its tokens are released when a job stops running, so it is a limit on how much work is in flight rather than on how often work starts. It takes neither :duration nor :burst, and passing either raises rather than being ignored — a budget that reads as though it set a refill period but did not is worse than a rejected one.

Capacity

A job's cost has to fit inside capacity/1 — the burst where one is set, and the allocation otherwise. With a burst set it is the smaller number that decides whether a job can ever run, so a cost well within the allocation may still be refused.

Bindings

Budgets are shared resources and Jobs are bound to named budgets. Typically jobs will bind to just one budget to enforce a rate limit or concurrency limit on that job, but they may also bound to more than one budget, in which case all budgets must be satisfied before the job is dispatched by the server. Strategies can be freely mixed. For example, you have have a rate limit on some upstream system that you need to adhere to, while also requiring a concurrency control on one particular job that hits that system.