LeetLLM
MediumRate LimitingPython 3

Token Bucket Rate Limiter

Model per-key API quotas with deterministic refill math and caller-friendly retry-after decisions.

35m3 sample tests5 hidden tests

Implement a per-key token bucket limiter.

Requirements

  • Define Decision(allowed, retry_after, reason).
  • Define TokenBucketLimiter(capacity, refill_rate_per_second, now).
  • allow(key, cost=1) returns a Decision.
  • Each key has an independent bucket.
  • Tokens refill continuously based on elapsed time.
  • If a request is blocked, retry_after is the seconds until enough tokens are available.

Example

python
1clock = {"t": 0.0} 2limiter = TokenBucketLimiter(capacity=2, refill_rate_per_second=1, now=lambda: clock["t"]) 3assert limiter.allow("org").allowed 4assert limiter.allow("org").allowed 5blocked = limiter.allow("org") 6assert not blocked.allowed and blocked.retry_after == 1

Constraints

  • Use deterministic injected time.
  • Use standard-library Python only.

Editor