Expand description
Token-bucket and sliding-window rate limiter per model.
Provides RateLimiterRegistry which holds a ModelRateLimiter for
each registered model. Each model limiter combines a TokenBucket
(burst-capable refilling bucket) with a SlidingWindow (rolling
request-count check) so that both instantaneous bursts and sustained
throughput are controlled.
Structs§
- Model
Rate Limiter - Combined rate limiter for a single model.
- Rate
Limiter Config - Configuration used when registering a model with the
RateLimiterRegistry. - Rate
Limiter Registry - Registry that holds one
ModelRateLimiterper model. - Sliding
Window - Rolling-window request counter.
- Token
Bucket - Refilling token bucket.
Enums§
- Rate
Limit Error - Errors returned when a rate-limit check fails.