Struct ModelRateLimiter 

pub struct ModelRateLimiter {
    pub model_id: String,
    pub total_throttled: AtomicU64,
    /* private fields */
}

Combined rate limiter for a single model.

Checks both the token bucket and the sliding window on every call to ModelRateLimiter::check_and_consume.

Fields

model_id: String

Model identifier.

total_throttled: AtomicU64

Cumulative count of requests that were throttled (either limiter).

Implementations

impl ModelRateLimiter

pub fn new(model_id: String, config: &RateLimiterConfig) -> Self

Create a limiter from a RateLimiterConfig.

pub fn check_and_consume(&self, tokens: u64) -> Result<(), RateLimitError>

Check whether a request consuming tokens may proceed.

On success, the tokens are deducted from the bucket and the request is recorded in the sliding window. On failure, total_throttled is incremented and the appropriate RateLimitError is returned.