Struct RequestDeduplicator 

pub struct RequestDeduplicator { /* private fields */ }

Deduplicates identical in-flight LLM requests by their content hash.

All methods take &mut self, so wrap in a Mutex or RwLock for concurrent access.

Implementations

impl RequestDeduplicator

pub const TTL: Duration

TTL for in-flight entries.

pub fn new() -> Self

Create a new, empty deduplicator.

pub fn hash_key(model_id: &str, prompt_text: &str) -> String

Compute the SHA-256 deduplication key for a given model ID and prompt.

pub fn submit( &mut self, request_id: impl Into<String>, model_id: &str, prompt_text: &str, ) -> DedupDecision

Submit a request and decide whether it is the original or a duplicate.

Prunes stale entries (older than 30 s) before processing the new request so that a crashed original does not block future requests with the same key forever.

pub fn complete( &mut self, model_id: &str, prompt_text: &str, result: Vec<String>, ) -> usize

Signal that the original request identified by model_id + prompt_text has completed with the given result.

All registered waiters are notified. Returns the number of waiters that were fanned out to.

If no in-flight entry is found (e.g., it was already pruned), returns 0.

pub fn stats(&self) -> DedupStats

Return current statistics.

pub fn prune_stale(&mut self)

Prune in-flight entries older than Self::TTL.

Called automatically by submit; can also be called manually.

pub fn active_count(&self) -> usize

Return the number of currently active (in-flight) requests.

Trait Implementations

impl Default for RequestDeduplicator

fn default() -> Self

Returns the “default value” for a type.