Module enhanced 

Enhanced resilience and performance features for the pipeline.

This module groups production-grade primitives that operate at the nanosecond-to-microsecond range and are safe to use concurrently across multiple pipeline stages:

Sub-module	Purpose
cache	In-process LRU cache with TTL for inference results
circuit_breaker	Prevents cascading failures; opens on threshold, probes recovery with exponential backoff
dedup	Request coalescing — collapses byte-identical in-flight prompts
semantic_dedup	Near-duplicate detection using SimHash LSH — catches paraphrases with configurable Hamming-distance threshold; when a near-duplicate is found, logs the similarity score and optionally returns the cached result
priority	Four-level priority queue for request scheduling
rate_limit	Token-bucket rate limiter with configurable burst
retry	Exponential back-off with jitter and configurable attempt cap
smart_batch	Adaptive micro-batching with prefix-based similarity grouping
tournament	Multi-provider tournament: fan-out and return the best response

All types are Clone and Send + Sync, suitable for sharing across Tokio tasks.

Re-exports

pub use adaptive_timeout::AdaptiveTimeout;

pub use bulkhead::Bulkhead;

pub use bulkhead::BulkheadPermit;

pub use cache::cache_key;

pub use cache::CacheLayer;

pub use circuit_breaker::CircuitBreaker;

pub use circuit_breaker::CircuitStatus;

pub use dedup::dedup_key;

pub use dedup::DeduplicationResult;

pub use dedup::DeduplicationToken;

pub use dedup::Deduplicator;

pub use dedup::DEDUP_CANCELLED_SENTINEL;

pub use dlq_replay::DlqReplayScheduler;

pub use dlq_replay::ReplayEntry;

pub use priority::Priority;

pub use priority::PriorityQueue;

pub use priority::QueueStats;

pub use rate_limit::RateLimiter;

pub use retry::retry_if;

pub use retry::RetryPolicy;

pub use semantic_dedup::SemanticDeduplicator;

pub use semantic_dedup::SimHashFingerprint;

pub use smart_batch::BatchConfig;

pub use smart_batch::BatcherStats;

pub use smart_batch::SmartBatcher;

pub use tournament::FastestResponseScorer;

pub use tournament::KeywordDensityScorer;

pub use tournament::LongestResponseScorer;

pub use tournament::ResponseScorer;

pub use tournament::TournamentConfig;

pub use tournament::TournamentResult;

pub use tournament::TournamentRunner;

pub use tournament::TournamentStats;

Modules

adaptive_timeout

Adaptive request timeout based on a rolling P95 of recent durations.

bulkhead

Bulkhead pattern — isolates concurrent execution pools to prevent cascade failures.

cache

Caching Layer

circuit_breaker

Circuit Breaker

dedup

Request Deduplication

dlq_replay

Dead-Letter Queue Replay Scheduler

priority

Priority Queue

rate_limit

Rate Limiting

retry

Retry Logic

semantic_dedup

Semantic Request Deduplication

smart_batch

Smart Adaptive Batcher

tournament

Provider Tournament Mode