Struct SemanticCache 

pub struct SemanticCache { /* private fields */ }

LRU-evicting semantic similarity cache.

Internally maintains an ordered list of keys (front = most recently used, back = least recently used) and a HashMap of entries. On lookup the matching key is moved to the front; on insert an LRU eviction is performed when the cache is full.

Implementations

impl SemanticCache

pub fn new(capacity: usize, similarity_threshold: f64) -> Self

Create a new SemanticCache with the given capacity and cosine similarity threshold in [0.0, 1.0].

pub fn lookup(&mut self, prompt: &str) -> Option<&str>

Look up prompt in the cache.

Returns Some(&str) with the stored response if any entry’s embedding has cosine similarity ≥ threshold with prompt’s embedding. The best match is returned and promoted to MRU position.

pub fn insert(&mut self, prompt: &str, response: String)

Insert a prompt/response pair into the cache.

If the cache is full the LRU entry is evicted first. If prompt already exists (exact key match) it is updated in place.

pub fn evict_lru(&mut self)

Remove the least recently used entry from the cache.

pub fn stats(&self) -> CacheStats

Return a statistics snapshot.