pub struct CacheWarmer {
pub patterns: Vec<WarmingPattern>,
pub strategy: WarmingStrategy,
/* private fields */
}Expand description
Orchestrates cache pre-warming across a set of WarmingPatterns.
Fields§
§patterns: Vec<WarmingPattern>Registered patterns.
strategy: WarmingStrategyActive strategy.
Implementations§
Source§impl CacheWarmer
impl CacheWarmer
Sourcepub fn new(strategy: WarmingStrategy) -> Self
pub fn new(strategy: WarmingStrategy) -> Self
Create a new warmer with the given strategy and no patterns.
Sourcepub fn add_pattern(&mut self, pattern: WarmingPattern)
pub fn add_pattern(&mut self, pattern: WarmingPattern)
Register a new pattern.
Sourcepub fn generate_jobs(&self) -> Vec<WarmingJob>
pub fn generate_jobs(&self) -> Vec<WarmingJob>
Generate warming jobs according to the current strategy.
Jobs are appended to the internal queue and also returned to the caller.
Sourcepub fn mark_completed(&self, job_id: u64)
pub fn mark_completed(&self, job_id: u64)
Mark a job as successfully completed.
Sourcepub fn mark_failed(&self, job_id: u64, reason: &str)
pub fn mark_failed(&self, job_id: u64, reason: &str)
Mark a job as failed with a descriptive reason.
Sourcepub fn pending_jobs(&self) -> Vec<WarmingJob>
pub fn pending_jobs(&self) -> Vec<WarmingJob>
Return clones of all pending jobs.
Sourcepub fn stats(&self) -> WarmingStats
pub fn stats(&self) -> WarmingStats
Snapshot current statistics.
Sourcepub fn record_time_ms(&self, ms: u64)
pub fn record_time_ms(&self, ms: u64)
Record additional wall-clock time spent on warming (in milliseconds).
Sourcepub fn default_patterns() -> Vec<WarmingPattern>
pub fn default_patterns() -> Vec<WarmingPattern>
Return a default set of common warming patterns.
Covers the four most common LLM task families: explain, summarise, translate, and code generation.
Auto Trait Implementations§
impl !Freeze for CacheWarmer
impl RefUnwindSafe for CacheWarmer
impl Send for CacheWarmer
impl Sync for CacheWarmer
impl Unpin for CacheWarmer
impl UnsafeUnpin for CacheWarmer
impl UnwindSafe for CacheWarmer
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
§impl<T> FutureExt for T
impl<T> FutureExt for T
§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
Source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
Wrap the input message
T in a tonic::Request