pub struct FallbackChain { /* private fields */ }Expand description
Ordered list of models with failure tracking and cooldown management.
Implementations§
Source§impl FallbackChain
impl FallbackChain
Sourcepub fn new(models: Vec<FallbackModel>) -> Self
pub fn new(models: Vec<FallbackModel>) -> Self
Build a new chain from the supplied models, sorted by ascending priority.
Sourcepub fn next_available(
&self,
require_tools: bool,
min_context: usize,
) -> Option<&FallbackModel>
pub fn next_available( &self, require_tools: bool, min_context: usize, ) -> Option<&FallbackModel>
Find the next available model that meets the given constraints.
Skips models that:
- Are currently within their cooldown window
- Have accumulated more than 3 consecutive failures
- Do not support tools when
require_toolsistrue - Have a context window smaller than
min_context
Sourcepub fn record_failure(&mut self, model_id: &str, error: &ModelError)
pub fn record_failure(&mut self, model_id: &str, error: &ModelError)
Record a failure for model_id.
Applies a 60-second cooldown for ModelError::RateLimit errors.
Sourcepub fn record_success(&mut self, model_id: &str)
pub fn record_success(&mut self, model_id: &str)
Record a successful call for model_id, resetting its failure counter.
Sourcepub fn reset_cooldown(&mut self, model_id: &str)
pub fn reset_cooldown(&mut self, model_id: &str)
Manually clear the cooldown for model_id.
Sourcepub fn chain_report(&self) -> Vec<FallbackReport>
pub fn chain_report(&self) -> Vec<FallbackReport>
Generate a status report for every model in the chain.
Auto Trait Implementations§
impl Freeze for FallbackChain
impl RefUnwindSafe for FallbackChain
impl Send for FallbackChain
impl Sync for FallbackChain
impl Unpin for FallbackChain
impl UnsafeUnpin for FallbackChain
impl UnwindSafe for FallbackChain
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
§impl<T> FutureExt for T
impl<T> FutureExt for T
§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
Source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
Wrap the input message
T in a tonic::Request