pub struct FallbackManager { /* private fields */ }Expand description
Thread-safe wrapper around FallbackChain that executes closures against
the best available model, retrying down the chain on failure.
Implementations§
Source§impl FallbackManager
impl FallbackManager
Sourcepub fn new(chain: FallbackChain) -> Self
pub fn new(chain: FallbackChain) -> Self
Create a new manager wrapping the given chain.
Sourcepub fn execute<F, T>(
&self,
f: F,
require_tools: bool,
context_size: usize,
) -> Result<T, String>
pub fn execute<F, T>( &self, f: F, require_tools: bool, context_size: usize, ) -> Result<T, String>
Execute f against the best available model.
Walks the chain until a model succeeds or all models are exhausted.
require_tools: skip models that do not support tool/function calling.context_size: skip models whose context window is too small.
Auto Trait Implementations§
impl Freeze for FallbackManager
impl RefUnwindSafe for FallbackManager
impl Send for FallbackManager
impl Sync for FallbackManager
impl Unpin for FallbackManager
impl UnsafeUnpin for FallbackManager
impl UnwindSafe for FallbackManager
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
§impl<T> FutureExt for T
impl<T> FutureExt for T
§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
Source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
Wrap the input message
T in a tonic::Request