Struct CascadeFailover 

pub struct CascadeFailover { /* private fields */ }

Priority-ordered cascade failover across named worker tiers.

Tries tiers from highest priority (index 0) to lowest. Within each tier the configured number of retries is attempted before advancing to the next. Each attempt is individually time-bounded by the tier’s timeout.

Typical tier ordering

1. Premium (e.g. Anthropic)  — lowest latency, highest cost, short timeout
2. Standard (e.g. OpenAI)    — medium cost, medium timeout
3. Local fallback (LlamaCpp) — zero cost, longer timeout

Example

use std::time::Duration;
use tokio_prompt_orchestrator::config::WorkerKind;
use tokio_prompt_orchestrator::routing::cascade::{CascadeFailover, FailoverTier};

let failover = CascadeFailover::new(vec![
    FailoverTier::new("premium",  WorkerKind::Anthropic, Duration::from_secs(10), 0),
    FailoverTier::new("standard", WorkerKind::OpenAi,    Duration::from_secs(20), 1),
    FailoverTier::new("local",    WorkerKind::LlamaCpp,  Duration::from_secs(60), 2),
]);

let result = failover.call(|kind| async move {
    // Your inference call here
    Ok::<String, String>(format!("response from {:?}", kind))
}).await;

match result {
    Ok(r) => println!("Succeeded on tier '{}': {}", r.tier_name, r.value),
    Err(e) => eprintln!("All tiers failed: {e}"),
}

Implementations

impl CascadeFailover

pub fn new(tiers: Vec<FailoverTier>) -> Self

Create a new failover chain. Tiers are tried in slice order.

Panics

This function does not panic.

pub fn with_tier(self, tier: FailoverTier) -> Self

Builder pattern: append a tier.

Panics

This function does not panic.

pub fn tier_count(&self) -> usize

Return the number of configured tiers.

Panics

This function does not panic.

pub fn is_empty(&self) -> bool

Return true if no tiers are configured.

Panics

This function does not panic.

pub async fn call<F, Fut, T>( &self, f: F, ) -> Result<FailoverResult<T>, FailoverExhausted>

where F: Fn(WorkerKind) -> Fut, Fut: Future<Output = Result<T, String>>,

Execute f against tiers in priority order, respecting per-tier timeouts and retries.

f receives the WorkerKind of the current tier and must return a future that resolves to Ok(T) on success or Err(String) on failure.

Errors

Returns FailoverExhausted if every attempt on every tier fails.

Panics

This function does not panic.

Trait Implementations

impl Debug for CascadeFailover

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.