Struct FailoverChain 

pub struct FailoverChain { /* private fields */ }

A priority-ordered chain of model workers with automatic failover.

Workers are tried in insertion order. Unhealthy workers (as determined by the ProviderHealthMonitor) are skipped without consuming an attempt slot. The max_attempts limit counts only actual inference attempts (i.e. workers that were healthy and returned a result — success or failure).

Implementations

impl FailoverChain

pub fn new(health: ProviderHealthMonitor) -> Self

Create a new, empty failover chain.

Panics

Never panics.

pub fn add_worker( self, provider_id: impl Into<String>, worker: Arc<dyn ModelWorker>, ) -> Self

Append a worker to the end of the chain.

Workers are tried in the order they are added.

Panics

Never panics.

pub fn with_max_attempts(self, n: usize) -> Self

Override the maximum number of attempted inferences before the chain gives up. Defaults to 3. A value of 0 is silently raised to 1.

Panics

Never panics.

pub async fn infer( &self, prompt: &str, ) -> Result<(String, String), OrchestratorError>

Run inference through the chain.

Workers are tried in priority order. For each candidate:

1. The health monitor is consulted; unhealthy providers are skipped (they do not count against max_attempts).
2. The worker’s ModelWorker::infer method is called.
3. On success the outcome is recorded with the health monitor and the (provider_id, response_text) tuple is returned.
4. On failure the outcome is recorded and the next candidate is tried (unless max_attempts has been exhausted).

Returns Err(OrchestratorError::Inference) if all candidates fail or the chain is empty.

Panics

Never panics.

pub fn len(&self) -> usize

Return the number of workers currently in the chain.

Panics

Never panics.

pub fn is_empty(&self) -> bool

Return true if the chain contains no workers.

Panics

Never panics.