Module cascade 

Multi-provider cascade fallback.

A ProviderCascade chains an ordered list of LLM providers and tries them in sequence — primary → secondary → tertiary — skipping any whose circuit breaker is currently open. This extends the binary LocalWithFallback routing decision to an arbitrarily long chain.

Guarantees

• Fully async — all provider calls are non-blocking.
• Circuit-breaker aware — open breakers are skipped without a call attempt.
• Per-provider telemetry — latency and success counters tracked with Arc<AtomicU64> for lock-free hot-path reads.
• Prometheus metrics — cascade_attempts_total{provider} and cascade_failures_total{provider} counters using lazy-static registration.

Example

use std::time::Duration;
use tokio_prompt_orchestrator::config::WorkerKind;
use tokio_prompt_orchestrator::routing::cascade::{
    CascadeEntry, CircuitBreakerConfig, ProviderCascade,
};

let cascade = ProviderCascade::new(vec![
    CascadeEntry::new(WorkerKind::LlamaCpp, CircuitBreakerConfig::default()),
    CascadeEntry::new(WorkerKind::OpenAi,   CircuitBreakerConfig::default()),
    CascadeEntry::new(WorkerKind::Anthropic, CircuitBreakerConfig::default()),
]);

Structs

CascadeEntry

A single slot in the provider cascade: one WorkerKind plus its own CircuitBreaker and per-call telemetry counters.

CascadeFailover

Priority-ordered cascade failover across named worker tiers.

CascadeResult

The outcome of a successful ProviderCascade::call invocation.

CircuitBreakerConfig

Configuration for a single circuit breaker inside a cascade entry.

FailoverExhausted

Error returned by CascadeFailover::call when all tiers are exhausted.

FailoverResult

Result produced by a successful CascadeFailover::call.

FailoverTier

A named tier in a CascadeFailover configuration.

ProviderCascade

A configurable chain of LLM providers that tries primary → secondary → tertiary on failure.

Enums

CascadeError

Errors returned by ProviderCascade::call.

Functions

cascade_registry

Return a reference to the cascade-specific Prometheus registry.