Skip to main content

Module health

Module health 

Source
Expand description

Provider health dashboard snapshot.

ProviderHealthSnapshot aggregates per-provider health metrics into a single value that can be serialised to JSON for a /health/providers REST endpoint or logged on demand.

§What It Aggregates

For each registered provider the snapshot captures:

FieldSource
last_success_atUnix timestamp (seconds) of the most recent success
last_failure_atUnix timestamp (seconds) of the most recent failure
consecutive_failuresFailures since the last success
success_rate_1hFraction of calls that succeeded in the last hour
p50_latency_msMedian latency over all recorded calls
p95_latency_ms95th-percentile latency over all recorded calls

§Usage

use tokio_prompt_orchestrator::routing::health::{ProviderHealthBuilder, ProviderHealthSnapshot};
use tokio_prompt_orchestrator::config::WorkerKind;

let mut builder = ProviderHealthBuilder::new();
builder.record_success(WorkerKind::Anthropic, 120);
builder.record_failure(WorkerKind::OpenAi);
let snapshot = builder.snapshot();
let json = snapshot.to_json();
assert!(json.is_object());

Structs§

ProviderHealthBuilder
Mutable builder that accumulates per-provider health observations and can produce a ProviderHealthSnapshot on demand.
ProviderHealthSnapshot
An immutable snapshot of health metrics for all tracked providers.
ProviderMetrics
Health metrics for a single provider at a point in time.