Struct ArbitrageEngine 

pub struct ArbitrageEngine { /* private fields */ }

The provider arbitrage engine.

Thread-safe; wrap in Arc to share across pipeline stages.

Panics

No method panics.

Implementations

impl ArbitrageEngine

pub fn new() -> Self

Create a new, empty arbitrage engine.

pub fn register(&self, profile: ProviderProfile)

Register a provider. If a provider with the same name already exists, its profile is updated (existing runtime state is preserved).

Panics

Does not panic.

pub fn record_latency(&self, provider: &str, latency: Duration)

Record an observed end-to-end latency for a provider.

Call this after every inference call completes (success or failure).

Panics

Does not panic.

pub fn record_success( &self, provider: &str, input_tokens: u64, output_tokens: u64, latency: Duration, )

Record a successful inference call for accounting purposes.

Panics

Does not panic.

pub fn record_error(&self, provider: &str, latency: Duration)

Record a failed inference call for a provider.

Panics

Does not panic.

pub fn select_provider( &self, sla_budget: Option<Duration>, ) -> Option<ProviderProfile>

Select the cheapest provider whose P95 latency is at or below sla_budget.

Selection algorithm

1. Filter providers whose P95 latency ≤ sla_budget.
2. Among those, pick the one with the lowest cost (input + output rate). Ties are broken by priority (lower = preferred), then name (alphabetical, for determinism).
3. If no provider meets the SLA (or sla_budget is None), fall back to the provider with the lowest P95 latency.
4. If no providers are registered, returns None.

Returns

A cloned ProviderProfile for the selected provider, or None if no providers are registered.

Panics

Does not panic.

pub fn snapshot(&self) -> Vec<ProviderSnapshot>

Return a point-in-time snapshot of all registered providers.

Panics

Does not panic.

pub fn total_selections(&self) -> u64

Total select_provider calls since engine creation.

pub fn total_sla_misses(&self) -> u64

Total selections that fell back to fastest (SLA could not be met).

Trait Implementations

impl Debug for ArbitrageEngine

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ArbitrageEngine

fn default() -> Self

Returns the “default value” for a type.