Struct ModelEndpoint 

pub struct ModelEndpoint {
    pub id: String,
    pub url: String,
    pub weight: u32,
    pub max_rps: f64,
    pub healthy: bool,
    pub latency_p99_ms: f64,
}

A single model inference endpoint.

Fields

id: String

Unique identifier for this endpoint (e.g. "gpt-4o-primary").

url: String

Base URL of the endpoint (e.g. "https://api.openai.com/v1").

weight: u32

Relative weight used for weighted round-robin selection. Higher weight = more traffic. Must be > 0.

max_rps: f64

Maximum requests per second this endpoint can sustain.

healthy: bool

Whether this endpoint is currently considered healthy.

latency_p99_ms: f64

99th-percentile observed latency in milliseconds.

Trait Implementations

impl Clone for ModelEndpoint

fn clone(&self) -> ModelEndpoint

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ModelEndpoint

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.