Struct LatencyLoggerPlugin 

pub struct LatencyLoggerPlugin {
    pub log: Mutex<Vec<u64>>,
}

A plugin that records per-request start/end timestamps (milliseconds since Unix epoch) into a shared latency log.

Call LatencyLoggerPlugin::record_start before inference and LatencyLoggerPlugin::record_end after inference to append the elapsed duration to the log.

Fields

log: Mutex<Vec<u64>>

Accumulated latency samples in milliseconds.

Implementations

impl LatencyLoggerPlugin

pub fn new() -> Self

Create a new logger with an empty latency log.

pub fn record_latency(&self, latency_ms: u64)

Record a completed request’s latency in milliseconds.

pub fn samples(&self) -> Vec<u64>

Return a copy of all recorded latencies.

Trait Implementations

impl Default for LatencyLoggerPlugin

fn default() -> Self

Returns the “default value” for a type.

impl Plugin for LatencyLoggerPlugin

fn name(&self) -> &str

Unique human-readable name, e.g. "profanity-filter".

fn version(&self) -> &str

SemVer string, e.g. "1.0.0".

fn on_request(&self, _req: &mut PromptRequest) -> Result<(), PluginError>

Inspect and/or mutate the request before it reaches inference.

fn on_response(&self, _resp: &mut Vec<String>) -> Result<(), PluginError>

Inspect and/or mutate the response tokens after inference.