Skip to main content

Plugin

tokio_prompt_orchestrator::plugin

Trait Plugin 

Source 
pub trait Plugin: Send + Sync {
    // Required methods
    fn name(&self) -> &str;
    fn version(&self) -> &str;
    fn on_request(&self, req: &mut PromptRequest) -> Result<(), PluginError>;
    fn on_response(&self, resp: &mut Vec<String>) -> Result<(), PluginError>;
}
Expand description

Core trait for request/response plugins.

Implement this trait and register it with a PluginV2Registry to intercept requests before inference and responses after inference.

Required Methods§

Source

fn name(&self) -> &str

Unique human-readable name, e.g. "profanity-filter".

Source

fn version(&self) -> &str

SemVer string, e.g. "1.0.0".

Source

fn on_request(&self, req: &mut PromptRequest) -> Result<(), PluginError>

Inspect and/or mutate the request before it reaches inference.

Return Err(PluginError::RequestRejected { .. }) to abort the request.

Source

fn on_response(&self, resp: &mut Vec<String>) -> Result<(), PluginError>

Inspect and/or mutate the response tokens after inference.

Return Err(PluginError::ResponseModified) or Err(PluginError::Fatal) to signal post-processing issues (pipeline may log and continue).

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§