Struct ResponseLengthCapPlugin 

pub struct ResponseLengthCapPlugin {
    pub max_tokens: usize,
}

A plugin that truncates response token lists to at most max_tokens items.

Fields

max_tokens: usize

Maximum number of token strings to keep in the response.

Implementations

impl ResponseLengthCapPlugin

pub fn new(max_tokens: usize) -> Self

Create a new cap plugin.

Trait Implementations

impl Plugin for ResponseLengthCapPlugin

fn name(&self) -> &str

Unique human-readable name, e.g. "profanity-filter".

fn version(&self) -> &str

SemVer string, e.g. "1.0.0".

fn on_request(&self, _req: &mut PromptRequest) -> Result<(), PluginError>

Inspect and/or mutate the request before it reaches inference.

fn on_response(&self, resp: &mut Vec<String>) -> Result<(), PluginError>

Inspect and/or mutate the response tokens after inference.