Struct CascadeEngine 

pub struct CascadeEngine { /* private fields */ }

Multi-turn cascading inference engine.

Drives a model through a tool-call loop until a termination condition is met.

Example

use std::sync::Arc;
use tokio_prompt_orchestrator::cascade::{CascadeEngine, CascadeConfig, NoopToolExecutor};

let engine = CascadeEngine::new(
    Arc::new(|prompt: String| Box::pin(async move {
        Ok(format!("Echo: {prompt}"))
    })),
    Arc::new(NoopToolExecutor),
);

let result = engine
    .run("Summarise the Rust docs", &Default::default())
    .await
    .unwrap();

println!("Final answer: {}", result.final_answer);
println!("Turns taken: {}", result.turns.len());

Implementations

impl CascadeEngine

pub fn new(infer_fn: InferFn, executor: Arc<dyn ToolExecutor>) -> Self

Create a new engine with the given infer function and tool executor.

pub fn with_tool_parser(self, parser: Arc<dyn ToolCallParser>) -> Self

Override the default <tool_call> XML parser with a custom one.

pub async fn run( &self, prompt: &str, config: &CascadeConfig, ) -> Result<CascadeResult, OrchestratorError>

Run the cascade for the given prompt, returning all turns and the final answer.

Errors

Returns an error only if the first inference call fails. Subsequent failures are recorded in CascadeResult::termination_reason.