Module cascade 

Cascading multi-turn inference engine.

Enables multi-turn reasoning loops where a model’s output can trigger additional pipeline passes — tool calls, function execution, chain-of-thought continuation — until a termination condition is met.

Architecture

PromptRequest
      │
      ▼
┌─────────────────────────────────────────┐
│          CascadeEngine                  │
│                                         │
│  Turn 1: infer → parse tools            │
│  Turn 2: execute tools → infer again    │
│  Turn N: model returns DONE / no tools  │
└─────────────────────────────────────────┘
      │
      ▼
CascadeResult (all turns + final answer)

Termination Conditions

The loop exits when ANY of the following is true:

• The model output contains no tool calls (ToolCallParser finds nothing)
• The output text contains the DONE sentinel string
• max_turns is reached (default: 10)
• A pipeline error occurs (propagated to caller)

Tool Call Format

The engine recognises tool calls in this JSON block format:

<tool_call>
{"name": "search", "arguments": {"query": "rust tokio"}}
</tool_call>

Custom parsers can be registered via CascadeEngine::with_tool_parser.

Structs

CascadeConfig

Configuration for a cascade run.

CascadeEngine

Multi-turn cascading inference engine.

CascadeMonitor

A thread-safe store of active cascade sessions for introspection.

CascadeResult

Final output of a complete cascade run.

CascadeTurn

The result of a single cascade turn.

NoopToolExecutor

Stub executor that returns a placeholder result without doing real work.

ToolCall

A single parsed tool call extracted from a model response.

ToolResult

The result of a tool execution.

XmlStyleToolParser

Default parser: finds <tool_call>…</tool_call> JSON blocks.

Enums

TerminationReason

Why a cascade loop exited.

Constants

DEFAULT_MAX_TURNS

Default maximum number of cascade turns before halting.

DONE_SENTINEL

Sentinel string that signals the model is done cascading.

Traits

ToolCallParser

Trait for parsing tool calls out of a raw model response.

ToolExecutor

Trait for executing tool calls and returning results.

Type Aliases

InferFn

Infer function type: takes a prompt string, returns the model response.