Module context_mgr 

Context Manager

Multi-turn LLM conversation context manager with token budget enforcement and configurable truncation strategies.

Overview

ConversationContext maintains an ordered deque of Message values and automatically enforces a token budget when new messages are added. Three truncation strategies are supported: dropping the oldest messages, keeping only the first N turns plus recent history, and replacing oldest messages with a summary placeholder.

Example

use tokio_prompt_orchestrator::context_mgr::{
    ConversationContext, ContextConfig, Role, TruncationStrategy,
};

let config = ContextConfig {
    max_tokens: 1000,
    system_prompt: Some("You are a helpful assistant.".into()),
    reserve_for_response: 100,
    truncation_strategy: TruncationStrategy::DropOldest,
};
let mut ctx = ConversationContext::new(config);
ctx.add_message(Role::User, "Hello!");
ctx.add_message(Role::Assistant, "Hi there!");
assert!(!ctx.messages_for_api().is_empty());

Structs

ContextConfig

Configuration for a ConversationContext.

ContextSummary

A lightweight summary of the current context state.

ConversationContext

Multi-turn conversation context with automatic token budget enforcement.

Message

A single message in a conversation.

TokenCounter

Simple word-count token estimator.

Enums

Role

The role of a message participant in a conversation.

TruncationStrategy

How to reduce the context when the token budget is exceeded.