Module cost_optimizer 

Module: Cost Optimizer

Responsibility

Analyses historical request cost patterns and produces actionable optimisation suggestions. Can optionally apply suggestions automatically when auto_optimize = true.

Detections

1. Cache candidates — prompts whose responses are highly similar across invocations (response fingerprint collision rate above threshold).
2. Model overuse — expensive models being used for structurally simple tasks (short prompt + short response).

Guarantees

• Thread-safe (Arc<CostOptimizer> shareable across tasks)
• Bounded memory: observation windows capped at window_size
• Non-blocking: all methods are O(window_size), no I/O
• Advisory only unless auto_optimize = true

Structs

AutoApplyAction

An automatically applied optimisation action.

CostObservation

A single cost observation recorded for one inference call.

CostOptimizer

Analyses cost patterns and generates (or applies) optimisation suggestions.

CostOptimizerConfig

Cost optimizer configuration.

ModelTierEntry

An entry in the model tier table.

OptimizationSuggestion

A single optimisation suggestion with computed ROI.

Enums

CostOptimizerError

Errors produced by the cost optimizer.

ModelTier

Tier classification for a model.

SuggestionKind

The kind of cost optimisation suggested.