Module adaptive_pool 

Adaptive worker pool with Kalman-filter latency prediction.

Monitors pipeline queue depths and inferred latency to decide when to spawn additional worker tasks or drain excess ones. Uses a one-dimensional Kalman filter to smooth noisy queue-depth observations and predict the short-term trend, avoiding oscillation from reactive on/off control.

Design

┌──────────────────────────────────────────────────────────┐
│  AdaptivePool                                             │
│                                                           │
│  Queue depth ──► KalmanFilter ──► predicted depth        │
│                                        │                 │
│  Latency EMA ──────────────────────► ScaleDecision       │
│                                        │                 │
│                            spawn task / mark idle        │
└──────────────────────────────────────────────────────────┘

Kalman Filter

A scalar 1-D Kalman filter tracks the queue depth signal:

• State x: estimated true queue depth
• Process noise Q: models how fast the queue can change (default 1.0)
• Measurement noise R: models observation noise (default 5.0)

The filter converges to the true depth in ~5–10 observations and provides a smooth signal that drives scaling without reacting to single-sample spikes.

Scaling Policy

Condition	Action
predicted_depth > scale_up_threshold AND latency > latency_threshold_ms	Recommend scale-up
predicted_depth < scale_down_threshold AND pool_size > min_workers	Recommend scale-down
otherwise	Stable

Structs

AdaptivePool

Adaptive pool controller.

AdaptivePoolConfig

Configuration for the adaptive pool controller.

KalmanFilter

Scalar 1-D Kalman filter.

LatencyEma

EMA tracker for latency observations.

PoolStats

Pool statistics snapshot.

Enums

ScaleDecision

Scaling recommendation produced by AdaptivePool::evaluate.

Functions

run_pool_controller

Runs the adaptive pool evaluation loop as a Tokio background task.