Expand description
§Stage: Declarative Pipeline Configuration
§Responsibility
Parse, validate, and hot-reload TOML pipeline configuration files. Users define an entire pipeline topology declaratively and run it with:
cargo run -- --config pipeline.toml§Guarantees
- Deterministic: same TOML input always produces the same
PipelineConfig - Validated: all semantic constraints are checked before a config is accepted
- Type-safe: invalid field combinations are caught at parse time via serde
- Hot-reloadable: file changes are detected and validated before applying
- Schema-exportable: JSON Schema output enables IDE autocomplete
§NOT Responsible For
- Building the runtime pipeline from config (that belongs to
stages) - Managing worker connections (that belongs to
worker) - Metrics collection (that belongs to
metrics)
Modules§
- loader
- Configuration file loading.
- validation
- Configuration validation engine.
- watcher
- Configuration hot-reload watcher.
Structs§
- Assemble
Stage Config - Assembly stage configuration.
- Channel
Sizes - Per-channel capacity overrides for the five inter-stage pipeline channels.
- Deduplication
Config - Deduplication configuration for request coalescing.
- Inference
Stage Config - Inference stage configuration.
- Observability
Config - Observability configuration: logging, metrics endpoint, and tracing.
- Pipeline
Config - Root configuration for a pipeline instance.
- Pipeline
Section - Pipeline identity and version metadata.
- Post
Process Stage Config - Post-processing stage configuration.
- RagStage
Config - RAG stage configuration.
- Rate
Limit Config - Rate limiting configuration.
- Resilience
Config - Resilience configuration for retries and circuit breaking.
- Stages
Config - Configuration for all five pipeline stages.
- Stream
Stage Config - Output streaming stage configuration.
- Worker
Config - Per-worker connection and model configuration.
- Workers
Config - Per-worker configuration map, keyed by provider name.
Enums§
- LogFormat
- Log output format.
- Provider
- Provider identifier enum used in per-worker configuration.
- Worker
Kind - Supported model worker backends.