Expand description
§A/B Testing Framework
Statistically rigorous A/B testing for LLM model and prompt experiments. Supports weighted traffic splits, deterministic user assignment via FNV hash, two-proportion z-test significance testing, and automatic experiment completion.
Structs§
- AbTest
Config - Configuration for a single legacy two-variant A/B experiment.
- AbTest
Manager - Manages multi-variant A/B experiments.
- AbTest
Result - Statistical result of a legacy two-variant A/B experiment.
- AbTest
Runner - Thread-safe legacy two-variant A/B test runner.
- Assignment
- Records that a user was assigned to a specific variant.
- Experiment
- A multi-variant experiment definition.
- Experiment
Result - Aggregated results for one variant.
- Experiment
Variant Spec - A single variant in an experiment.
Enums§
- EndCondition
- Condition that triggers automatic experiment completion.
- Experiment
Status - Status of an experiment lifecycle.
- Success
Metric - The metric used to evaluate variant quality (legacy API).
- Variant
- Which A/B variant a request is assigned to (legacy two-variant API).