Release Inference Autopilot v0.1.25 — Adaptive Search and Confirmation Scheduling · rednote-machine-learning/Inference-autopilot

Technical highlights

Extract static and dynamically generated, model-specific recipes from the current SGLang Cookbook, including JSX-based configuration matrices.
Validate every documented flag and value against the installed SGLang ServerArgs contract before execution.
Translate deprecated Cookbook flags only when the live CLI explicitly names their replacement.
Adapt documented TP/PP/DP/EP topology to the selected GPU pool while preserving independent model and workload controls and removing only incompatible distributed companions.
Treat model-specific recipe handlers as compatibility adapters; candidate selection remains driven by the live model, hardware, workload, SLO, and measured evidence.

Reduce Nsys, runtime-log, model, workload, and memory evidence into canonical bottleneck classes, including co-primary classes when scores are effectively tied.
Use a unified Candidate Registry to track applicability, dependencies, conflicts, risk, scheduling, measurements, rejection reasons, and parent/child relationships.
Match parameters to the full (hardware, model, workload, SLO, bottleneck) context instead of relying on one global ranking.
Cover distinct semantic controls within each mechanism before spending additional budget on nearby values of the same parameter.
Derive continuous candidate values from effective defaults, VRAM headroom, workload length, topology, and allocator reserves.

Pack independent candidates onto disjoint GPUs according to each configuration's TP/PP/DP requirement.
Backfill otherwise idle GPU slots with valuable deferred refinements only when doing so does not add an estimated execution wave or consume confirmation reserves.
Compare topology candidates with resource-normalized objectives such as throughput per GPU instead of allowing wider configurations to win on raw service throughput alone.
Keep plan-time and runtime Nsys-overlap qualification consistent and report why overlap was disabled.

Use champion augmentation to test whether an individually positive parameter still adds value when composed with the current best configuration.
Require generated combinations to beat their strongest measured direct parent before they can enter confirmation.
Confirm the first nominee against the original baseline, then compare competitive challengers directly with the confirmed champion.
Reserve minimum evidence for later challengers so one ambiguous candidate cannot consume the complete confirmation budget.
Stop early with Bayesian sequential evidence when a gain or loss is already clear.
Run same-configuration confirmation replicas on disjoint GPUs and reuse resident services across repeated windows when a configuration occupies the full GPU pool.

Keep fresh and reused Nsys capacity evidence consistent by recording the actual pressure ceiling used by profiling.
Preserve the original safe baseline even after champion/challenger comparisons.
Report GPU placement, effective runtime settings, Cookbook adaptation, bottleneck evidence, idle-slot backfill, composition edges, statistical anchors, and replica/resident execution.

235 tests pass on the published repository snapshot.
Real GPU validation covered Cookbook discovery, resource-normalized topology comparison, idle-GPU refinement backfill, champion augmentation, Bayesian confirmation, and replica-parallel resident confirmation.

python3 -m pip install \
  --no-deps \
  --force-reinstall \
  "git+https://github.com/Jacki1223/inference-autopilot.git@v0.1.25"

See the README for the end-to-end inferopt init -> doctor -> plan -> run -> report workflow.