Skip to content

Inference Autopilot v0.1.24 — Initial Release

Choose a tag to compare

View all tags
@Jacki1223 Jacki1223 released this 26 Aug 06:16
· 34 commits to main since this release
v0.1.24
d5354af
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Initial release

Inference Autopilot is an evidence-driven CLI for hardware- and workload-aware SGLang deployment optimization. It discovers the installed SGLang parameter surface, validates feasibility, benchmarks applicable configurations, profiles the serving path, confirms gains under SLO and stability gates, and emits reproducible deployment commands.

Highlights

  • Canonical Candidate Registry with merged provenance, lifecycle state, risk, dependencies, measurements, and parent/child relationships.
  • Mechanism-level adaptive search that covers applicable mechanisms before deep refinement, then spends follow-up budget only on positive or uncertain evidence.
  • Workload- and hardware-derived parameter values for prefill chunking, KV capacity, scheduling, CUDA Graph, model-native mechanisms, and topology.
  • Nsight Systems and SGLang log analysis with a canonical bottleneck classifier and intervention-informed posterior diagnosis.
  • Parent-relative composition gates so weak or conflicting parameter combinations do not pollute the final command.
  • Bayesian repeated-window confirmation, SLO gating, confidence intervals, and explicit baseline retention.
  • Safe resume semantics: raw profiles may be reused, while search-policy changes invalidate screening and downstream decisions.
  • Minimal and reproducible SGLang launch commands, structured JSON artifacts, and a Markdown report.

Validation

  • 159 automated tests pass.
  • The wheel was built, installed in isolation, and imported successfully.
  • End-to-end GPU experiments verified screening, refinement, composition, confirmation, reporting, cleanup, and resume behavior.

Install

python3 -m pip install \
  --no-deps \
  --no-build-isolation \
  --force-reinstall \
  "git+https://github.com/Jacki1223/inference-autopilot.git@v0.1.24"

This is the initial public release. Results are intentionally bounded to the recorded model, SGLang version, hardware, workload, experiment budget, and acceptance gates; they are not claims of a universal optimum.

Assets 3
Loading