Skip to content

Releases

AISimulate Release v0.13.0

Choose a tag to compare

@saturley-hall saturley-hall released this 21 Oct 23:51
289f8fb

AISimulate - Release 0.13.0

Summary

AISimulate 0.13.0 pairs with Dynamo 1.6.0. This release completes the move from AIConfigurator to AISimulate and adds forward-pass model (FPM) self-service onboarding for vLLM, public accuracy reporting, experimental AgentX trace replay, KV cache offload simulation, and broader model and serving-mode coverage.

pip install aisimulate==0.13.0
  • Python 3.11 to 3.13. Python 3.14 is not yet supported; pip reports a resolution failure without a clear hint.
  • Wheels: Linux x86-64, Linux aarch64, macOS arm64.
  • Validated pairing: AISimulate 0.13.0 with Dynamo 1.6.0.

Key Highlights

FPM self-service onboarding (vLLM)

Onboard a new model on your own GPUs by measuring its forward passes. aisimulate onboard init declares the model, target hardware and worker topology. onboard plan writes the collection plan plus predict and recommend configs. onboard collect-fpm previews the collection, then runs Dynamo's vLLM self-benchmark with --execute --smoke and --execute. onboard validate-collection checks the measured data, and onboard finalize produces the simulation inputs. Further subcommands cover runtime probing, importing saved observations, serving validation and resumable checkpoints (#196, #248). Guide: FPM self-service.

  • Runtime probing ships audited instrumentation for vLLM 0.27.0 only; Dynamo 1.6.0 runs vLLM 0.31.0.
  • Collecting dense models needs Dynamo 1.6.0 GA (it includes ai-dynamo/dynamo#16804), and very large checkpoints may not complete (see Known issues).
  • SGLang and TRT-LLM self-service are on the roadmap.

Public accuracy reporting

Forward-pass and end-to-end accuracy dashboards are published at https://ai-dynamo.org/aisimulate/fpm-accuracy/ and https://ai-dynamo.org/aisimulate/e2e-accuracy/ (#253, #368, #375).

AgentX trace replay (experimental)

Replay agentic coding traces offline for vLLM and SGLang: direct Weka AgentX ingestion, deterministic play outcomes, seeded snapshots and continuous duration profiles (#115, #121, #122, #190, #207, #307). In release testing, a finite AgentX trace completed all 722 requests with 275 MB peak memory.

KV cache offload simulation

AISimulate models three KV cache tiers: G1 (GPU memory), G2 (host memory) and G3 (storage).

  • vLLM G2 offload, local and cluster-shared, including attention DP and prefill/decode disaggregation (#358, #381).
  • vLLM G3 write-through offload for predict (#203). G3 restores through G2 and requires aggregated vLLM replay with attention DP 1; it is not available with prefill/decode disaggregation, AgentX replay or recommend.
  • Hybrid models: Kimi K3 cache geometry is inferred from the model configuration (#293), and the G1 state cache for Mamba/KDA layers can be sized manually (#243).
  • Partial prefix hits via vLLM prefix_match_unit (#314). For Kimi K3, see Known issues.

Better recommendations

  • recommend can size for the fewest GPUs: optimization.target: min_gpus selects the smallest configuration found that meets the SLA at a fixed request rate or concurrency. Request-rate traffic also needs optimization.constraints.min_goodput_rps (#270).
  • Bayesian search no longer shows the memory growth seen in 0.12: a bounded release test (64 trials at parallelism 2) peaked at 0.87 GB.
  • Heterogeneous prefill and decode hardware: set engine.workers.prefill.hardware or engine.workers.decode.hardware in disaggregated predict and recommend (#186, #204).

Clearer output and errors

  • Predictions print their estimator, timing source, data version and KV precision.
  • Clear errors for unsupported models, missing data versions, zero-completion runs and unsupported FP8 compute (for example FP8 on A100). Parallel shapes that do not split the model's attention heads are skipped instead of failing.
  • --overwrite now validates the configuration (YAML, unknown sections and fields, value types, missing stacks or output adapters) before clearing previous outputs. A misspelled model or hardware name still clears them; see Known issues.

Serving modes and models

  • Serving modes, all in predict:
    • Attention-FFN disaggregation (AFD), modeled analytically and configured with engine.mode: afd and engine.afd; also in recommend (#107 to #111).
    • Analytical encode/prefill/decode disaggregation: an encoder pool (engine.workers.encoder) with fixed image inputs, in front of aggregated or disaggregated workers; also in recommend. Qwen3-VL models are supported in this release (#156).
    • Context parallelism: parallelism.prefill_context (SGLang models only) and parallelism.decode_context (vLLM and SGLang, model-dependent), predict only (#240).
    • TRT-LLM disaggregated serving (#77) and attention DP with different sizes per prefill and decode pool (#105).
    • N-gram prompt-lookup speculative decoding via engine.speculation, for vLLM with op-level timing; also in recommend (#227).
  • Models:
    • DeepSeek V4.1 Flash: FPM self-service plans it with the required eager execution, and measured SGLang FPM profiles (H200, B200 and GB200 at TP4; B300 at TP2 and TP4; GB300 at TP2) load through the Python SDK. predict has no op-level path for it yet (see Known issues).
    • Kimi K3 (moonshotai/Kimi-K3), and Qwen3.8-Max (Qwen/Qwen3.8-2.4T-A95B and -FP8) on vLLM and SGLang. Qwen3.8-Max needs backend_version: next; the default data version has no FP8 MoE measurements for it.
    • Vision encoder modeling for Qwen3.5, Kimi K2.5 and Gemma 4 through the image options of the bundled aiconfigurator compatibility command (#84, #86).
  • New measurements: opt-in FP8 FlashInfer TRT-LLM MoE data for Qwen3.8-Max on SGLang 0.5.17 (B200, GB200 and GB300; select with engine.moe_kernel_source: sglang_flashinfer_trtllm_moe), GB200 multi-node communication data, and a DeepEP low-latency decode communication model.

Dynamo 1.6.0 integration

Tested with Dynamo 1.6.0 release candidates: Planner estimator, profiler rapid mode, mocker with AISimulate timing, frontend AISimulate router, and the DynoSim tutorial.

Also in this release

  • Slurm deployment and benchmark bundles: --deployment-target slurm in the bundled aiconfigurator compatibility command (or the aisimulate.generator API) writes a single-node Slurm (Pyxis/Enroot) bundle with a deploy job and an AIPerf benchmark job for vLLM, SGLang and TRT-LLM, aggregated or disaggregated (#502, #503 and earlier). See deployment generation.
  • Output adapter interface: recommend --output NAME runs an installed output adapter that writes extra artifacts or observes candidates live (#334). No adapters ship with AISimulate; see the output adapter ABI.
  • Faster replay: scheduler queue indexes, cheaper SGLang admission, faster prompt hashing, reduced bookkeeping (#348, #356, #386, #397).
  • Agent skills for developing with AISimulate (#322).

Migration and Compatibility Notes

  • The aiconfigurator and aiconfigurator_core Python import namespaces are removed. Import from aisimulate and aisimulate_core instead (for example aiconfigurator_core.sdk becomes aisimulate_core.sdk). The aiconfigurator command remains for compatibility workflows. The Rust crate is renamed from aiconfigurator-core to aisimulate-core; Cargo users may alias it under the old dependency key. Estimator construction takes a single ForwardPassPerfModelConfig. Convert any pickles that encode the old module paths before upgrading. See the migration guide.
  • Host resource supervision. Every predict and recommend run now checks host CPU and memory budgets and runs inside a super...
Read more

AISimulate v0.12.1

Choose a tag to compare

@saturley-hall saturley-hall released this 01 Oct 16:14
468f05d

AISimulate - Release 0.12.1

Summary

AISimulate 0.12.1 is a patch release on top of 0.12.0. It carries a single functional change: a replay fix that keeps timing and capacity on the selected database version (#344, cherry-picked from #291), together with the crate and wheel version bump to 0.12.1 (#340).

There are no new features, model additions, collector upgrades, API changes or deprecations in this release. Anyone running replay against an explicitly selected perf-database version should take this patch, since timing and capacity results could previously be resolved off a different version than the one selected. Upgrading from 0.12.0 requires no configuration or CLI changes.


Bug Fixes

Replay

  • Keep timing and capacity on the selected database version (#344, cherry-pick of #291).

Build and Packaging

  • Bump release/0.12.1 crate and wheel versions to 0.12.1 (#340).

Full Changelog

Full Changelog: v0.12.0...v0.12.1

AISimulate Release v0.12.0

Choose a tag to compare

@saturley-hall saturley-hall released this 17 Sep 18:08
1f72853

AISimulate - Release 0.12.0

Summary

AISimulate 0.12.0 is the inaugural release of the AISimulate repository. It is the convergence point for two codebases: DynoSim and AIConfigurator. From this release forward, AISimulate can predict and recommend LLM inference configurations for TRT LLM, vLLM, and SGLang, and Dynamo. Furthermore, users can simulate both static traffic (ISL/OSL/concurrency) and dynamic traffic (for e.g. a mooncake trace). Because the two trees kept moving during the migration, parity is maintained by explicit bulk syncs — AIC parity through 095f58a (#31, AIC-1788), 23 further AIC PRs through ce2824e (#73), and the AIC PyO3 operation unification (#10) — with CI validation restored across the merged packages (#9).

The CLI. ASimulate now supports “predict” and “recommend” commands in on one interface (#29), while the existing aiconfigurator command identity is deliberately preserved so current users are not broken by the move (#26). Strict aggregate SLA filtering arrives (#12, AIC-1646) alongside documentation mapping AIC request latency onto the E2E SLA definition (#40).

Replay engine. Typed agentic graph execution lands (#24), joined by randomized synthetic request lengths (#5), routing regret diagnostics (#54), policy-neutral telemetry sampling (#60), native vLLM host offload simulation (#57), and an engine replay parity skill (#47) backed by vLLM prefix cache lifecycle coverage (#33).

A concentrated round of Rust core indexing work removes repeated hot-path scans: Grid hold lookups (#51), DSV4 top-k interpolation (#55), UtilGrid lookup indexing and caching (#58), FP8 static delta lookup (#65), and elimination of redundant DSV4 context grid rebuilds (#66).

On the Sweeper and generator side, candidates are lowered into deployment requests (#11) and canonical result semantics are defined (#15, AIC-1471). Configuration gains a server knob taxonomy with backend enums (#67), and a measured FPE support matrix is published (#41, AIC-1819). Because this is a new public repository, a substantial share of the release is governance, licensing and compliance groundwork: CODEOWNERS (#8), contribution and dependency policy (#28), DCO (#32), a security policy (#45), OSRB filing requirements (#92), third-party license notices (#148) and hardened SPDX checks (#150).

Note: this is the first tagged release of the AISimulate repository, so the changelog below covers the full history of the repo rather than a delta against a previous tag.


Key Highlights

Repository Migration and AIC Parity

  • AISimulate migrated out of Dynamo into its own package (#3, AIC-1703) and the full AIConfigurator tree migrated in (#2, AIC-1702).
  • Ongoing parity syncs: bulk AIC parity through 095f58a (#31, AIC-1788) and 23 further AIC PRs through ce2824e (#73).
  • AIC PyO3 operation unification carried across (#10); CI validation restored and merged packages reconciled (#9).
  • Release artifacts consolidated under a single packaging story (#35).

Unified CLI

  • Prediction and recommendation inputs unified on one interface (#29), with unified engine workflows stabilized (#43).
  • aiconfigurator command identity preserved so existing invocations keep working after the move (#26).
  • Strict aggregate SLA filtering (#12, AIC-1646).

Replay and Simulation

  • Typed agentic graph execution (#24).
  • Native vLLM host offload simulated (#57).
  • Routing regret diagnostics exposed (#54) and policy-neutral telemetry sampling added (#60).
  • Randomized synthetic request lengths (#5) and an engine replay parity skill (#47).

Performance — Rust Core Indexing

  • Grid hold lookups indexed (#51).
  • DSV4 top-k interpolation indexed (#55); DSV4 context grid rebuilds avoided (#66).
  • UtilGrid lookup indexed and cached (#58).
  • FP8 static delta lookup indexed (#65).

Sweeper and Generator

  • Sweeper candidates lowered into deployment requests (#11).
  • Canonical Sweeper result semantics defined (#15, AIC-1471).

Configuration and Support Matrix

  • Server knob taxonomy and backend enums (#67).
  • Measured FPE support matrix published (#41, AIC-1819).

Governance, Licensing and Compliance

  • CODEOWNERS policy (#8), contribution and dependency policy (#28), DCO (#32) and a security policy (#45).
  • OSRB filing requirements addressed (#92, covering #81); third-party license notices completed (#148, cherry-pick of #95); SPDX checks hardened and Jinja headers corrected (#150, cherry-pick of #146).
  • Repository governance files canonicalized (#149, cherry-pick of #106).

Migration and Compatibility Notes

Please review these before upgrading or re-pointing automation:

  • AISimulate is the successor home for AIConfigurator. The full AIConfigurator tree now lives in this repository (#2); the AISimulate package no longer ships from Dynamo (#3).
  • The aiconfigurator command still works. Command identity is preserved deliberately (#26), so existing CLI invocations and scripts do not need to be rewritten for this release.
  • Prediction and recommendation now share one input surface (#29, #43) — worth re-checking any wrapper that constructed inputs for the two paths separately.
  • Aggregate SLA filtering is strict (#12); see the E2E SLA mapping doc for how AIC request latency translates (#40).
  • Sweeper result semantics are now canonical (#15) — downstream consumers parsing Sweeper output should validate against the new definition.
  • Documentation paths and static pages were rebranded (#7, #50); bookmarks and deep links into the old AIConfigurator docs tree may need updating.
  • **plotext is constrained ...
Read more