Repository navigation
Releases
AISimulate Release v0.13.0
AISimulate - Release 0.13.0
Summary
AISimulate 0.13.0 pairs with Dynamo 1.6.0. This release completes the move from AIConfigurator to AISimulate and adds forward-pass model (FPM) self-service onboarding for vLLM, public accuracy reporting, experimental AgentX trace replay, KV cache offload simulation, and broader model and serving-mode coverage.
pip install aisimulate==0.13.0- Python 3.11 to 3.13. Python 3.14 is not yet supported; pip reports a resolution failure without a clear hint.
- Wheels: Linux x86-64, Linux aarch64, macOS arm64.
- Validated pairing: AISimulate 0.13.0 with Dynamo 1.6.0.
Key Highlights
FPM self-service onboarding (vLLM)
Onboard a new model on your own GPUs by measuring its forward passes. aisimulate onboard init declares the model, target hardware and worker topology. onboard plan writes the collection plan plus predict and recommend configs. onboard collect-fpm previews the collection, then runs Dynamo's vLLM self-benchmark with --execute --smoke and --execute. onboard validate-collection checks the measured data, and onboard finalize produces the simulation inputs. Further subcommands cover runtime probing, importing saved observations, serving validation and resumable checkpoints (#196, #248). Guide: FPM self-service.
- Runtime probing ships audited instrumentation for vLLM 0.27.0 only; Dynamo 1.6.0 runs vLLM 0.31.0.
- Collecting dense models needs Dynamo 1.6.0 GA (it includes ai-dynamo/dynamo#16804), and very large checkpoints may not complete (see Known issues).
- SGLang and TRT-LLM self-service are on the roadmap.
Public accuracy reporting
Forward-pass and end-to-end accuracy dashboards are published at https://ai-dynamo.org/aisimulate/fpm-accuracy/ and https://ai-dynamo.org/aisimulate/e2e-accuracy/ (#253, #368, #375).
AgentX trace replay (experimental)
Replay agentic coding traces offline for vLLM and SGLang: direct Weka AgentX ingestion, deterministic play outcomes, seeded snapshots and continuous duration profiles (#115, #121, #122, #190, #207, #307). In release testing, a finite AgentX trace completed all 722 requests with 275 MB peak memory.
KV cache offload simulation
AISimulate models three KV cache tiers: G1 (GPU memory), G2 (host memory) and G3 (storage).
- vLLM G2 offload, local and cluster-shared, including attention DP and prefill/decode disaggregation (#358, #381).
- vLLM G3 write-through offload for
predict(#203). G3 restores through G2 and requires aggregated vLLM replay with attention DP 1; it is not available with prefill/decode disaggregation, AgentX replay orrecommend. - Hybrid models: Kimi K3 cache geometry is inferred from the model configuration (#293), and the G1 state cache for Mamba/KDA layers can be sized manually (#243).
- Partial prefix hits via vLLM
prefix_match_unit(#314). For Kimi K3, see Known issues.
Better recommendations
recommendcan size for the fewest GPUs:optimization.target: min_gpusselects the smallest configuration found that meets the SLA at a fixed request rate or concurrency. Request-rate traffic also needsoptimization.constraints.min_goodput_rps(#270).- Bayesian search no longer shows the memory growth seen in 0.12: a bounded release test (64 trials at parallelism 2) peaked at 0.87 GB.
- Heterogeneous prefill and decode hardware: set
engine.workers.prefill.hardwareorengine.workers.decode.hardwarein disaggregatedpredictandrecommend(#186, #204).
Clearer output and errors
- Predictions print their estimator, timing source, data version and KV precision.
- Clear errors for unsupported models, missing data versions, zero-completion runs and unsupported FP8 compute (for example FP8 on A100). Parallel shapes that do not split the model's attention heads are skipped instead of failing.
--overwritenow validates the configuration (YAML, unknown sections and fields, value types, missing stacks or output adapters) before clearing previous outputs. A misspelled model or hardware name still clears them; see Known issues.
Serving modes and models
- Serving modes, all in
predict:- Attention-FFN disaggregation (AFD), modeled analytically and configured with
engine.mode: afdandengine.afd; also inrecommend(#107 to #111). - Analytical encode/prefill/decode disaggregation: an encoder pool (
engine.workers.encoder) with fixed image inputs, in front of aggregated or disaggregated workers; also inrecommend. Qwen3-VL models are supported in this release (#156). - Context parallelism:
parallelism.prefill_context(SGLang models only) andparallelism.decode_context(vLLM and SGLang, model-dependent),predictonly (#240). - TRT-LLM disaggregated serving (#77) and attention DP with different sizes per prefill and decode pool (#105).
- N-gram prompt-lookup speculative decoding via
engine.speculation, for vLLM with op-level timing; also inrecommend(#227).
- Attention-FFN disaggregation (AFD), modeled analytically and configured with
- Models:
- DeepSeek V4.1 Flash: FPM self-service plans it with the required eager execution, and measured SGLang FPM profiles (H200, B200 and GB200 at TP4; B300 at TP2 and TP4; GB300 at TP2) load through the Python SDK.
predicthas no op-level path for it yet (see Known issues). - Kimi K3 (
moonshotai/Kimi-K3), and Qwen3.8-Max (Qwen/Qwen3.8-2.4T-A95Band-FP8) on vLLM and SGLang. Qwen3.8-Max needsbackend_version: next; the default data version has no FP8 MoE measurements for it. - Vision encoder modeling for Qwen3.5, Kimi K2.5 and Gemma 4 through the image options of the bundled
aiconfiguratorcompatibility command (#84, #86).
- DeepSeek V4.1 Flash: FPM self-service plans it with the required eager execution, and measured SGLang FPM profiles (H200, B200 and GB200 at TP4; B300 at TP2 and TP4; GB300 at TP2) load through the Python SDK.
- New measurements: opt-in FP8 FlashInfer TRT-LLM MoE data for Qwen3.8-Max on SGLang 0.5.17 (B200, GB200 and GB300; select with
engine.moe_kernel_source: sglang_flashinfer_trtllm_moe), GB200 multi-node communication data, and a DeepEP low-latency decode communication model.
Dynamo 1.6.0 integration
Tested with Dynamo 1.6.0 release candidates: Planner estimator, profiler rapid mode, mocker with AISimulate timing, frontend AISimulate router, and the DynoSim tutorial.
Also in this release
- Slurm deployment and benchmark bundles:
--deployment-target slurmin the bundledaiconfiguratorcompatibility command (or theaisimulate.generatorAPI) writes a single-node Slurm (Pyxis/Enroot) bundle with a deploy job and an AIPerf benchmark job for vLLM, SGLang and TRT-LLM, aggregated or disaggregated (#502, #503 and earlier). See deployment generation. - Output adapter interface:
recommend --output NAMEruns an installed output adapter that writes extra artifacts or observes candidates live (#334). No adapters ship with AISimulate; see the output adapter ABI. - Faster replay: scheduler queue indexes, cheaper SGLang admission, faster prompt hashing, reduced bookkeeping (#348, #356, #386, #397).
- Agent skills for developing with AISimulate (#322).
Migration and Compatibility Notes
- The
aiconfiguratorandaiconfigurator_corePython import namespaces are removed. Import fromaisimulateandaisimulate_coreinstead (for exampleaiconfigurator_core.sdkbecomesaisimulate_core.sdk). Theaiconfiguratorcommand remains for compatibility workflows. The Rust crate is renamed fromaiconfigurator-coretoaisimulate-core; Cargo users may alias it under the old dependency key. Estimator construction takes a singleForwardPassPerfModelConfig. Convert any pickles that encode the old module paths before upgrading. See the migration guide. - Host resource supervision. Every
predictandrecommendrun now checks host CPU and memory budgets and runs inside a super...
AISimulate v0.12.1
AISimulate - Release 0.12.1
Summary
AISimulate 0.12.1 is a patch release on top of 0.12.0. It carries a single functional change: a replay fix that keeps timing and capacity on the selected database version (#344, cherry-picked from #291), together with the crate and wheel version bump to 0.12.1 (#340).
There are no new features, model additions, collector upgrades, API changes or deprecations in this release. Anyone running replay against an explicitly selected perf-database version should take this patch, since timing and capacity results could previously be resolved off a different version than the one selected. Upgrading from 0.12.0 requires no configuration or CLI changes.
Bug Fixes
Replay
Build and Packaging
- Bump
release/0.12.1crate and wheel versions to 0.12.1 (#340).
Full Changelog
Full Changelog: v0.12.0...v0.12.1
AISimulate Release v0.12.0
AISimulate - Release 0.12.0
Summary
AISimulate 0.12.0 is the inaugural release of the AISimulate repository. It is the convergence point for two codebases: DynoSim and AIConfigurator. From this release forward, AISimulate can predict and recommend LLM inference configurations for TRT LLM, vLLM, and SGLang, and Dynamo. Furthermore, users can simulate both static traffic (ISL/OSL/concurrency) and dynamic traffic (for e.g. a mooncake trace). Because the two trees kept moving during the migration, parity is maintained by explicit bulk syncs — AIC parity through 095f58a (#31, AIC-1788), 23 further AIC PRs through ce2824e (#73), and the AIC PyO3 operation unification (#10) — with CI validation restored across the merged packages (#9).
The CLI. ASimulate now supports “predict” and “recommend” commands in on one interface (#29), while the existing aiconfigurator command identity is deliberately preserved so current users are not broken by the move (#26). Strict aggregate SLA filtering arrives (#12, AIC-1646) alongside documentation mapping AIC request latency onto the E2E SLA definition (#40).
Replay engine. Typed agentic graph execution lands (#24), joined by randomized synthetic request lengths (#5), routing regret diagnostics (#54), policy-neutral telemetry sampling (#60), native vLLM host offload simulation (#57), and an engine replay parity skill (#47) backed by vLLM prefix cache lifecycle coverage (#33).
A concentrated round of Rust core indexing work removes repeated hot-path scans: Grid hold lookups (#51), DSV4 top-k interpolation (#55), UtilGrid lookup indexing and caching (#58), FP8 static delta lookup (#65), and elimination of redundant DSV4 context grid rebuilds (#66).
On the Sweeper and generator side, candidates are lowered into deployment requests (#11) and canonical result semantics are defined (#15, AIC-1471). Configuration gains a server knob taxonomy with backend enums (#67), and a measured FPE support matrix is published (#41, AIC-1819). Because this is a new public repository, a substantial share of the release is governance, licensing and compliance groundwork: CODEOWNERS (#8), contribution and dependency policy (#28), DCO (#32), a security policy (#45), OSRB filing requirements (#92), third-party license notices (#148) and hardened SPDX checks (#150).
Note: this is the first tagged release of the AISimulate repository, so the changelog below covers the full history of the repo rather than a delta against a previous tag.
Key Highlights
Repository Migration and AIC Parity
- AISimulate migrated out of Dynamo into its own package (#3, AIC-1703) and the full AIConfigurator tree migrated in (#2, AIC-1702).
- Ongoing parity syncs: bulk AIC parity through
095f58a(#31, AIC-1788) and 23 further AIC PRs throughce2824e(#73). - AIC PyO3 operation unification carried across (#10); CI validation restored and merged packages reconciled (#9).
- Release artifacts consolidated under a single packaging story (#35).
Unified CLI
- Prediction and recommendation inputs unified on one interface (#29), with unified engine workflows stabilized (#43).
aiconfiguratorcommand identity preserved so existing invocations keep working after the move (#26).- Strict aggregate SLA filtering (#12, AIC-1646).
Replay and Simulation
- Typed agentic graph execution (#24).
- Native vLLM host offload simulated (#57).
- Routing regret diagnostics exposed (#54) and policy-neutral telemetry sampling added (#60).
- Randomized synthetic request lengths (#5) and an engine replay parity skill (#47).
Performance — Rust Core Indexing
- Grid hold lookups indexed (#51).
- DSV4 top-k interpolation indexed (#55); DSV4 context grid rebuilds avoided (#66).
- UtilGrid lookup indexed and cached (#58).
- FP8 static delta lookup indexed (#65).
Sweeper and Generator
- Sweeper candidates lowered into deployment requests (#11).
- Canonical Sweeper result semantics defined (#15, AIC-1471).
Configuration and Support Matrix
- Server knob taxonomy and backend enums (#67).
- Measured FPE support matrix published (#41, AIC-1819).
Governance, Licensing and Compliance
- CODEOWNERS policy (#8), contribution and dependency policy (#28), DCO (#32) and a security policy (#45).
- OSRB filing requirements addressed (#92, covering #81); third-party license notices completed (#148, cherry-pick of #95); SPDX checks hardened and Jinja headers corrected (#150, cherry-pick of #146).
- Repository governance files canonicalized (#149, cherry-pick of #106).
Migration and Compatibility Notes
Please review these before upgrading or re-pointing automation:
- AISimulate is the successor home for AIConfigurator. The full AIConfigurator tree now lives in this repository (#2); the AISimulate package no longer ships from Dynamo (#3).
- The
aiconfiguratorcommand still works. Command identity is preserved deliberately (#26), so existing CLI invocations and scripts do not need to be rewritten for this release. - Prediction and recommendation now share one input surface (#29, #43) — worth re-checking any wrapper that constructed inputs for the two paths separately.
- Aggregate SLA filtering is strict (#12); see the E2E SLA mapping doc for how AIC request latency translates (#40).
- Sweeper result semantics are now canonical (#15) — downstream consumers parsing Sweeper output should validate against the new definition.
- Documentation paths and static pages were rebranded (#7, #50); bookmarks and deep links into the old AIConfigurator docs tree may need updating.
- **
plotextis constrained ...