Skip to content

v1.6.1

Latest
Latest

Choose a tag to compare

View all tags
@podkidyshev podkidyshev released this 02 Jun 14:59
v1.6.1
bbd1628
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

New Changes

  • Added support for the following workloads:
    • vLLM - LLM serving benchmark support with Slurm execution, disaggregated prefill/decode mode, multi-node serving, reporting, DSE metrics, and NIXL-related options
    • SGLang - LLM serving benchmark support sharing the common vLLM/SGLang serving flow, reporting, health checks, and multi-node execution
    • NIXL EP - NIXL Expert Parallelism workload with Slurm command generation, log parsing, reporting, and tests
  • Added DSE reporting, including richer visualization of design-space exploration results and best-configuration selection
  • Added report generation for MegatronRun and OSU benchmarks
  • Added support for CNI specification configuration for NCCL and AI Dynamo workloads on Kubernetes

Backward Compatibility Notes

  1. AI Dynamo configuration schema

    • Worker settings now use explicit prefill_worker and decode_worker blocks with nested args.
    • Older fields such as prefill-cmd, decode-cmd, top-level worker parallelism keys, run_script, and huggingface_home_container_path should be migrated to the new schema.

  2. Megatron-Bridge configuration schema

    • model_family_name and model_recipe_name replace the earlier model_name and model_size fields.
    • time_limit is now taken from the test run rather than cmd_args.
    • A Megatron-Bridge git repo only overrides the container copy when mount_as = "/opt/Megatron-Bridge" is set.

  3. Custom workload implementations

    • Custom workloads that override constraint_check(self, tr) should update the method signature to accept the new system argument.

LLM Serving Improvements

CloudAI now includes first-class support for vLLM and SGLang serving workloads. The implementation includes shared serving infrastructure, Slurm command generation, result reporting, disaggregated prefill/decode support, two-node serving flows, custom health check endpoints, and more robust startup, shutdown, and cleanup handling. vLLM also supports DSE metrics, NIXL thread options, boolean flag handling, and constraint checks.

Megatron and Megatron-Bridge Improvements

Megatron-Bridge support was updated for r0.3.0 recipes and improved configuration handling. GPU counts can be derived from the system configuration, time limits are managed by the test run, VP parameters are handled more reliably, and status checks reduce false passes. MegatronRun now has report generation support and improved success detection, including timeout handling.

NIXL, Kubernetes, and Networking

NIXL workloads gained a new EP workload, updated CLI argument handling, support for separate ETCD containers, improved ETCD failure handling, safer mount cleanup, and installable fixes around nested Docker image paths and submodules. Kubernetes support was improved with CNI spec handling for NCCL and AI Dynamo, while NCCL Kubernetes tests were refactored for better reuse and temporary-resource management.

Reporting, Configuration, and Parsing

Reporting now includes DSE reports, OSU benchmark reports, MegatronRun reports, and reward override support for constraint failures. Configuration handling is more robust with improved duplicate-key errors, system config detection, path expansion/storage, first-sweep messaging, and agent configuration/caching updates.

Architecture, Reliability, and Tooling

Job monitoring no longer relies on asyncio, heavy imports are blocked at module level, and command shell checks no longer run during object creation. Slurm handling was improved around node exclusion, reservation nodes, GPU resource requesting, and propagation of extra Slurm arguments. Tooling was refreshed with pre-commit, updated CI workflows, uv usage in CI, Node 24-compatible GitHub Actions, broader tests organized by system/workload, and dependency updates.

Documentation

Documentation was expanded for vLLM, SGLang, NIXL EP, Systems, workload requirements, reporting, troubleshooting, and tutorial/user guide content. Workload pages and release configurations were updated to match the new workloads and configuration flows.

All Changed

New Contributors

Full Changelog: v1.5.0...v1.6.1

Contributors

karya0, amaslenn, and 8 other contributors
Assets 2
Loading