NVBench Python Library (v0.3.0)

These are the release notes for the cuda-bench Python package version 0.3.0, dated August 5th, 2026.

cuda-bench is in "experimental" status, meaning that its API and feature set can change quite rapidly.

New Features

• Support for using decorators to register Python benchmarks, set axes and options
• Support for using timers to control timed bracket with cuda.bench.State.exec(launchable, timer=True)
• New cuda.bench.results.BenchmarkResult class for parsing JSON output
• New features from NVBench C++ library:
  • Improved performance of stdrel and entropy stopping criteria
  • New sample-count stopping criterion with criterion parameter --target-samples <count>
  • Control for number of warm-up runs with --cold-warmup-runs <count> and --cold-max-warmup-walltime <seconds>
  • Control for batch size in batched-measurements loop with --batch-target-time <seconds>
• Updates to scripts:
  • New nvbench-compare-robust script for robust benchmark data comparison that takes bulk data into account to classify comparison as FAST, SLOW, SAME, AMBG (undecided) with diagnostics and ability to generate matched data loader for further processing or visualization
  • New nvbench-json-summary script to recreate Markdown table generated by NVBench-instrumented script from saved JSON output
  • All scripts load third-party dependencies lazily and report actionable error message upon failure to import
• cuda-bench packages NVBench library installation prefix, including headers, libraries and CMake scripts. Location of the prefix can be obtained via executable module: python -m cuda.bench.nvbench --prefix

Notable Bug Fixes

• Issue with interruptibility of Python benchmark scripts was fixed
• Issue with editable install of the package was fixed

---

What's Changed

• Update python examples by @oleksandr-pavlyk in #331
• Correct description/hint entries for summaries with name "Noise" by @oleksandr-pavlyk in #335
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in #333
• Modularize color handling by @oleksandr-pavlyk in #336
• Port changes to the package from #323 by @oleksandr-pavlyk in #337
• Correct Python API signature of State.get_axis_values_as_strings by @oleksandr-pavlyk in #346
• Correct cuda cccl examples per change in api by @oleksandr-pavlyk in #353
• Create directories for output files by @oleksandr-pavlyk in #360
• Add .coderabbit.yaml following in footsteps of CCCL by @oleksandr-pavlyk in #359
• Implement CLI option to control warmups for cold measurements by @oleksandr-pavlyk in #339
• Make nvbench::main exported cmake target map to static library by @oleksandr-pavlyk in #350
• Enable building of NVBench on Windows by @oleksandr-pavlyk in #362
• Provide BenchmarkResult class for parsing JSON output of NVBench-instrumented benchmarks by @oleksandr-pavlyk in #356
• Implement decorators to register benchmarks add axis and options by @oleksandr-pavlyk in #347
• Implement Timer, and support State.exec(fn, timer=True) by @oleksandr-pavlyk in #364
• Fix exception safety by @oleksandr-pavlyk in #358
• Add stopping criterion sample count by @oleksandr-pavlyk in #341
• Add python api for cold warmup parameters by @oleksandr-pavlyk in #363
• Fix docutil error when building docs by @oleksandr-pavlyk in #365
• Add Windows support by @mfranzrebsal in #354
• nvbench::cpu_timer changed to use steady_clock by @oleksandr-pavlyk in #371
• include cleanup across nvbench/ by @oleksandr-pavlyk in #377
• docs: separate measurement options from stopping criteria by @omribz156 in #373
• Reduce stdrel criterion complexity and ensure termination by @oleksandr-pavlyk in #374
• Updated devcontainer imageset to 26.08 by @oleksandr-pavlyk in #381
• Remove unused python_wheel section in ci/matrix by @oleksandr-pavlyk in #382
• Introduce robust metrics by @oleksandr-pavlyk in #379
• Build and test cuda-bench wheels for Python 3.10-3.14 by @oleksandr-pavlyk in #380
• Rename IR to IQR in */ir/absolute and */ir/relative tags by @oleksandr-pavlyk in #385
• Use UV to build Python wheels by @oleksandr-pavlyk in #400
• Avoid std::back_insert_iterator of fmt::memory_buffer in fmt::format by @oleksandr-pavlyk in #397
• Update NVbench to use fmt 12.2 by @oleksandr-pavlyk in #398
• Corrections for python/README.md by @oleksandr-pavlyk in #401
• Implemented the NVBench PR review workflow. by @oleksandr-pavlyk in #403
• Harden quartile statistics and measurement timeout handling by @oleksandr-pavlyk in #392
• Store jsonbin sidecar paths relative to JSON output file location by @oleksandr-pavlyk in #405
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in #409
• Add Python 3.14 to classifiers of project section in TOML file by @oleksandr-pavlyk in #413
• Fix cuda_timer move ownership by @fallintoplace in #418
• Readability improvement by using device_clock alias by @oleksandr-pavlyk in #419
• Bump CTK 13.2->13.3 by @oleksandr-pavlyk in #420
• Reject --jsonbin stdout or --jsonbin stderr by @oleksandr-pavlyk in #423
• Avoid redefining existing preprocessor variable in version.cuh by @oleksandr-pavlyk in #428
• Made some classes immovable, in addition to already being not-copyable by @oleksandr-pavlyk in #422
• Cuda bench package headers cmake and main by @oleksandr-pavlyk in #417
• Fix editable install of cuda-bench by @oleksandr-pavlyk in #430
• Evolution of nvbench compare that uses bulk data by @oleksandr-pavlyk in #386
• Fix GCC 14 maybe-uninitialized warning by @PointKernel in #434
• Fix test failure for nvbench tooling deps in editable install by @oleksandr-pavlyk in #435
• Destroy CUDA stream after timing is finished by @upsj-nv in #438
• Changed plot title in nvbench_compare.py by @oleksandr-pavlyk in #432
• Implement cuda.bench.State.set_stream(provider) by @oleksandr-pavlyk in #443
• Workflow for executing examples by @oleksandr-pavlyk in #445
• Add nvbench-compare-robust --plot-output/--plot-along-output options by @oleksandr-pavlyk in #433
• Fix duplicate fmt ALIAS targets when fmt is already provided by @igorpeshansky in #447
• Add CLI option to set target-time parameter for batch measurement loop by @oleksandr-pavlyk in #439
• Bump torch from 2.11.0 to 2.13.0 in /python/examples/requirements by @dependabot[bot] in #446
• print_constraints_candidate return status of 1 should not fail job by @oleksandr-pavlyk in #450

New Contributors

• @mfranzrebsal made their first contribution in #354
• @omribz156 made their first contribution in #373
• @fallintoplace made their first contribution in #418
• @upsj-nv made their first contribution in #438
• @igorpeshansky made their first contribution in #447
• @dependabot[bot] made their first contribution in #446

Full Changelog: python-0.2.1...python-0.3.0