Skip to content

Implement cuda.bench.State.set_stream(provider) - #443

Open
oleksandr-pavlyk wants to merge 6 commits into
NVIDIA:mainfrom
oleksandr-pavlyk:implement-cuda-bench-state-set-stream
Open

Implement cuda.bench.State.set_stream(provider)#443
oleksandr-pavlyk wants to merge 6 commits into
NVIDIA:mainfrom
oleksandr-pavlyk:implement-cuda-bench-state-set-stream

Conversation

@oleksandr-pavlyk

Copy link
Copy Markdown
Collaborator

The stream provide type is expected to implement __cuda_stream__ protocol, and not be an instance of type cuda.bench.CudaStream.

The reason is that cuda.bench.CudaStream type is not user-constructible. It is returned by cuda.bench.State.get_stream method. It stores reference to the stream stored in nvbench::state referenced from cuda.bench.State.

Disallowing instances of cuda.bench.CudaStream as argument to set_stream avoids footguns.

Closes #440

The stream provide type is expected to implement __cuda_stream__
protocol, and not be an instance of cuda.bench.CudaStream.

The reason is that cuda.bench.CudaStream type is not
user-constructible. It is returns by cuda.bench.State.get_stream
method. It stores reference to stream stored in nvbench::state
referenced from cuda.bench.State.

Disallowing cuda.bench.CudaStream as argument to set_stream
avoid footguns.
@oleksandr-pavlyk

This comment was marked as outdated.

@coderabbitai

This comment was marked as outdated.

@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f45da714-8fd8-40a7-baf8-f2be78bc3c3c

📥 Commits

Reviewing files that changed from the base of the PR and between e57ecad and 4287b99.

📒 Files selected for processing (3)
  • python/cuda/bench/__init__.pyi
  • python/src/py_nvbench.cpp
  • python/test/test_cuda_bench.py

📝 Walkthrough

Summary by CodeRabbit

  • New Features
    • Added support for configuring benchmark state with external CUDA stream providers.
    • Providers can expose CUDA stream handles through the standard __cuda_stream__ protocol.
    • Stream providers remain available while configured and can be replaced as needed.
    • Added validation and clear errors for unsupported or invalid providers.
    • External stream handles are now reflected in benchmark results.
    • Benchmark execution callbacks now run with the Python GIL held.

Walkthrough

The PR adds the _SupportsCudaStream protocol and State.set_stream. The binding validates external stream providers, converts version-0 protocol results to CUDA streams, preserves provider lifetime, and adds CPU-only coverage.

Changes

External CUDA stream support

Layer / File(s) Summary
Stream provider contract
python/cuda/bench/__init__.pyi
Defines _SupportsCudaStream with __cuda_stream__ and exposes State.set_stream.
Stream extraction, binding, lifetime, and validation
python/src/py_nvbench.cpp, python/test/test_cuda_bench.py
Validates providers, rejects CudaStream, registers changed handles, preserves provider lifetime, propagates handles to launches, and tests valid and invalid inputs.
Execution callback documentation
python/src/py_nvbench.cpp
Documents that State.exec, run_all_benchmarks, and launcher callbacks run with the Python GIL held.
API documentation validation
python/test/test_cuda_bench.py
Validates the State.set_stream docstring.

Assessment against linked issues

Objective Addressed Explanation
Add cuda.bench.State.set_stream(stream_provider) for non-CudaStream objects supporting __cuda_stream__. [#440]
Set the stream only when the protocol handle differs from the handle held by the underlying state. [#440]

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]

This comment was marked as resolved.

Replace pybind11 keep_alive with an explicit per-state provider cache so
repeated State.set_stream calls release previously installed providers instead
of retaining every historical stream provider.
@oleksandr-pavlyk

This comment has been minimized.

@coderabbitai

This comment has been minimized.

Add test for repeated set_stream of the same external
owning stream Python object, with expectation that it
does not double-free.
Docstring should not require that external stream provider must own
stream, only that it keeps it alive.

Added notes that both run_all_benchmarks and State.exec evaluate
with GIL held.

Add note about intended way of using State.set_stream
@oleksandr-pavlyk

This comment was marked as outdated.

@coderabbitai

This comment was marked as outdated.

@oleksandr-pavlyk oleksandr-pavlyk self-assigned this Aug 2, 2026
@github-project-automation github-project-automation Bot moved this to Todo in CCCL Aug 2, 2026
@oleksandr-pavlyk oleksandr-pavlyk moved this from Todo to In Review in CCCL Aug 2, 2026
@oleksandr-pavlyk
oleksandr-pavlyk marked this pull request as ready for review August 2, 2026 12:15
@oleksandr-pavlyk

Copy link
Copy Markdown
Collaborator Author

@PointKernel This PR fixes the limitation mentioned in NVIDIA/cuvs#2371 (comment) that prompted filing for #440

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: In Review

Development

Successfully merging this pull request may close these issues.

Support cuda.bench.State.set_stream method

1 participant