Skip to content

v4.0.0

Choose a tag to compare

View all tags
@jcfr jcfr released this 09 Mar 19:47
v4.0.0
874dc9d
This commit was signed with the committer’s verified signature.
jcfr Jean-Christophe Fillion-Robin
GPG key ID: A35021947D88E8B3
Verified
Learn about vigilant mode.

Release Artifacts

See supported platforms for compatibility.

Release Notes

New Features and Improvements

Core

  • GPU-resident graph (C++): Holoscan SDK introduces a new execution
    mode, called GPU-resident graph execution, that keeps the CUDA compute pipeline on
    the GPU for the lifetime of the application, reducing CPU scheduling
    overheads, CUDA kernel launch cost, CPU-GPU coordination and improving
    deterministic, low-latency behavior.

    • GPU-resident operators and fragments: Operators inheriting from holoscan::GPUResidentOperator use device memory for ports and are captured into CUDA Graphs. All operators in a fragment must be GPU-resident for that fragment to use GPU-resident graph execution; currently only linear chains of operators are supported.
    • Port declaration: Ports can be declared by memory block size (executor-allocated shared device memory) or by device pointer (operator-managed); see the GPU-resident user guide for connection semantics.
    • Data ready handler (optional): A separate GPU-resident fragment can be registered as a data ready handler to run at the start of each iteration and decide when input data is ready, enabling sensor-driven pipelines with GPU-direct technologies (e.g., Holoscan Sensor Bridge) without host CPU involvement.
    • Host control API: After run_async(), use Fragment::gpu_resident() to signal data ready, poll result ready, and configure timeout and optional host sync; see the GPU-resident user guide and examples.
    • GPU-resident graph execution is supported in C++ only; Python support is planned for a future release.

  • Data Flow Tracking - programmatic access and probing: Data flow tracking now supports richer observability and targeting of arbitrary operators.

    • Programmatic access in operators: Operators can read data flow tracking information during execution via get_data_flow_tracking_label() (C++/Python). This returns a MessageLabel for a given input port, with path names, number of paths, and timing information, enabling adaptive behavior, debugging, in-operator and runtime analysis without waiting for post-run results.
    • Tracking to arbitrary operators: Use DataFlowTracker::add_probe_operator() to treat any intermediate (non-root, non-leaf) operator as a probe point. The tracker then reports latency from root operators to the probed operator and the number of messages published by that operator. Probe operators are currently validated with DoubleBufferTransmitter/DoubleBufferReceiver and default connection configurations; see the Data Flow Tracking documentation for details and the flow_tracker example for usage.
Operators/Resources/Conditions
  • New classes related to publish/subscribe (pub/sub) messaging were added (PubSubContext, PubSubReceiver, PubSubTransmitter) as wrappers around underlying GXF pub/sub interfaces. In this release, Holoscan does not ship a concrete pub/sub backend implementation, so these APIs should be considered experimental scaffolding for future releases. Relatedly, IOSpec now includes topic and qos methods to describe intended pub/sub topic and QoS settings, which will become active once a concrete backend is provided.
Examples
  • The flow_tracker example demonstrates the new data flow tracking capabilities: programmatic access via get_data_flow_tracking_label() inside an operator and tracking to intermediate operators via add_probe_operator().
  • Added gpu_resident_example, gpu_resident_input, and gpu_resident_inference_example demonstrating GPU-resident execution: operator chains, device memory ports, (in gpu_resident_input) device-driven data ready control, and (in gpu_resident_inference_example) GPU-resident inference with the TensorRT (TRT) backend.
  • Added matx_allocator example (examples/matx/matx_allocator/) demonstrating how to use the MatXAllocator adapter with RMMAllocator in a Holoscan pipeline with DLPack tensor interop.
Holoviz module
Holoinfer module
Utils
  • Added MatXAllocator utility class (holoscan/utils/matx_allocator.hpp) — a lightweight adapter that enables any Holoscan SDK allocator (e.g., RMMAllocator, BlockMemoryPool, StreamOrderedAllocator) to be used with MatX GPU tensor operations. Supports stream-aware async allocation/deallocation when used with CudaAllocator-derived allocators and a CUDA stream.
HoloHub
Source build & Release Artifacts

  • Added Holoscan Sensor Bridge 2.5.0
    basic feature functionality in the Holoscan SDK developer container environment. Holoscan Sensor Bridge libraries and
    tools are built with RDMA over Converged Ethernet (RoCE) support targeting discrete GPU platforms with
    ConnectX Network Interface Card (NIC) such as DGX Spark or IGX platforms. Libraries may be found under
    the /opt/nvidia/hololink path in the discrete GPU developer containers. Please refer to the Holoscan
    Sensor Bridge User Guide for
    detailed requirements.

    • Note that the containerized Holoscan Sensor Bridge libraries do not support integrated GPU networking stacks (Jetson, IGX Orin 500). Please rebuild Holoscan Sensor Bridge from source for iGPU platform support.
    • Holoscan Sensor Bridge support for x86_64 configurations is experimental and not formally verified. Holoscan Sensor Bridge binaries have been included in the Holoscan SDK x86_64 development container to support developers with Holoscan Sensor Bridge emulation via software loopback. We do not recommend using real Holoscan Sensor Bridge hardware with x86_64 platforms at this time.
    • Please run the container as the root user to enable full Holoscan Sensor Bridge runtime features.

  • Updated Holoscan SDK CMake import rules with improved handling for external dependencies.
    Holoscan SDK C++ and Python binary distributions (Debian packages, Python wheels, etc) traditionally
    contain runtime libraries and development rules for dependencies such as RMM, SPDLOG, YAML-CPP and more,
    to reduce the downstream developer burden. In this version we improve "holoscan-config.cmake" and
    associated files in the Holoscan SDK binary distribution to describe embedded dependencies with
    dedicated project configuration files such as "rmm-config.cmake", and rely on CMake's "find_package"
    mechanism to relax dependency embeddings.

    • Downstream projects may override Holoscan SDK embedded CMake transitive dependencies with "find_package":

      # Find RMM from a custom installation in your development environment
find_package(rmm HINTS path/to/my/rmm)

# Holoscan SDK will favor the custom RMM installation over its own embedded distribution
find_package(holoscan)

    • Downstream projects may depend directly on the Holoscan SDK embedded dependencies:

      # Populate any embedded dependencies not already provided
find_package(holoscan)

# If not set in advance, will use the RMM installation embedded in the Holoscan SDK distribution
find_package(rmm)     # result: rmm_DIR path is already set to the Holoscan SDK installation
...
target_link_libraries(mylibrary PUBLIC rmm::rmm)   # dependency imported targets are available

    • Note that some header-only dependencies (concurrentqueue, MatX) remain in the Holoscan SDK
      distribution without explicit CMake imported targets to describe them.

  • Removed the legacy Holoscan SDK runtime Dockerfile as not supported. Please refer to
    Holoscan CLI and
    HoloHub sample apps for guidance and examples for
    writing an optimized Dockerfile for your Holoscan-based project.

  • The NGC holoscan_dev_deb resource (Debian packages up to HSDK 0.6) has been archived and
    removed from NGC. Debian packages are now distributed exclusively via
    NVIDIA Holoscan downloads; use the container
    image, Python wheel, or Debian package options listed in the Release Artifacts section above.

Documentation

Breaking Changes

  • Removed the holoscan.cli Python module stub that was deprecated in v2.9. Users who need CLI
    functionality should install the standalone holoscan-cli
    package via pip install holoscan-cli.

Bug fixes

Issue Description
5601128 On IGX Orin iGPU, fix segmentation fault by ensuring matx::make_tensor does not fall back from managed memory to host-pinned memory during allocation
5898338 Fix error where InferenceOp ignores CudaStreamPool passed positionally (only the named Arg works)

Known Issues

Issue Description
5211869 On IGX Orin dGPU, error "Failed to start server on 0.0.0.0:10002" is reported when debugging the Distributed Endoscopy Tool Tracking application in VS Code.
5606400 On AGX Thor Jedha, v4l2_camera_usb_webcam Python example application fails with a segmentation fault.
5928213 On IGX Thor dGPU (Blackwell), holoviz_conditions C++ example application launches with no frame detected and reports the error VK_KHR_present_wait is not available.
5929120 On AGX Orin iGPU (Concord), running public/examples/v4l2_camera/python/v4l2_camera.py can intermittently crash with Fatal Python error: take_gil: PyMUTEX_LOCK(gil->mutex) failed or SIGSEGV; investigation is ongoing.
5933258 On AGX Orin iGPU (Concord), the V4L2 camera app can crash at exit/teardown with SIGILL/segmentation fault (observed around libXfixes.so.3) after scheduler stop; root cause analysis is in progress.
5946872 CuPy fails with a Permission denied error when the container runs as a non-root user other than the default Ubuntu user (UID 1000), e.g. docker run -u 1001:1001, because it writes the kernel cache to $HOME/.cupy/kernel_cache, which may not exist or be writable. Workaround: Set CUPY_CACHE_DIR=/tmp/.cupy/kernel_cache before running Python (or add ENV CUPY_CACHE_DIR=/tmp/.cupy/kernel_cache in your Dockerfile).
Assets 2
Loading