Holoscan SDK v0.4

Release Notes

New Features

This new release of Holoscan SDK brings the following main features:

• Python API
  A new Python API which allows for rapid prototyping and deployment of AI workflows has been added. The python API is accessible through the provided container or source build as well as PyPi.

• Inference module
  A new Holoscan Inference module (HoloInfer) has been added. The HoloInfer module facilitates designing and executing inference and processing applications through its APIs.

• Multi-AI Inference Extension
  A Multi AI Inference codelet is provided, which takes multiple AI inference related parameters and can ingest multiple GXF messages and transmits multiple tensors as the output via a single GXF Transmitter.

• Native Operators using C++/Python APIs
  The capability to write native operators in C++ and Python has been added and several sample Holoscan native operators are provided.

• FPGA Alpha-Blending
  The FPGA Alpha-Blending using AJA capture card allows very-low latency passthrough of the input video to output, while blending the AI inference overlay directly on the input signal on the FPGA on AJA PCIe card.

Changes from previous release

• The container available from NGC has been renamed and now includes development tools.
  Developers can build new applications in C++ and Python directly from the container.
• Holoviz operator is now used for all pipelines and handles compositing, blending, and visualization of RGB or RGBA images, masks, geometric primitives and text. The operator also supports headless rendering and streaming the rendered images to an output.
• Along with the distribution of Python package on PyPi, this release includes pre-compiled Debian package available on NGC

Issues Fixed

• Handle incorrect DISPLAY env configuration in Holoviz (3772272)

Please see the list of known issues below for more information.

Supported Platforms

Note: This release is intended for use with the listed platforms only. NVIDIA does not provide support for this release on products other than those listed below.

Description	Supported Version
Supported NVIDIA® Tegra® Linux Driver Package (L4T)	NVIDIA® Holopack 1.1 -- R34.1.2
Supported Jetson Platforms	Holoscan Developer Kits
Supported x86 Platforms	Ubuntu 20.04 with Turing/Ampere GPU
Supported Software for Clara AGX Developer Kit with NVIDIA® RTX6000 and IGX Orin Developer Kit with NVIDIA® A6000	NVIDIA® Driver 510.73.08 CUDA 11.6.1 TensorRT 8.2.3 GXF 2.5 AJA NTV2 SDK 16.2

Known Issues

This section supplies details about issues discovered during development and QA but not resolved in this release.

Issue	Description
3878494	Inference fails after tensorrt engine file is first created using BlockMemoryPool
3762996	nvidia-peermem.ko fails to load using insmod on Holoscan devkits in dGPU mode. Install nvidia-peer-memory following the RDMA instructions in the Holoscan SDK User Guide.
3655489	Installing dGPU drivers can remove nvgpuswitch.py script from the executable search path. Explicitly including /opt/nvidia/l4t-gputools/bin in the PATH environment variable ensures this script can be found for execution.
3599303	Linux kernel is not built with security hardening flags. Future releases will include a Linux kernel built with security hardening flags.
3633688	RDMA on the NVIDIA IGX Orin Developer Kit (holoscan-devkit) is not functional. PCIe switch firmware update fixed the issue. RDMA for the Clara AGX Developer Kit is functional and unaffected by this issue.
3834424	Ultrasound segmentation application is not functional on NVIDIA IGX Orin Developer Kit (holoscan-devkit) with iGPU configuration in deployment stack
3842899	High-Speed Endoscopy application is not supported in deployment stack.
3881725	VK_ERROR_INITIALIZATION_FAILED with segmentation fault while running High-Speed Endoscopy gxf/cpp app (workaround implemented in v0.4 and fix in available in 520 drivers)
3897810	Applications not working on x86_64 systems with multiple GPUs
3936290	Cannot run exclusive display from docker container