Releases: ROCm/aomp
rocm-6.2.0
ROCm release v6.2.0
AOMP Release 19.0-3
These are the release notes for AOMP 19.0-3 AOMP uses AMD developer modifications to the upstream LLVM development trunk. These differences are managed in a branch called the "amd-staging". This branch is found in a mirror of upstream LLVM found at https://github.com/ROCm/llvm-project. The amd-staging branch is constantly changing as it merges the upstream development trunk with its downstream development updates. The AMD modifications are experimental while under review for the upstream trunk. AOMP uses a snapshot of amd-staging at the commit ids and dates listed below. AOMP also includes builds of related ROCm components. We call AOMP a "standalone" build as it does not use or require ROCm with the exception of the kernel module (dkms) and libdrm which are often part of the Linux distribution. AOMP is isolated from any ROCm installations by installing into /usr/lib/aomp and the use of RPATH for runtime libraries.
For AOMP 19.0-3, the last LLVM trunk commit is 40954d7f9bb38b2407fe48a524befc5216f13cccon July 22, 2024. This was the last trunk commit before the trunk forked to LLVM-20. The last amd-only commit is baa883c3ad5d70e1f4da5b6f80f6d06c00b73c3a on July 22, 2024. These commits form a frozen branch now called "aomp-19.0-3". See https://github.com/ROCm/llvm-project/tree/aomp-19.0-3.
The integrated ROCm components for this AOMP release were built with ROCM 6.1.2 sources.
This is the 3rd AOMP release based on upstream LLVM 19 development. Since the LLVM trunk has moved to development of LLVM 20, the next AOMP release will be based on LLVM-20.
Changes since AOMP 19.0-2
- Support for requires atomic default mem order clause was added.
- OMPT no longer falls back into synchronous execution mode when profiler is attached.
- OMPT now supports callbacks for
omp_target_associate_ptr
andomp_target_disassociate_ptr
. - Xteam Reduction enabled by default at all opt levels.
- Some HIP interoperability issues with tracking HIP memory allocations on Mi200 were resolved.
- Remove deprecated utility offload-arch. This was replaced with amdgpu-arch or nvptx-arch.
AOMP Release 19.0-2
These are the release notes for AOMP 19.0-2 AOMP uses AMD developer modifications to the upstream LLVM development trunk. These differences are managed in a branch called the "amd-staging". This branch is found in a mirror of upstream LLVM found at https://github.com/ROCm/llvm-project. The amd-staging branch is constantly changing as it merges the upstream development trunk with its downstream development updates. The AMD modifications are experimental and/or/while contributions under review for the upstream trunk. AOMP uses a snapshot of amd-staging at the commit ids and dates listed below. AOMP also includes builds of related ROCm components. We call AOMP a "standalone" build as it does not use or require ROCm with the exception of the kernel module (dkms) and libdrm which are often part of the Linux distribution. AOMP is isolated from any ROCm installations by installing into /usr/lib/aomp and the use of RPATH for runtime libraries.
For AOMP 19.0-2, the last trunk commit is 6012de2b4ec24826574fe9f2d74c7d2ff2b52f23on June 20, 2024. The last amd-only commit is c3a455408b118b8c22f23c7a65d2b5dbf491ab56 on June 20, 2024. These commits forms a frozen branch now called "aomp-19.0-2". See https://github.com/ROCm/llvm-project/tree/aomp-19.0-2.
The integrated ROCm components for this AOMP release were built with ROCM 6.1.2 sources.
This is the 2nd AOMP release based on LLVM 19 development.
AOMP 19.0-1 was tagged, but will not be released.
Changes since AOMP 19.0-0:
- Significant runtime features to support zero-copy for CPU-GPU unified shared memory. See subsections below.
- Merge of the LLVM upstream relocation of libomptarget into the high level "offload" directory. This establishes the long term objective of the LLVM community to unify offload support for different offloading programming models.
- The integrated ROCm components (non-compiler) were built from ROCM 6.1.2 sources.
- Significant enhancements to the gpurun utility including: support for multiple devices, heterogeneous devices, malloc control inherited from numa-ctl -m -l options, and CPU core binding to same numa node as selected GPU. These changes preserve gpurun's ability to oversubscribe (run multiple processes per GPU) by segmenting a GPUs CUs to different processes. To be fixed in 19.0-3, gpurun fails in TPX mode on MI300X.
- Changes in runtime library locations unique to CPU target triple including fixes for lib64 in Red Hat package.
- Support for fp16 and bfloat16 reductions
- Removed long deprecated utilities mygpu, mymcpu, aompcc, aompExtractRegion, clang-ocl, and cloc.sh.
Errata for AOMP 19.0-2
- gpurun fails in TPX mode for MI300X
- LIBOMPTARGET_SYNC_COPY_BACK default is still true. This is to circumvent a long-standing SDMA problem where signal values appear incorrect to SDMA engines.
- Failure in dynamic_module_load which impacts application termination that uses offloading in shared objects.
Implicit Zero-Copy behavior on MI300A
OpenMP provides a relaxed shared memory model. Map clauses provided in the source code indicate how data is used and copied to and from the GPU device for each target region. On GPUs that provide unified shared memory like the MI300A, these clauses are optional but provide portability to discreet memory GPUs. There is an OpenMP pragma called "requires unified_shared_memory" which tell the compiler and runtime that the code is NOT portable to discreet memory GPUs, and must be compiled and executed on GPUs such as the MI300A. The MI300A is one of several AMD GPUs that has a feature to disable/enable page migration between CPU and GPU called xnack. In this release of the compiler and runtime, we set the runtime behavior depending on the status of xnack and existence of the pragma "requires unified_shared_memory".
MI300A | NO requires unified_shared_memory |
requires unified_shared_memory |
---|---|---|
XNACK enabled | Implicit Zero-Copy | Zero-Copy |
XNACK disabled | Copy | Runtime warning* |
(*) The runtime warning when running an application using #pragma omp requires unified_shared_memory
in XNACK disabled mode can be turned into a runtime error by setting environment variable OMPX_STRICT_SANITY_CHECKS to true (e.g., OMPX_STRICT_SANITY_CHECKS=true ./app_exec).
Implicit Zero-Copy on MI200 and MI300X and any other discrete GPU:
- On discrete memory GPUs, for applications not using
#pragma omp requires unified_shared_memory
, turn on implicit zero-copy behavior by running applications in XNACK enabled environment and setting to true the environment variableOMPX_APU_MAPS
(e.g.HSA_XNACK=1 OMPX_APU_MAPS=1 ./app_exec
) - All other configurations, for applications not using
#pragma omp requires unified_shared_memory
, will be run in copy behavior.
MI200, MI300X, etc. | not unified_shared_memory |
unified_shared_memory |
---|---|---|
XNACK enabled and OMPX_APU_MAPS=1 | Implicit Zero-Copy | Zero-Copy |
XNACK enabled | Copy | Zero-Copy |
XNACK disabled | Copy | Runtime warning(*) |
MI300A host memory pre-faulting in Zero-Copy modes
On MI300A, host memory TLB prefaulting applies when running in in Implicit Zero-Copy and when using #pragma omp requires unified_shared_memory
- By default, for all memory copies with size larger or equal to 1MB, the OpenMP runtime makes the copied host memory visible to the target device agent before calling the copy function
- The environment variable LIBOMPTARGET_APU_PREFAULT_MEMCOPY controls this behavior and it is set to true by default. Setting it to false will disable prefaulting for all memory copy sizes (e.g., disable prefaulting with
LIBOMPTARGET_APU_PREFAULT_MEMCOPY=false ./app_exec
) - The environment variable LIBOMPTARGET_APU_PREFAULT_MEMCOPY_SIZE controls the minimum size after which prefaulting is performed. It is currently set to 1MB, meaning that all memory copies that are performed in a synchronous way will have the host memory first prefaulted. Changing the minimum size enables prefaulting at sizes different than larger or equal to 1MB (e.g., to prefault all memory copies larger than 1KB, run with LIBOMPTARGET_APU_PREFAULT_MEMCOPY_SIZE=1024 ./app_exe)
rocm-6.1.2
ROCm release v6.1.2
rocm-6.1.1
ROCm release v6.1.1
rocm-6.1.0
ROCm release v6.1.0
AOMP Release 19.0-0
These are the release notes for AOMP 19.0-0. AOMP uses AMD developer modifications to the upstream LLVM development trunk. These differences are managed in a branch called the "amd-staging". This branch is found in a mirror of upstream LLVM found at https://github.com/ROCm/llvm-project. The amd-staging branch is constantly changing as it merges the upstream development trunk with its downstream development updates. The AMD modifications are experimental and/or/while contributions under review for the upstream trunk. AOMP uses a snapshot of amd-staging at the commit ids and dates listed below. AOMP also includes builds of related ROCm components. We call AOMP a "standalone" build as it does not use or require ROCm with the exception of the kernel module (dkms) and libdrm which are often part of the Linux distribution. AOMP is isolated from any ROCm installations by installing into /usr/lib/aomp and the use of RPATH for runtime libraries.
For AOMP 19.0-0, the last trunk commit is 601e102bdb55e12a2f791e0d68fd6f81ffc21e21 on March 17, 2024. The last amd-only commit is a3c2cd57a6f99709d61d35d17527c84d1af0c780 on March 16, 2024. These commits forms a frozen branch now called "aomp-19.0-0". See https://github.com/ROCm/llvm-project/tree/aomp-19.0-0.
The integrated ROCm components for this AOMP release were built with ROCM 6.0.2 sources.
This is the 1st AOMP release based on LLVM 19 development.
These are the changes since AOMP 18.0-1:
- Now use ROCM 6.0.2 sources for non compiler components.
- Default to not use multiple SDMA engines. This could be changed for testing by setting:
LIBOMPTARGET_AMDGPU_USE_MULTIPLE_SDMA_ENGINES=true - Updates to gpurun utility
- uses preset HSA_CU_MASK and ROCR_VISIBLE_DEVICES.
- OpenMP lit testing pre-install, 100% pass rate.
- [Perf] Implement the loop directive as used in the metadirective for spec.
- Support for testing QMCpack NiO performance.
- ROCgdb support for gfx1103.
- Circumvented driver timing issue with complex signal chains. Post kernel copies occasionally started before kernel completion. The temporary fix forces the wait for kernel completion before scheduling post-kernel copies. For testing purposes, this behavior could be reverted by setting LIBOMPTARGET_SYNC_COPY_BACK=false
Fix regressions:
#616 llvm-addr2line does not work on AOMP-generated binary.
Known Failures:
- Smoke
- targetid_multi_image
- get_mapped_ptr
rocm-6.0.2
ROCm release v6.0.2
AOMP Release 18.0-1
These are the release notes for AOMP 18.0-1. AOMP uses AMD developer modifications to the upstream LLVM development trunk. These differences are managed in a branch called the "amd-stg-open". This branch is found in a mirror of upstream LLVM found at https://github.com/RadeonOpenCompute/llvm-project. The amd-stg-open branch is constantly changing as AMD merges upstream development trunk with its internal open development efforts. The AMD modifications are experimental and/or/while contributions under review for the upstream trunk. AOMP uses a snapshot of amd-stg-open at the commit ids and dates listed below. AOMP also includes builds of related ROCm components. We call AOMP a "standalone" build as it does not use or require ROCm with the exception of the kernel module (dkms) and libdrm which are often part of the Linux distribution. AOMP is isolated from any ROCm installations by installing into /usr/lib/aomp and the use of RPATH for runtime libraries.
For AOMP 18.0-1, the last trunk commit is 5f71aa9270c3d680babfbc6e766773d113c2a79a on January 9, 2023. The last amd-only commit is 71a82c97d882bfece529db41130ef7aaf9696a6e on January 9, 2023 . These commits forms a frozen branch now called "aomp-18.0-1". See https://github.com/RadeonOpenCompute/llvm-project/tree/aomp-18.0-1.
The integrated ROCm components for this AOMP release were built with ROCM 6.0 sources.
This is the 2nd AOMP release based on LLVM 18 development.
These are the changes since AOMP 18.0-0:
-
Move to ROCm 6.0 sources.
-
Python 3.8 minimum required for building of ompd.
-
Change the default optimization level to -O2 if user does not specify any -O option on the command line. Before 18.0-1, the default was -O0 if none was specified.
-
In this release we are requesting users to install the amdgpu kernel driver found in the ROCm 6.0 amdgpu-dkms package. Nothing prevents the use of an older driver, but for support with current and new problems, we ask users to upgrade to this kernel driver. Run this command to test that your amdgpu kernel driver is on the correct version "modinfo -F version amdgpu". That command should return the string "6.3.6".
-
Remove dependency on ROCm libraries for rocgdb. Reminder: A goal of aomp is to be isolated from ROCm stack with the exception of rocm-dkms which installs the correct kernel driver.
-
- The following issues were resolved in this release:
- #601 AOMP 17.0.3 crashes at compile time with conjunction in if clause
- The following issues were resolved in this release:
-
While the source build of aomp, supports ASAN (address sanitizer), the release build does not due to complexities in packaging asan. Therefore, the aomp release does not support ASAN.
-
This release has many build enhancements meant for developers that build aomp from source. These changes have little impact to user of AOMP.
- The build scripts for this release now build openmp host and device libraries using LLVM_ENABLE_RUNTIMES=openmp with the COMPONENT build_project.sh. This was formerly done in the COMPONENT build_openmp.sh. This change allows lit testing of openmp. Extra libraries found in lib-debug and lib-perf directories are still build with the COMPONENT build_openmp.sh. ROCm-device-libs are also built as an external llvm project, which was formerly done by build_libdevice.sh.
- Developers can now use ccache to speed rebuild process. The use of ccache is the default. To turn this off set AOMP_USE_CCACHE=0.
- The build scripts have added a hidden directory in the installation called .aomp_component_stats showing the files added to the installation by each build COMPONENT. The current list of build COMPONENTs for AOMP are: prereq project roct rocr openmp extras comgr rocminfo flang-legacy pgmath flang flang_runtime hipcc hipamd rocdbgapi rocgdb roctracer rocprofiler
- Changes to build_supp.sh which builds prequisite and supplemental components needed for the build and/or testing include: change to version 6.0 for of hsa-amd-aqlprofile, update wget location of silo tarball, change to version 6.0.x for rocmsmilib.
- Build optimizations to reduce the size of the installation. Previously release binaries had llvm libraries statically linked in. We are no linking in the shared library build of llvm, and this reduces overall size by ~ 50%.
- Changes to the build of libomptarget DeviceRTL to be code object version agnostic. These changes reduce the amount of files installed on to one per architecture for the opaque-linker + the fat archive for the linker wrapper as well as improve build time for the libraries.
-
The following enhancements were made to the AOMP test infrastructure:
- Support for openmp lit testing.
- New testing of UMT.
- babelstream updates to support USM.
- babelstream updates to support FORTRAN babelstream.
rocm-6.0.0
ROCm release v6.0.0