Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use HIP stream-ordered allocator #3980

Merged
merged 6 commits into from
Jun 9, 2024

Conversation

BenWibking
Copy link
Contributor

@BenWibking BenWibking commented Jun 9, 2024

Summary

When available, this uses the HIP stream-ordered memory allocator hipMallocAsync/hipFreeAsync to make device arena allocations. This should improve performance (performance testing TBD).

AMReX tests pass on MI210 with ROCm 6.0.

Closes #3979.

Additional background

ROCm 5.2.0 added these APIs: https://rocm.docs.amd.com/en/latest/about/changelog.html#id665
HIP documentation: https://rocm.docs.amd.com/projects/HIP/en/latest/doxygen/html/group___stream_o.html

The CUDA stream-ordered allocator is already used by AMReX (with CUDA >= 11.2).

Checklist

The proposed changes:

  • fix a bug or incorrect behavior in AMReX
  • add new capabilities to AMReX
  • changes answers in the test suite to more than roundoff level
  • are likely to significantly affect the results of downstream AMReX users
  • include documentation in the code and/or rst files, if appropriate

@BenWibking
Copy link
Contributor Author

BenWibking commented Jun 9, 2024

Hmm, these tests fail here and also fail on the latest development commit:

92% tests passed, 3 tests failed out of 38

Total Test time (real) = 657.96 sec

The following tests FAILED:
         24 - Particles_ParallelContext_3d (Failed)
         30 - Particles_Redistribute_3d (Failed)
         31 - Particles_RedistributeSOA_3d (Failed)
Errors while running CTest

See #3981.

@BenWibking BenWibking changed the title [WIP] use HIP stream-ordered allocator use HIP stream-ordered allocator Jun 9, 2024
@BenWibking BenWibking marked this pull request as ready for review June 9, 2024 17:52
@WeiqunZhang WeiqunZhang merged commit c40b1ba into AMReX-Codes:development Jun 9, 2024
68 of 69 checks passed
@BenWibking BenWibking deleted the hip-malloc-async branch June 9, 2024 21:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

use HIP stream-ordered allocator
2 participants