Semi-Transparent Buffers #1016
Open
Semi-Transparent Buffers #1016
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ransparent-buffers
…ransparent-buffers
More fixes Fixing the fixes Part 2 Part 3 Part 4 Part 5
…ransparent-buffers
…ransparent-buffers
…ransparent-buffers
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## master #1016 +/- ##
==========================================
- Coverage 13.80% 13.68% -0.13%
==========================================
Files 268 269 +1
Lines 14998 15110 +112
==========================================
- Hits 2071 2068 -3
- Misses 12927 13042 +115 ☔ View full report in Codecov by Sentry. |
16 tasks
…ransparent-buffers
…ransparent-buffers
sebwolf-de
approved these changes
May 15, 2024
| @@ -171,8 +172,11 @@ ProxyOutput runProxy(ProxyConfig config) { | |||
| initDataStructuresOnDevice(enableDynamicRupture); | |||
| #endif // ACL_DEVICE | |||
|
|
|||
| if (config.verbose) | |||
| runtime = new seissol::parallel::runtime::StreamRuntime(); | |||
There was a problem hiding this comment.
I agree—but it's probably better IMO to do it in a general proxy refactor? We have multiple of such raw-pointer new statements at the moment; all of them could be eventually deleted
| @@ -262,6 +270,8 @@ ProxyOutput runProxy(ProxyConfig config) { | |||
| delete m_dynRupTree; | |||
| delete m_allocator; | |||
|
|
|||
| delete runtime; | |||
davschneller
force-pushed
the
davschneller/semi-transparent-buffers
branch
from
fe434fc
to
2498c15
Compare
davschneller
force-pushed
the
davschneller/semi-transparent-buffers
branch
from
2498c15
to
3c7adc6
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Or: No More Mandatory USM.
That's right, we finally switch to pure device buffers; in a first implementation (that is, we still keep many indexing arrays etc...).
Currently, we allocate a buffer twice, once on the host, and once on the device—any initialization is done on the host first; then the buffer is copied over. For output, we transfer the data in the reverse direction. We do so to allow explicit host-device memory, while keeping open the possibility to use CPU and GPU at the same time (especially on systems like Grace Hopper, or the MI300A) for different clusters. At least, if it doesn't turn out that using two instances of SeisSol is better in that case anyways.
Consequently, the performance should improve when using IO on simulations which do not have a shared memory space like Grace Hopper and MI300A. As a bonus, we can now also support AMD GPUs without the xnack functionality (or
HSA_XNACK=0). That particularly includes in consumer cards, especially the RDNA-based series (since those didn't support shared memory yet, to my knowledge). But it also allows e.g. a more modern ROCm to be used on LUMI.Next, we also remove the synchronization as far as possible and refactor the graph-handling code a bit.
Furthermore, we allow the execution of CPU and GPU at the same time—which is currently only correctly functional with unified memory enabled.