Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explore removing std::vector mData; completely from Tensor in favour of always storing data in hostVisible buffer memory (TBC) #144

Closed
axsaucedo opened this issue Feb 10, 2021 · 0 comments · Fixed by #177
Labels
enhancement New feature or request redesign/refactor triage Issue still needs to be discussed, researched and scoped

Comments

@axsaucedo
Copy link
Member

As followup of #136, opening issue to focus on researching phase 4 specifically, which was presented as following:

4) Remove std::vector mData; completely from Tensor in favour of always storing data in hostVisible buffer memory (TBC)

After doing some resarch I was able to confirm:

  • All GPU devices will always have both HostVisible and DeviceVisible memory available for use
  • Performance of GPU hostVisible memory access from CPU is relatively efficeint (even if not as efficient as RAM), this AMD document mentions "up to 64 GB/s peak theoretical transport data bandwidth from CPU to GPU per card." - still needs to be further confirmed
  • It is possible to leverage std-compatible containers such as std:span-like containers (or msft's gsl/span) to have a container with non-owning reference to the host-visible memory

Due to these points, it may be possible to completely remove the std::vector mData attribut completely whilst still keeping the best of both worlds - namely 1) access to the memory that was added to the GPU, and 2) optimum memory usage (mainly by not storing an extra copy of the data).

This point would still need to be explored further as tehre are some nuances in regards to how this would be implemented in a memory-safe way, as well as ensuring that the python interface can be exposed efficiently, as ideally it can be built in a way in which it can be passed to and from python with minimal amount of copies (as it's not clear what the support for non-owning containers like span is)

@axsaucedo axsaucedo added this to To do in 0.7.0 via automation Feb 10, 2021
@axsaucedo axsaucedo added this to To do in 0.6.1 via automation Feb 10, 2021
@axsaucedo axsaucedo added enhancement New feature or request redesign/refactor triage Issue still needs to be discussed, researched and scoped labels Feb 10, 2021
0.7.0 automation moved this from To do to Done Mar 7, 2021
0.6.1 automation moved this from To do to Done Mar 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request redesign/refactor triage Issue still needs to be discussed, researched and scoped
Projects
0.6.1
Done
0.7.0
Done
1 participant