Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add functions to interact with external memory #152

Merged
merged 7 commits into from
May 22, 2023

Conversation

ViliamVadocz
Copy link
Contributor

  • Add unsafe functions to crate::driver::result in a new external_memory module which wrap raw bindings for interacting with external memory.
  • Add safe(?) wrapper in the form of two new types ExternalMemory and MappedBuffer which abstract away interactions with external memory and to manage cleanup on drop.

I did not add any tests because I don't yet know how to get a valid file descriptor using only CUDA. At least I confirmed that this interface works with a file descriptor exported from Vulkan (via vulkano crate). The code exists in my repo https://github.com/ViliamVadocz/nvidia-video-codec-sdk on the cudarc-external-memory branch (very experimental and unorganized).

Copy link
Owner

@coreylowman coreylowman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great to me. I'm inclined to merge this. Left a few questions.

Also this seems kind of like a mmap functionality in cuda? Am I interpreting this right? In llama-dfdx I've been experimenting with using mmap on CPU to not actually allocate on heap. Could this functionality be used to do something similar with cuda?

src/driver/safe/external_memory.rs Show resolved Hide resolved
src/driver/result.rs Outdated Show resolved Hide resolved
src/driver/safe/external_memory.rs Show resolved Hide resolved
src/driver/safe/external_memory.rs Show resolved Hide resolved
Copy link
Owner

@coreylowman coreylowman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition, thanks for the contribution 👍

@coreylowman coreylowman merged commit 15141ed into coreylowman:main May 22, 2023
3 checks passed
@ViliamVadocz ViliamVadocz deleted the external_memory branch May 22, 2023 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants