Support for GPUDirectStorage

[gdkrmr]

We are using compressed disk array storage (Zarr.jl [1]). In Python there is kvikIO [2], which allows loading Zarr arrays directly into GPU memory. There are two steps, one is loading a chunk of the Zarr array into memory and the second one is to decompress it.

What would it take to implement this in Julia?

@Sonicious @meggart @felixcremer

[1] https://github.com/JuliaIO/Zarr.jl
[2] https://docs.rapids.ai/api/kvikio/stable/

[maleadt]

What would it take to implement this in Julia?

I don't know, it was never clear to me what exactly consists the GPUDirectStorage APIs or libraries. I did have a look at cuFile, #1235, but it was completely unusable (requiring an extremely peculiar software setup down to specific OS versions and filesystem mount options) so I didn't continue that effort.

[gdkrmr]

Microsoft included this into DIrectX recently, so I thought this would be easier. Thanks for the info!

[gitsd]

you can do integration and testing without requiring MOFED.

[jakirkham]

FWIW KvikIO also has a C++ layer. It also has fallbacks if run on a system that doesn't support GDS. So maybe that would be an easier entry point.

cc @madsbk

[madsbk]

Yes, please let's know if you need any help with KvikIO.
The C++ API is header-only and only requires the CUDA Toolkit installed.
I guess the first step would be to implement Julia bindings?

[maleadt]

@madsbk Do you have a pointer to the API docs?
Julia only really supports C APIs though, C++ FFI isn't great.

[madsbk]

The API docs and a C++ example.
What about using something like https://github.com/JuliaInterop/CxxWrap.jl?

[maleadt]

The API docs and a C++ example.

I didn't mean the Python/RAPIDS API, but the underlying CUDA one. So I guess this is built on top of cuFile?

[madsbk]

Yes, KvikIO is built on top of cuFile and will work even on systems that doesn't support or have cuFile installed.

cuFile doc: https://docs.nvidia.com/gpudirect-storage/api-reference-guide/index.html

[maleadt]

cuFile supposedly also doesn't need a compatible set-up and should just work.

In any case, there's already a proof-of-concept in #1235, so that should be easy for anybody to pick up. I probably won't; this is very low on my TODO list, in part because I never managed to set-up GDS on my workstation (Arch Linux).

Support for KvikIO is much harder (since it requires C++), and doesn't need to happen in CUDA.jl, but can just live in an external package instead.

[Moelf]

this could be very useful, for our use case, we need something that can load bytes onto GPU directly given (file handler, offset, N bytes), which should be the most basic API?