C++ bindings to the Batch API

[madsbk]

An example use of the Batch API (requires CUDA v12.1+):

    // Here we use the batch API to read "/tmp/test-file" into `b_dev` by
    // submitting 4 batch operations.
    constexpr int num_ops_in_batch = 4;
    constexpr int batchsize        = SIZE / num_ops_in_batch;
    kvikio::DriverProperties props;
    check(num_ops_in_batch < props.get_max_batch_io_size());

    // We open the file as usual.
    kvikio::FileHandle f("/tmp/test-file", "r");

    // Then we create a batch
    auto batch = kvikio::BatchHandle(num_ops_in_batch);

    // And submit 4 operations each with its own offset
    std::vector<kvikio::BatchOp> ops;
    for (int i = 0; i < num_ops_in_batch; ++i) {
      ops.push_back(kvikio::BatchOp{.file_handle   = f,
                                    .devPtr_base   = b_dev,
                                    .file_offset   = i * batchsize,
                                    .devPtr_offset = i * batchsize,
                                    .size          = batchsize,
                                    .opcode        = CUFILE_READ});
    }
    batch.submit(ops);

    // Finally, we wait on all 4 operations to be finished and check the result
    auto statuses = batch.status(num_ops_in_batch, num_ops_in_batch);
    check(statuses.size() == num_ops_in_batch);
    size_t total_read = 0;
    for (auto status : statuses) {
      check(status.status == CUFILE_COMPLETE);
      check(status.ret == batchsize);
      total_read += status.ret;
    }
    check(cudaMemcpy(b, b_dev, SIZE, cudaMemcpyDeviceToHost) == cudaSuccess);
    for (int i = 0; i < NELEM; ++i) {
      check(a[i] == b[i]);
    }
    cout << "Batch read using 4 operations: " << total_read << endl;

[KiranModukuri]

An example use of the Batch API (requires CUDA v12.1+):

  
    // Here we use the batch API to read "/tmp/test-file" into `b_dev` by
    // submitting 4 batch operations.
    constexpr int num_of_batches = 4;

may be it will be clear to say this is "num_ops_in_batch".

[madsbk]

An example use of the Batch API (requires CUDA v12.1+):

  
    // Here we use the batch API to read "/tmp/test-file" into `b_dev` by
    // submitting 4 batch operations.
    constexpr int num_of_batches = 4;

may be it will be clear to say this is "num_ops_in_batch".

Good point, fixed

[wence-]

A few places where we can do error checking I think.

Also, I don't like the name-mangled lookup, is there not a better way?!

[wence-]

The right thing to do is to attempt to compile and link a call to this function, surely this is something that cmake supports? To ensure that both the header file and the library actually offer the function.

This is fine, I guess, but there must be a canonical way.

[madsbk]

I tried using check_symbol_exists() with no luck. Got irritated and just rolled my own :)

This doesn't work for some reason:

  include(CheckSymbolExists)
  check_symbol_exists(
    cuFileBatchIOSetUp
    "${cuFile_INCLUDE_DIRS}/cufile.h"
    cuFile_BATCH_API_FOUND
  )

[robertmaynard]

check_symbol_exists will try to compile and link a small executable that verifies that symbol exists. For this to work properly for libcufile you would also need to tell check_symbol_exists all the extra include directories, and link requirements.

The examples in the docs are easy since they are testing for values found in headers provided by the compiler/runtime and therefore no extra information is required.

Using CMake's try_compile would be easier in comparison but would still require setting up a code snippet and specifying the include directory for the libcufile and cuda rutime headers

[wence-]

This will produce a generic error if io_batch_params.size() is larger than the requested max batch size. Does it make more sense to check if operations.size() > _max_num_events and provide a useful message?

[madsbk]

Good point, done.

[wence-]

Can we check the CTK version somehow? This is unstable wrt compilers (since name mangling is not standardised).

[madsbk]

No, not AFAIK :/
I think this will suffice for now. When CUDA v12.2 is released, I think we should make the Batch API require v12.2 and then use cuFileReadAsync to determine the availability of both the batch and async API.

[wence-]

These are just copied from cufile.h I guess. Do you endeavour to be ABI compatible with the cufile definitions or is this just "anything, as long as it has the appopriate fields/names"?

[madsbk]

It is "anything, as long as it has the appopriate fields/names". Added a comment.

[madsbk]

Thanks @wence-, I think I have addressed all of your issues?

[wence-]

Thanks @madsbk

[madsbk]

Awesome, thanks for the review @wence-. Just in time for the code freeze :)

[madsbk]

/merge