Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correct signatures for torch allocator plug in #1407

Merged
merged 5 commits into from
Dec 14, 2023

Commits on Dec 12, 2023

  1. Correct signatures for torch allocator plug in

    Since pytorch/pytorch#91398, the signature of
    the pluggable allocate and deallocate functions must accept the device
    id. The current version only accepts a device id for allocate, which
    means that when using a stream ordered allocator with devices other
    than device zero, we pass an invalid stream into the deallocation
    function. To fix this, adapt the signature to match the one pytorch
    expects.
    
    Now, since we have the device available during allocation and
    deallocation, we would like to use that device to obtain the
    appropriate memory resource.
    
    Unfortunately, since RMM's cuda_device_id does not have a nullary
    constructor, we can't use it in Cython without some hacky workarounds.
    
    However, since we don't actually need to build a Python module, but
    rather just a single shared library that offers two extern "C"
    functions, let's just write our allocator hooks directly in C++.
    
    - Closes rapidsai#1405
    wence- committed Dec 12, 2023
    Configuration menu
    Copy the full SHA
    dcca9bb View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2023

  1. Configuration menu
    Copy the full SHA
    2f6e6f2 View commit details
    Browse the repository at this point in the history
  2. Ensure current active device is the requested one in torch hooks

    PyTorch does not guarantee that plug-in allocator functions are called
    with the active device matching the device on which the
    allocation/deallocation is requested. Hence, we must explicitly handle
    that here by selecting the appropriate device.
    wence- committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    93a0427 View commit details
    Browse the repository at this point in the history
  3. Fix thinko in comment

    wence- committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    eb8f107 View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2023

  1. Fix docstring

    wence- committed Dec 14, 2023
    Configuration menu
    Copy the full SHA
    1c426b3 View commit details
    Browse the repository at this point in the history