Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ROCm AMD Unified Memory Architecture (UMA) handling #4449

Merged
merged 7 commits into from Dec 21, 2023

Commits on Dec 13, 2023

  1. AMD ROCm: handle UMA memory VRAM expansions

    This resolves ggerganov#2797 by allowing ROCm AMD GPU users with a UMA to
    dynamically expand the VRAM allocated to the GPU.
    
    Without this, AMD ROCm users with shared CPU/GPU memory usually are
    stuck with the BIOS-set (or fixed) framebuffer VRAM, making it
    impossible to load more than 1-2 layers.
    
    Note that the model is duplicated in RAM because it's loaded once for
    the CPU and then copied into a second set of allocations that are
    managed by the HIP UMA system. We can fix this later.
    ekg committed Dec 13, 2023
    Configuration menu
    Copy the full SHA
    d59c0b3 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e754a83 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    405fc54 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    6caf33c View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2023

  1. Configuration menu
    Copy the full SHA
    7ee8df3 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    1e946c5 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    87cfad3 View commit details
    Browse the repository at this point in the history