Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize RAM to VRAM transfer #6312

Merged
merged 11 commits into from
May 24, 2024
Merged

Optimize RAM to VRAM transfer #6312

merged 11 commits into from
May 24, 2024

Commits on May 5, 2024

  1. avoid copying model back from cuda to cpu

    Lincoln Stein committed May 5, 2024
    Configuration menu
    Copy the full SHA
    c32be43 View commit details
    Browse the repository at this point in the history

Commits on May 7, 2024

  1. handle models that don't have state dicts

    Lincoln Stein committed May 7, 2024
    Configuration menu
    Copy the full SHA
    f8d5f13 View commit details
    Browse the repository at this point in the history

Commits on May 13, 2024

  1. add assertions that models need a device() method

    @lstein
    lstein committed May 13, 2024
    Configuration menu
    Copy the full SHA
    33590ef View commit details
    Browse the repository at this point in the history

  2. do not rely on torch.nn.Module having the device() method

    @lstein
    lstein committed May 13, 2024
    Configuration menu
    Copy the full SHA
    5c46343 View commit details
    Browse the repository at this point in the history

Commits on May 18, 2024

  1. Merge branch 'main' into lstein/feat/cpu_to_vram_optimization

    @lstein
    lstein committed May 18, 2024
    Configuration menu
    Copy the full SHA
    7f107af View commit details
    Browse the repository at this point in the history

  2. apply all patches after model is on the execution device

    Lincoln Stein committed May 18, 2024
    Configuration menu
    Copy the full SHA
    90e5dfb View commit details
    Browse the repository at this point in the history

  3. fix model patching in latents too

    Lincoln Stein committed May 18, 2024
    Configuration menu
    Copy the full SHA
    7000c0b View commit details
    Browse the repository at this point in the history

  4. log patched tokenizer

    Lincoln Stein committed May 18, 2024
    Configuration menu
    Copy the full SHA
    0a0c1d1 View commit details
    Browse the repository at this point in the history

Commits on May 19, 2024

  1. closes #6375

    Lincoln Stein committed May 19, 2024
    Configuration menu
    Copy the full SHA
    c775b59 View commit details
    Browse the repository at this point in the history

Commits on May 24, 2024

  1. Merge branch 'main' into lstein/feat/cpu_to_vram_optimization

    @lstein
    lstein committed May 24, 2024
    Configuration menu
    Copy the full SHA
    9364ffd View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into lstein/feat/cpu_to_vram_optimization

    @lstein
    lstein committed May 24, 2024
    Configuration menu
    Copy the full SHA
    71cdafa View commit details
    Browse the repository at this point in the history