Stable Diffusion v1.5 setup on a Radeon Pro VII (AMD GPU)

These are personal notes cataloging steps needed to do to get Stable Diffusion v1.5 running locally on Radeon Pro VII (AMD GPU). The initial system configuration was Ubuntu v18.04, Linux kernel version Linux 5.4.0-131-generic, no ROCm installed, and no previous version of Stable Diffusion or corresponding Python environment. These notes cover roughly 4 steps:

• Upgrade Ubuntu (to ROCm-supported version)
• Upgrade Linux kernel (for compatibility with the AMD GPU)
• Install ROCm
• Download Stable Diffusion and set up Python environment

After these are a few sections covering miscellaneous tips, comparisons of images generated using different model weights, and miscellaneous issues encountered during the setup:

• Stable Diffusion tips
• Image comparisons
  • EMA-only vs. EMA & non-EMA weights
  • Stable Diffusion v1.4 vs. v1.5
• Issues encountered

Contents of this repository are as follows:

• README.md: guide of steps needed to do to set up and run Stable Diffusion locally on Radeon Pro VII (an AMD GPU).
• requirements.txt: Python environment that worked for running Stable Diffusion on Radeon Pro VII.

Upgrade Ubuntu

The starting point was Ubuntu v18.04, so needed to first upgrade Ubuntu to a ROCm-supported version, prior to installing ROCm. Running these commands completed a full system update:

$ sudo apt update
$ sudo apt upgrade
$ sudo apt dist-upgrade

and removed unneeded packages:

$ sudo apt autoremove

which allowed to upgrade from Ubuntu v18.04 to a ROCm-supported version (v20.04 or v22.04), with the following:

$ sudo do-release-upgrade -f DistUpgradeViewGtk3

This downloads an upgrade release tool; follow prompts from the GUI installer.

Upgrade Linux kernel

Different AMD GPUs may require a different Linux kernel than what is installed on your system. E.g., if ROCm is installed and you run rocminfo with an incompatible Linux kernel version, you can get an error such as:

HSA Error:  Incompatible kernel and userspace, AMD Radeon (TM) Pro VII disabled. Upgrade amdgpu.

This suggests that the Linux kernel should be upgraded. Linux kernel version Linux 5.4.0-131-generic, e.g., is incompatible with Radeon Pro VII GPU, and needs to be upgraded to something that is compatible. Download this Bash script for Linux kernel installation:

$ wget https://raw.githubusercontent.com/pimlie/ubuntu-mainline-kernel.sh/master/ubuntu-mainline-kernel.sh

Ensure to add executable permissions for the script:

$ chmod go+x ubuntu-mainline-kernel.sh

Also move the Linux kernel bash script to /usr/local/bin/:

$ sudo mv ubuntu-mainline-kernel.sh /usr/local/bin/

The Linux kernel version is updated by running the script:

$ sudo ubuntu-mainline-kernel.sh –i

Reboot, then verify the Linux kernel version:

$ uname -r
6.0.3-060003-generic

After this, the installed Linux kernel is compatible with Radeon Pro VII.

Install ROCm

Installation instructions for Ubuntu are here. To check installation, run rocminfo and you should see output e.g.:

==========               
HSA Agents               
==========               
*******                  
Agent 1                  
*******                  
  Name:                    AMD Ryzen Threadripper 3960X 24-Core Processor
  Uuid:                    CPU-XX                             
  Marketing Name:          AMD Ryzen Threadripper 3960X 24-Core Processor
  Vendor Name:             CPU

⋮

*******                  
Agent 2                  
*******                  
  Name:                    gfx906
  Uuid:                    GPU-a784386172dc76bd  
  Marketing Name:          AMD Radeon (TM) Pro VII            
  Vendor Name:             AMD

As another check, I ran rocminfo, which shows some information about the GPU:

======================= ROCm System Management Interface =======================
================================= Concise Info =================================
GPU  Temp   AvgPwr  SCLK    MCLK    Fan    Perf  PwrCap  VRAM%  GPU%  
0    49.0c  25.0W   860Mhz  350Mhz  9.41%  auto  190.0W    2%   0%    
================================================================================
============================= End of ROCm SMI Log ==============================

Download Stable Diffusion and set up Python environment

With a compatible Ubuntu, Linux kernel, and ROCm installed, Stable Diffusion is ready for set up. Conda (via Miniconda) is convenient for setting up the Python environment needed to run Stable Diffusion locally. Miniconda can be installed with the following commands, selecting yes when prompted about whether to initialize Conda:

$ wget https://repo.anaconda.com/miniconda/Miniconda3-4.7.12-Linux-x86_64.sh
$ sh Miniconda3-4.7.12-Linux-x86_64.sh

Clicking Enter gets through to the end of the installation. There are more recent versions of Miniconda, but with those an error can occur during downloads when trying to set up a new Conda environment:

CondaHTTPError: HTTP 000 CONNECTION FAILED

but this was avoided with Miniconda v4.7.12.

From here cloned the Stable Diffusion repository:

$ git clone https://github.com/runwayml/stable-diffusion.git

then navigated into the Stable Diffusion directory to set up a Python environment suitable for running Stable Diffusion on the Radeon Pro VII. The required environment is different than the one specified in environment.yaml included in the Stable Diffusion repository, which assumes the user would run on an Nvidia GPU. Provided in this repository is a requirements.txt that worked for running locally on Radeon Pro VII (AMD GPU). The main difference from the official Stable Diffusion environment is that this specifies PyTorch for ROCm rather than CUDA. From within the Stable Diffusion repository, these commands will get the environment set up (where requirements.txt is the one included in this repository):

$ cd stable-diffusion
$ conda create --name ldm python=3.8.5
$ conda activate ldm
$ pip install -r requirements.txt

If the ROCm and PyTorch installations all went OK, PyTorch should be able to see the GPU, which can be checked from within Python:

>>> import torch
>>> torch.cuda.is_available()
True

If the installation has gone OK, torch.cuda.is_available() should return True as above; but if this is still returning False it is possible the user needs to be added to render and video groups, which can be done with the following:

$ sudo usermod -a -G render,video $LOGNAME

Logging out then back in and checking groups should confirm the user is part of the render and video groups.

Next download Stable Diffusion model weights from Hugging Face:

• v1-5-pruned-emaonly.ckpt

Accessing the model weights requires to make an account and accept an agreement.

You can observe at Hugging Face there are two choices for the weights:

• v1-5-pruned-emaonly.ckpt
• v1-5-pruned.ckpt

The 'emaonly' weights take less memory and are suitable for inference ('EMA' refers to 'exponential moving average' technique, a performance optimization for stochastic gradient descent; you can read more about it in the original 'Adam' paper). If interested in fine-tuning the model, v1-5-pruned.ckpt is suitable, which contains both EMA & non-EMA weights. Out of curiosity, I compared images generated with both sets of weights, but couldn't discern a difference in quality; details could differ, but prompts seem to be captured about equally well using either of the weights. A representative comparison is shown further below.

Stable Diffusion tips

• There are different possibiliites for image generation, described in detail at the Stable Diffusion official repository:
  • Text-to-image: generates an image from an input text prompt.
  • Image modification: generates an image from a user-provided image, a text prompt, and a parameter controlling the amount of noise added to the user-provided image.
  • Inpainting: generates an image from a user-provided image, an image mask, and a text prompt.
    • This requires to download the inpainting weights: sd-v1-5-inpainting.ckpt.
• Image modification seems to work best with images of size 512 x 512 pixels.

Image comparisons

These are some comparisons between images generated using the text-to-image capability, with different choices for Stable Diffusion model weights. The comparisons are EMA-only vs. EMA & non-EMA weights (both v1.5), and v1.4 vs. v1.5 weights (both EMA-only); these are 'anecdotal,' hand-selected comparisons, so none of this should be taken as a rigorous statement about the result of using different weights. Still, some of the comparisons are interesting.

EMA-only vs. EMA & non-EMA weights

The images below were generated using the following prompt, with EMA-only, as well EMA & non-EMA weights (left and right, respectively); both sets of weights are v1.5. The image below is just a single case, but representative of the kind of similarities and differences I've seen. Images were of similar quality, whether using the EMA-only or EMA & non-EMA weights.

$ python scripts/txt2img.py --prompt "80s style floating-head family portrait of cute scottish fold cats, in starcraft 2 space, Canon EOS R3, 80mm" --plms

v1.5, EMA-only (v1-5-pruned-emaonly.ckpt)	v1.5, EMA & non-EMA (v1-5-pruned.ckpt)

Stable Diffusion v1.4 vs. v1.5

The following three prompts (corresponding to the following three rows of images, respectively) were used to generate the images below, using v1.4 or v1.5 weights (left and right columns, respectively); both sets of weights are EMA-only. In testing, the v1.5 weights tended to capture the input prompts more closely than v1.4 weights, which is probably expected. On average, the images generated using v1.4 weights seemed to have stronger artifacts, as compared with v1.5.

$ python scripts/txt2img.py --prompt "Painting of a person painting a person painting a person" --plms
$ python scripts/txt2img.py --prompt "High quality photo of Darth Vader at the Golden Gate Bridge" --plms
$ python scripts/txt2img.py --prompt "80s style floating-head family portrait of cute scottish fold cats, fantasy scifi space background, vintage 80s camera, 35mm" --plms

v1.4, EMA-only (sd-v1-4.ckpt)	v1.5, EMA-only (v1-5-pruned-emaonly.ckpt)

Issues encountered

This section covers miscellaneous issues encountered during the quest to run Stable Diffusion locally:

• Warning appeared when running Stable Diffusion:

  IOpen(HIP): Warning [SQLiteBase] Missing system database file: gfx906_60.kdb Performance may degrade. 
  Please follow instructions to install: https://github.com/ROCmSoftwarePlatform/MIOpen#installing-miopen-kernels-package

  Attempting to download the suggested file following the linked instructions, apt-get install miopenkernels-gfx906-60, doesn't appear to be available. This did not prevent from running Stable Diffusion in the described setup.

• Downloading Miniconda was taking a long time on a slow and unreliable wireless connection; the attempted remedy was to continue downloads even when the computer went into 'Blank Screen' mode or similar; these intructions helped.

• In a futile attempt to improve wireless connection, firmware for wireless adapter was updated. Updated firmware for Intel wireless adapter is available here, dropped it into /lib/firmware, and restarted wifi with:

  $ sudo modprobe -r iwlwifi
  $ sudo modprobe iwlwifi

• Screen flickering issues; these instructions helped.

References

• Radeon Pro VII
• ROCm supported GPUs
• How to upgrade Ubuntu to v20.04
• How to upgrade from Ubuntu v18.04 to v20.04
• Bash script for Linux kernel installation
• Install or upgrade to latest Linux kernel on Ubuntu 20.04
• How to install ROCm
• Miniconda
• Ubuntu 20.04: CondaHTTPError
• PyTorch
• How to add user to video group
• Stable Diffusion GitHub repository
• Stable Diffusion v1.5
• Stable Diffusion inpainting
• Stable Diffusion v1.4
• How to prevent wifi sleep after suspend
• Slow wireless on Ubuntu v20.04
• Intel wireless adapters firmware
• Ubuntu screen flickering