CUDA version

[armandal88]

Hello,
Thanks for sharing this promissing work.
I have an issue for installing Flash Attention following your Installation process.
I believe the problem comes from the CUDA version that is 11.8 on the VM I use on GCP.
Can you share the CUDA version you are using in your project ?
Thanks a lot,
Best

Alex

[AFeng-x]

Hi, my CUDA version is 12.4~

[armandal88]

thanks @AFeng-x for your feedback,
I still have troubles for installing the environment, especially flash-attn. I use a nvidia A100 40Go.
It would be helpful if you shared a few additional information regarding your setup:

• your GPU model
• the cuda version used to compile pytorch
• the cuda toolkit version
• the flash-attn version (is the latest version ok ?)

thanks a lot
best
Alex

[armandal88]

Hi, I got a config that works. It's probably not the most straightforward manner to make it work, but after several hours of tentatives to install the PAM env, it was ok on my machine (VM on GCP, with 1 nvidia A100 40Go).

I installed:

• the cuda toolkit 12.4
• after installation of PAM & SAM2, I installed another version of pytorch 2.4.1 compiled with cuda 12.4
• and finally installed flash-attn 2.7.2.post1
  This seems to satisfy all libs compatibilities, and the models works well.

# Install cuda toolkit 12.4
wget https://developer.download.nvidia.com/compute/cuda/12.4.0/local_installers/cuda_12.4.0_550.54.14_linux.run
sudo sh cuda_12.4.0_550.54.14_linux.run

# Follow PAM github instructions to install PAM & SAM2
# ...

# Update version of torch
pip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 --index-url https://download.pytorch.org/whl/test/cu124
# then check if it worked
python -c "import torch; print('PyTorch:', torch.__version__, '| CUDA (compil):', torch.version.cuda, '| CUDA dispo:', torch.cuda.is_available())"

# and install one version of flash-attn, that is compatible with torch, cuda, etc.
pip install flash-attn==2.7.2.post1 --no-build-isolation