[BUG] Triton Error [CUDA]: invalid argument #3382
Comments
|
Hi @abhijitpal1247, perhaps my comment in another issue could be of help: microsoft/DeepSpeed-MII#170 (comment) |
|
Run this in a colab to reproduce #2968 and this one Here's some code to speedrun the error: Which I believe is still not fixed. !pip install diffusers==0.15.0 torch==1.13.1 transformers==4.28.1 triton==2.0.0.dev20221105
%cd /content/sample_data
!git clone https://github.com/microsoft/DeepSpeed.git
%cd /content/sample_data/DeepSpeed/requirements
!pip install -r requirements.txt
%cd /content/sample_data/DeepSpeed
!pip install .
!export PYTHONPATH="$PYTHONPATH:/content/sample_data/DeepSpeed"
import os, torch, diffusers, deepspeed
pipe = diffusers.StableDiffusionPipeline.from_pretrained(
"CompVis/stable-diffusion-v1-4",
torch_dtype=torch.float16,
revision="fp16",
replace_with_kernel_inject=True # replace the model with the kernel injector
)
model = deepspeed.init_inference(pipe.to("cuda"), dtype=torch.float16)
model("hello from here")Here's some code to speedrun the error: !pip install torch
!pip install diffusers==0.14.0 triton==2.0.0.dev20221202
!pip install transformers accelerate
%cd /content/sample_data
!git clone https://github.com/microsoft/DeepSpeed.git
%cd /content/sample_data/DeepSpeed/requirements
!pip install -r requirements.txt
%cd /content/sample_data/DeepSpeed
!pip install .
!export PYTHONPATH="$PYTHONPATH:/content/sample_data/DeepSpeed"
import torch
import deepspeed
from diffusers import StableDiffusionPipeline
print(deepspeed.__version__)
# load vanilla pipeline
ds_pipeline = StableDiffusionPipeline.from_pretrained(
"CompVis/stable-diffusion-v1-4",
torch_dtype=torch.float16
).to("cuda")
# init deepspeed inference engine
deepspeed.init_inference(
model=getattr(ds_pipeline,"model", ds_pipeline), # Transformers models
mp_size=1, # Number of GPU
dtype=torch.float16, # dtype of the weights (fp16)
replace_method="auto", # Lets DS autmatically identify the layer to replace
replace_with_kernel_inject=True, # replace the model with the kernel injector
)
print("DeepSpeed Inference Engine initialized")
image = ds_pipeline("a photo of an astronaut riding a horse on mars").images[0]
image.show() |
|
Can you test again with the latest DeepSpeed with the triton versions updated? If you are still seeing this, can you re-open this issue? |
|
I ran into a similar error. I can confirm I updated both triton (to 2.0.0) and deepspeed (to 0.10.0) but the problem persists. Here is the error message. miniconda3/envs/py39/lib/python3.9/site-packages/triton_pre_mlir/run │ |
|
@hayday100 - for now, can you use the triton version listed in requirements-sd.txt? That version specifically works when running our unit tests. |
Describe the bug
A clear and concise description of what the bug is.
Facing this error
RuntimeError: Triton Error [CUDA]: invalid argument, while using deepspeed inference for stable-diffusion model.To Reproduce
Steps to reproduce the behavior:
deepspeed==0.9.1+fef5aa6e
diffusers==0.13.1
transformers==4.27.3
triton==2.0.0.dev20221202
accelerate==0.16.0
xformers==0.0.16
huggingface_hub==0.12.0
torch==1.13.1
Expected behavior
Execute without any issue.
ds_report output
System info (please complete the following information):
GPU count and types
one Tesla T4
(if applicable) Hugging Face Transformers/Accelerate/etc. versions
Python version
Python 3.10.9
Docker context
Using Conda to maintain environments
Additional context
Error log:
The text was updated successfully, but these errors were encountered: