-
Notifications
You must be signed in to change notification settings - Fork 329
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[inference] RuntimeError: CUBLAS_STATUS_NOT_SUPPORTED on cards compute capability greater than 80 #87
Comments
env information: NVIDIA-SMI 455.32.00 |
@letiantony After googling it. It seems a problem with this CUDA version, could you try different CUDA versions? |
@Taka152 thanks for the reply, it worked with cuda 10.2,but still hope it support cuda 11.1 |
@letiantony Thanks for your feedback, we have reproduced this error on CUDA 11. It seems to be a CUBLAS API deprecated in the newest CUDA. We'll find a way to solve this. BTW, when you try CUDA 10.2, did you downgrade the driver version or it was still 455.32.00? |
@Taka152 NO, the Driver Version is 440.33.01 |
I see, I guess it is the driver version, not CUDA that is incompatible with LightSeq. Because CUDA is statically linked in LightSeq lib. @letiantony Would you mind trying the 455 driver version with CUDA 10.2? That would help a lot to confirm my guess. |
BTW, our test on driver 450 is OK |
@Taka152 Sorry, because of the limits of authority,I can't upgrade or downgrade the version. I just use another GPU to try cuda 10.2. |
That's ok, we'll find it |
@Taka152 Hi Ying Xiong. Thank you for making this awesome library available to public. Actually I encountered the same issue as mentioned in this post. Now I am trying to downgrade the graphics driver as well as CUDA to see if it works. For your information, right now I am using NVIDIA RTX 3090 graphics card, and the versions of the graphics driver and CUDA are 460.56 and 11.2 respectively. |
We found on driver greater than 450, some cublas API has changed, so currently lightseq inference can't run driver greater than 450 |
Ying Xiong thanks for your prompt reply. I just checked the supported product list of the 450 driver, and it seems that this driver does not support NVIDIA RTX 3090 card. I was thinking if there would be another lightseq release to resolve the CUBLAS issue on more recent drivers in the near future. Thank you! |
Sure, we'll try to fix it. |
Have you fix this issue? |
Here is the solution.#251 (comment). This problem is not caused by driver or cuda version. It is the card that not support pre-built lightseq. |
How to fix this? |
I had the same issue. ====================START warmup==================== NVIDIA-SMI 460.106.00 |
2021-06-30 20:37:00.954006: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
initializing gpt tokenizer...
lightseq tokenizer pad token id: 0
huggingface tokenizer pad token id: 0
creating lightseq model...
finish initializing emb_wei from host to device
finish initializing enc_wei from host to device
finish initializing all weight from host to device
gpt2 buf_bytesize: 37281664
creating huggingface model...
====================START warmup====================
=========lightseq=========
lightseq generating...
Traceback (most recent call last):
File "ls_gpt.py", line 118, in
main()
File "ls_gpt.py", line 94, in main
warmup(ls_tokenizer, hf_tokenizer, ls_model, hf_model, sentences)
File "ls_gpt.py", line 56, in warmup
ls_generate(ls_model, ls_tokenizer, ls_inputs)
File "ls_gpt.py", line 33, in ls_generate
ls_res_ids, ls_time = ls_gpt2(model, inputs)
File "ls_gpt.py", line 12, in ls_gpt2
generated_ids = model.sample(inputs)
RuntimeError: [CUDA][ERROR] /tmp/build-via-sdist-ifaem8qq/lightseq-2.0.3/lightseq/inference/model/gpt_encoder.cc.cu(397): CUBLAS_STATUS_NOT_SUPPORTED
The text was updated successfully, but these errors were encountered: