Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question]: 运行 oneflow 出现 'no kernel image is available for execution on the device' #10445

Open
wuwei-bit opened this issue Mar 13, 2024 · 1 comment

Comments

@wuwei-bit
Copy link

Description

oneflow 版本为 0.9.1.dev20240203+cu118

:/workspace# pip list | grep oneflow
oneflow                   0.9.1.dev20240203+cu118

cuda版本为 11.5

:/workspace# nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0

显卡信息

NVIDIA-SMI 525.147.05   Driver Version: 525.147.05   CUDA Version: 12.0  

运行“快速上手”教程中的代码:https://docs.oneflow.org/master/basics/01_quickstart.html#
报错如下:

The CUDA device 'NVIDIA TITAN X (Pascal)' with capability 61 is not compatible with the current OneFlow installation. The current program may th
row a 'no kernel image is available for execution on the device' error or hang for a long time. Please reinstall OneFlow compiled with a newer version of CUDA.

oneflow使用 cuda11.8 编译的,按理说可以进行向下兼容 cuda11.5。但是为什么报错信息中说应该使用更新版本的 cuda 重新编译 oneflow? 我应该如何处理这个问题?
另外, README.md 中建议使用 cuda 10.2, 这个是文档更新问题吗?

Alternatives

No response

@levi131
Copy link
Contributor

levi131 commented Mar 13, 2024

看起来您在尝试使用OneFlow的 nightly 安装包,nightly 安装包是使用 cuda 11.8 针对 75,80,86 三种 CUDA 架构设备编译,您的设备是61 CUDA架构的设备,且cuda版本为11.5,使用中会有问题。OneFlow 框架是支持 10.0 以上版本 cuda,支持 60 以上CUDA架构 设备编译的,您可以选择从源代码编译 OneFlow 使用,在CMake 命令中通过如 -DCMAKE_CUDA_ARCHITECTURES=61 指定 CUDA 架构,通过如 -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda -DCUDNN_ROOT_DIR=/usr/local/cudnn 指定使用的cuda和cudnn路径。我们之后也会更新README.md中的描述。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants