Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

出现此错误:RuntimeError: [f 0514 20:13:59.276076 48 log.cc:608] Check failed ret(256) == 0(0) Run cmd failed: “/home/hongtauo/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc” #438

Closed
Hongtauo opened this issue May 14, 2023 · 8 comments

Comments

@Hongtauo
Copy link

Describe the bug

安装jittor运行测试样例的时候出现此错误
系统版本:Ubuntu 22.04.2 LTS
anaconda:Anaconda3-2023.03-1-Linux-x86_64
python:3.8

Full Log

[i 0514 20:13:58.758510 48 compiler.py:955] Jittor(1.3.7.16) src: /home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor
[i 0514 20:13:58.760140 48 compiler.py:956] g++ at /usr/bin/g++(11.3.0)
[i 0514 20:13:58.760189 48 compiler.py:957] cache_path: /home/hongtauo/.cache/jittor/jt1.3.7/g++11.3.0/py3.8.16/Linux-5.19.0-3x84/12thGenIntelRCx5f/default
[i 0514 20:13:58.773380 48 install_cuda.py:93] cuda_driver_version: [12, 0]
[i 0514 20:13:58.775870 48 init.py:411] Found /home/hongtauo/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc(11.2.152) at /home/hongtauo/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc.
[i 0514 20:13:58.810841 48 init.py:411] Found gdb(12.1) at /usr/bin/gdb.
[i 0514 20:13:58.812979 48 init.py:411] Found addr2line(2.38) at /usr/bin/addr2line.
[i 0514 20:13:58.884676 48 compiler.py:1010] cuda key:cu11.2.152_sm_86
[i 0514 20:13:58.971765 48 init.py:227] Total mem: 15.30GB, using 5 procs for compiling.
/usr/include/stdio.h(189): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(201): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(223): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(260): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(285): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(294): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(303): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(309): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(315): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(830): error: attribute "malloc" does not take arguments

/usr/include/stdlib.h(566): error: attribute "malloc" does not take arguments

/usr/include/stdlib.h(570): error: attribute "malloc" does not take arguments

/usr/include/stdlib.h(799): error: attribute "malloc" does not take arguments

/usr/include/c++/11/type_traits(1406): error: type name is not allowed

/usr/include/c++/11/type_traits(1406): error: type name is not allowed

/usr/include/c++/11/type_traits(1406): error: identifier "__is_same" is undefined

/usr/include/wchar.h(155): error: attribute "malloc" does not take arguments

/usr/include/wchar.h(582): error: attribute "malloc" does not take arguments

/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor/src/misc/cstr.h(19): error: no instance of overloaded function "std::unique_ptr<_Tp [], _Dp>::reset [with _Tp=char, _Dp=std::default_delete<char []>]" matches the argument list
argument types are: (char *)
object type is: jittor::unique_ptr<char []>

/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor/src/misc/cstr.h(25): error: no instance of overloaded function "std::unique_ptr<_Tp [], _Dp>::reset [with _Tp=char, _Dp=std::default_delete<char []>]" matches the argument list
argument types are: (char *)
object type is: jittor::unique_ptr<char []>

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long, std::is_same<int, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long, _Ret=int, _CharT=char, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6620): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long, std::is_same<long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long, _Ret=long, _CharT=char, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6625): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const unsigned long, std::is_same<unsigned long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=unsigned long, _Ret=unsigned long, _CharT=char, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6630): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long long, std::is_same<long long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long long, _Ret=long long, _CharT=char, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6635): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const unsigned long long, std::is_same<unsigned long long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=unsigned long long, _Ret=unsigned long long, _CharT=char, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6640): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const float, std::is_same<float, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=float, _Ret=float, _CharT=char, _Base=<>]"
/usr/include/c++/11/bits/basic_string.h(6646): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const double, std::is_same<double, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=double, _Ret=double, _CharT=char, _Base=<>]"
/usr/include/c++/11/bits/basic_string.h(6650): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long double, std::is_same<long double, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long double, _Ret=long double, _CharT=char, _Base=<>]"
/usr/include/c++/11/bits/basic_string.h(6654): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long, std::is_same<int, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long, _Ret=int, _CharT=wchar_t, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6751): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long, std::is_same<long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long, _Ret=long, _CharT=wchar_t, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6756): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const unsigned long, std::is_same<unsigned long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=unsigned long, _Ret=unsigned long, _CharT=wchar_t, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6761): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long long, std::is_same<long long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long long, _Ret=long long, _CharT=wchar_t, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6766): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const unsigned long long, std::is_same<unsigned long long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=unsigned long long, _Ret=unsigned long long, _CharT=wchar_t, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6771): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const float, std::is_same<float, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=float, _Ret=float, _CharT=wchar_t, _Base=<>]"
/usr/include/c++/11/bits/basic_string.h(6777): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const double, std::is_same<double, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=double, _Ret=double, _CharT=wchar_t, _Base=<>]"
/usr/include/c++/11/bits/basic_string.h(6781): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long double, std::is_same<long double, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long double, _Ret=long double, _CharT=wchar_t, _Base=<>]"
/usr/include/c++/11/bits/basic_string.h(6785): here

36 errors detected in the compilation of "/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor/src/misc/nan_checker.cu".
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor_utils/init.py", line 197, in do_compile
return cc.cache_compile(cmd, cache_path, jittor_path)
RuntimeError: [f 0514 20:13:59.276076 48 log.cc:608] Check failed ret(256) == 0(0) Run cmd failed: "/home/hongtauo/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc" "/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor/src/misc/nan_checker.cu" -std=c++14 -Xcompiler -fPIC -Xcompiler -march=native -Xcompiler -fdiagnostics-color=always -I"/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor/src" -I/home/hongtauo/anaconda3/envs/jittor/include/python3.8 -I/home/hongtauo/anaconda3/envs/jittor/include/python3.8 -DHAS_CUDA -DIS_CUDA -I"/home/hongtauo/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/include" -I"/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor/extern/cuda/inc" -I"/home/hongtauo/.cache/jittor/jt1.3.7/g++11.3.0/py3.8.16/Linux-5.19.0-3x84/12thGenIntelRCx5f/default/cu11.2.152_sm_86" -O2 -c -o "/home/hongtauo/.cache/jittor/jt1.3.7/g++11.3.0/py3.8.16/Linux-5.19.0-3x84/12thGenIntelRCx5f/default/cu11.2.152_sm_86/obj_files/nan_checker.cu.o" -x cu --cudart=shared -ccbin="/usr/bin/g++" -w -I"/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor/extern/cuda/inc"
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/runpy.py", line 185, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/runpy.py", line 111, in get_module_details
import(pkg_name)
File "/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor/init.py", line 18, in
from . import compiler
File "/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor/compiler.py", line 1353, in
compile(cc_path, cc_flags+opt_flags, files, 'jittor_core'+extension_suffix)
File "/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor/compiler.py", line 151, in compile
jit_utils.run_cmds(cmds, cache_path, jittor_path, "Compiling "+base_output)
File "/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor_utils/init.py", line 251, in run_cmds
for i,
in enumerate(p.imap_unordered(do_compile, cmds)):
File "/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/multiprocessing/pool.py", line 868, in next
raise value
RuntimeError: [f 0514 20:13:59.276076 48 log.cc:608] Check failed ret(256) == 0(0) Run cmd failed: "/home/hongtauo/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc" "/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor/src/misc/nan_checker.cu" -std=c++14 -Xcompiler -fPIC -Xcompiler -march=native -Xcompiler -fdiagnostics-color=always -I"/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor/src" -I/home/hongtauo/anaconda3/envs/jittor/include/python3.8 -I/home/hongtauo/anaconda3/envs/jittor/include/python3.8 -DHAS_CUDA -DIS_CUDA -I"/home/hongtauo/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/include" -I"/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor/extern/cuda/inc" -I"/home/hongtauo/.cache/jittor/jt1.3.7/g++11.3.0/py3.8.16/Linux-5.19.0-3x84/12thGenIntelRCx5f/default/cu11.2.152_sm_86" -O2 -c -o "/home/hongtauo/.cache/jittor/jt1.3.7/g++11.3.0/py3.8.16/Linux-5.19.0-3x84/12thGenIntelRCx5f/default/cu11.2.152_sm_86/obj_files/nan_checker.cu.o" -x cu --cudart=shared -ccbin="/usr/bin/g++" -w -I"/home/hongtauo/anaconda3/envs/jittor/lib/python3.8/site-packages/jittor/extern/cuda/inc"

@Hongtauo
Copy link
Author

报错语句为:import jittor as jt
jittor版本:1.3.7.16

@Hongtauo
Copy link
Author

已解决,安装最新版本的cuda和cudnn即可

@frank7716
Copy link

请问如何解决的,系统为unbuntu2022.04,nvidia驱动更新为530了,还是出现这个错误

[i 0523 15:15:38.751416 56 compiler.py:955] Jittor(1.3.7.16) src: /home/frank/jittor/lib/python3.10/site-packages/jittor
[i 0523 15:15:38.754852 56 compiler.py:956] g++ at /usr/bin/g++(11.3.0)
[i 0523 15:15:38.754886 56 compiler.py:957] cache_path: /home/frank/.cache/jittor/jt1.3.7/g++11.3.0/py3.10.6/Linux-5.19.0-4x62/IntelRXeonRGolx7a/default
[i 0523 15:15:38.783151 56 install_cuda.py:93] cuda_driver_version: [12, 1]
[i 0523 15:15:38.788650 56 init.py:411] Found /home/frank/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc(11.2.152) at /home/frank/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc.
[i 0523 15:15:38.875488 56 init.py:411] Found gdb(12.1) at /usr/bin/gdb.
[i 0523 15:15:38.879856 56 init.py:411] Found addr2line(2.38) at /usr/bin/addr2line.
[i 0523 15:15:38.991496 56 compiler.py:1010] cuda key:cu11.2.152_sm_61
[i 0523 15:15:39.178956 56 init.py:227] Total mem: 125.52GB, using 16 procs for compiling.
/usr/include/stdio.h(189): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(201): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(223): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(260): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(285): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(294): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(303): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(309): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(315): error: attribute "malloc" does not take arguments

/usr/include/stdio.h(830): error: attribute "malloc" does not take arguments

/usr/include/stdlib.h(566): error: attribute "malloc" does not take arguments

/usr/include/stdlib.h(570): error: attribute "malloc" does not take arguments

/usr/include/stdlib.h(799): error: attribute "malloc" does not take arguments

/usr/include/c++/11/type_traits(1406): error: type name is not allowed

/usr/include/c++/11/type_traits(1406): error: type name is not allowed

/usr/include/c++/11/type_traits(1406): error: identifier "__is_same" is undefined

/usr/include/wchar.h(155): error: attribute "malloc" does not take arguments

/usr/include/wchar.h(582): error: attribute "malloc" does not take arguments

/home/frank/jittor/lib/python3.10/site-packages/jittor/src/misc/cstr.h(19): error: no instance of overloaded function "std::unique_ptr<_Tp [], _Dp>::reset [with _Tp=char, _Dp=std::default_delete<char []>]" matches the argument list
argument types are: (char *)
object type is: jittor::unique_ptr<char []>

/home/frank/jittor/lib/python3.10/site-packages/jittor/src/misc/cstr.h(25): error: no instance of overloaded function "std::unique_ptr<_Tp [], _Dp>::reset [with _Tp=char, _Dp=std::default_delete<char []>]" matches the argument list
argument types are: (char *)
object type is: jittor::unique_ptr<char []>

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long, std::is_same<int, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long, _Ret=int, _CharT=char, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6620): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long, std::is_same<long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long, _Ret=long, _CharT=char, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6625): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const unsigned long, std::is_same<unsigned long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=unsigned long, _Ret=unsigned long, _CharT=char, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6630): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long long, std::is_same<long long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long long, _Ret=long long, _CharT=char, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6635): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const unsigned long long, std::is_same<unsigned long long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=unsigned long long, _Ret=unsigned long long, _CharT=char, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6640): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const float, std::is_same<float, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=float, _Ret=float, _CharT=char, _Base=<>]"
/usr/include/c++/11/bits/basic_string.h(6646): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const double, std::is_same<double, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=double, _Ret=double, _CharT=char, _Base=<>]"
/usr/include/c++/11/bits/basic_string.h(6650): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long double, std::is_same<long double, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long double, _Ret=long double, _CharT=char, _Base=<>]"
/usr/include/c++/11/bits/basic_string.h(6654): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long, std::is_same<int, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long, _Ret=int, _CharT=wchar_t, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6751): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long, std::is_same<long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long, _Ret=long, _CharT=wchar_t, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6756): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const unsigned long, std::is_same<unsigned long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=unsigned long, _Ret=unsigned long, _CharT=wchar_t, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6761): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long long, std::is_same<long long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long long, _Ret=long long, _CharT=wchar_t, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6766): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const unsigned long long, std::is_same<unsigned long long, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=unsigned long long, _Ret=unsigned long long, _CharT=wchar_t, _Base=]"
/usr/include/c++/11/bits/basic_string.h(6771): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const float, std::is_same<float, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=float, _Ret=float, _CharT=wchar_t, _Base=<>]"
/usr/include/c++/11/bits/basic_string.h(6777): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const double, std::is_same<double, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=double, _Ret=double, _CharT=wchar_t, _Base=<>]"
/usr/include/c++/11/bits/basic_string.h(6781): here

/usr/include/c++/11/ext/string_conversions.h(85): error: no instance of overloaded function "_Range_chk::_S_chk" matches the argument list
argument types are: (const long double, std::is_same<long double, int>)
detected during instantiation of "_Ret __gnu_cxx::__stoa(_TRet (*)(const _CharT *, _CharT **, _Base...), const char *, const _CharT *, std::size_t *, _Base...) [with _TRet=long double, _Ret=long double, _CharT=wchar_t, _Base=<>]"
/usr/include/c++/11/bits/basic_string.h(6785): here

36 errors detected in the compilation of "/home/frank/jittor/lib/python3.10/site-packages/jittor/src/misc/nan_checker.cu".
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/home/frank/jittor/lib/python3.10/site-packages/jittor_utils/init.py", line 197, in do_compile
return cc.cache_compile(cmd, cache_path, jittor_path)
RuntimeError: [f 0523 15:15:39.809532 56 log.cc:608] Check failed ret(256) == 0(0) Run cmd failed: "/home/frank/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc" "/home/frank/jittor/lib/python3.10/site-packages/jittor/src/misc/nan_checker.cu" -std=c++14 -Xcompiler -fPIC -Xcompiler -march=native -Xcompiler -fdiagnostics-color=always -I"/home/frank/jittor/lib/python3.10/site-packages/jittor/src" -I/usr/include/python3.10 -I/usr/include/python3.10 -DHAS_CUDA -DIS_CUDA -I"/home/frank/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/include" -I"/home/frank/jittor/lib/python3.10/site-packages/jittor/extern/cuda/inc" -I"/home/frank/.cache/jittor/jt1.3.7/g++11.3.0/py3.10.6/Linux-5.19.0-4x62/IntelRXeonRGolx7a/default/cu11.2.152_sm_61" -O2 -c -o "/home/frank/.cache/jittor/jt1.3.7/g++11.3.0/py3.10.6/Linux-5.19.0-4x62/IntelRXeonRGolx7a/default/cu11.2.152_sm_61/obj_files/nan_checker.cu.o" -x cu --cudart=shared -ccbin="/usr/bin/g++" -w -I"/home/frank/jittor/lib/python3.10/site-packages/jittor/extern/cuda/inc"
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/frank/jittor/JittorLLMs/cli_demo.py", line 8, in
model = models.get_model(args)
File "/home/frank/jittor/JittorLLMs/models/init.py", line 38, in get_model
globals()f"get_{model_name}"
File "/home/frank/jittor/JittorLLMs/models/util.py", line 51, in get_chatglm
new_path.append(download_fromhub(f"jittorhub://{f}", tdir="chat-glm"))
File "/home/frank/jittor/JittorLLMs/models/util.py", line 5, in download_fromhub
import jittor as jt
File "/home/frank/jittor/lib/python3.10/site-packages/jittor/init.py", line 18, in
from . import compiler
File "/home/frank/jittor/lib/python3.10/site-packages/jittor/compiler.py", line 1353, in
compile(cc_path, cc_flags+opt_flags, files, 'jittor_core'+extension_suffix)
File "/home/frank/jittor/lib/python3.10/site-packages/jittor/compiler.py", line 151, in compile
jit_utils.run_cmds(cmds, cache_path, jittor_path, "Compiling "+base_output)
File "/home/frank/jittor/lib/python3.10/site-packages/jittor_utils/init.py", line 251, in run_cmds
for i,_ in enumerate(p.imap_unordered(do_compile, cmds)):
File "/usr/lib/python3.10/multiprocessing/pool.py", line 873, in next
raise value
RuntimeError: [f 0523 15:15:39.809532 56 log.cc:608] Check failed ret(256) == 0(0) Run cmd failed: "/home/frank/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/bin/nvcc" "/home/frank/jittor/lib/python3.10/site-packages/jittor/src/misc/nan_checker.cu" -std=c++14 -Xcompiler -fPIC -Xcompiler -march=native -Xcompiler -fdiagnostics-color=always -I"/home/frank/jittor/lib/python3.10/site-packages/jittor/src" -I/usr/include/python3.10 -I/usr/include/python3.10 -DHAS_CUDA -DIS_CUDA -I"/home/frank/.cache/jittor/jtcuda/cuda11.2_cudnn8_linux/include" -I"/home/frank/jittor/lib/python3.10/site-packages/jittor/extern/cuda/inc" -I"/home/frank/.cache/jittor/jt1.3.7/g++11.3.0/py3.10.6/Linux-5.19.0-4x62/IntelRXeonRGolx7a/default/cu11.2.152_sm_61" -O2 -c -o "/home/frank/.cache/jittor/jt1.3.7/g++11.3.0/py3.10.6/Linux-5.19.0-4x62/IntelRXeonRGolx7a/default/cu11.2.152_sm_61/obj_files/nan_checker.cu.o" -x cu --cudart=shared -ccbin="/usr/bin/g++" -w -I"/home/frank/jittor/lib/python3.10/site-packages/jittor/extern/cuda/inc"

@frank7716
Copy link

安装最新版本的cuda和cudnn是直接手动安装吗?我看jittor使用的cuda和cudnn是jittor中的cuda11.2_cudnn8_linx.tgz,在 /home/frank/.cache/jittor/jtcuda/下,应该是自动下载的

@Hongtauo
Copy link
Author

我的解决过程如下:
1、安装cuda的最新版本,安装匹配的cudnn版本
2、在终端中使用nvcc -V命令检查环境变量,如果提示缺失xxx依赖,按照提示内容安装
3、在jittor内将cuda环境变量添加进去“export nvcc_path="/usr/local/cuda/bin/nvcc" (我就是这一步一直没成功,导致调用gpu的时候报错)
4、按照官方安装教程中的方法检查jittor框架是否完整“python3.x -m jittor.test.test_resnet” x是你的python版本

@Hongtauo
Copy link
Author

安装最新版本的cuda和cudnn是直接手动安装吗?我看jittor使用的cuda和cudnn是jittor中的cuda11.2_cudnn8_linx.tgz,在 /home/frank/.cache/jittor/jtcuda/下,应该是自动下载的

我是自己配置的

@Hongtauo
Copy link
Author

安装最新版本的cuda和cudnn是直接手动安装吗?我看jittor使用的cuda和cudnn是jittor中的cuda11.2_cudnn8_linx.tgz,在 /home/frank/.cache/jittor/jtcuda/下,应该是自动下载的

看你的报错,你检查下你的后端编译器设置了没有,然后在检查系统中cuda和cudnn是否配置成功,最后在jittor内设置一下cuda的环境路径

@frank7716
Copy link

Hongtauo,谢谢,按你的指导,安装解决了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants