LibTorch gpu cmake error

[zhaoyinjiang9825]

Hello, when I execute " mkdir build && cd build && cmake -DGRPC=ON ..", the following error is reported,
Native environment:
centors 7.9
nvidia: 11.3
cuda version: 11

---

(wenet_gpu) [ZYJ@localhost build]$ cmake -DGPU=ON ..
-- The C compiler identification is GNU 4.8.5
-- The CXX compiler identification is GNU 4.8.5
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Populating libtorch
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-subbuild
[ 11%] Performing download step (download, verify and extract) for 'libtorch-populate'
-- verifying file...
file='/home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-subbuild/libtorch-populate-prefix/src/libtorch-shared-with-deps-1.10.0%2Bcu113.zip'
-- File already exists and hash match (skip download):
file='/home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-subbuild/libtorch-populate-prefix/src/libtorch-shared-with-deps-1.10.0%2Bcu113.zip'
SHA256='0996a6a4ea8bbc1137b4fb0476eeca25b5efd8ed38955218dec1b73929090053'
-- extracting...
src='/home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-subbuild/libtorch-populate-prefix/src/libtorch-shared-with-deps-1.10.0%2Bcu113.zip'
dst='/home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src'
-- extracting... [tar xfz]
-- extracting... [analysis]
-- extracting... [rename]
-- extracting... [clean up]
-- extracting... done
[ 22%] No patch step for 'libtorch-populate'
[ 33%] No update step for 'libtorch-populate'
[ 44%] No configure step for 'libtorch-populate'
[ 55%] No build step for 'libtorch-populate'
[ 66%] No install step for 'libtorch-populate'
[ 77%] No test step for 'libtorch-populate'
[ 88%] Completed 'libtorch-populate'
[100%] Built target libtorch-populate
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found CUDA: /usr/local/cuda-11.3 (found version "11.3")
-- Caffe2: CUDA detected: 11.3
-- Caffe2: CUDA nvcc is: /usr/local/cuda-11.3/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda-11.3
CMake Error at fc_base/libtorch-src/share/cmake/Caffe2/public/cuda.cmake:75 (message):
Caffe2: Couldn't determine version from header: Change Dir:
/home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/gmake cmTC_3d968/fast

/usr/bin/gmake -f CMakeFiles/cmTC_3d968.dir/build.make
CMakeFiles/cmTC_3d968.dir/build

gmake[1]:
进入目录“/home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/build/CMakeFiles/CMakeTmp”

Building CXX object CMakeFiles/cmTC_3d968.dir/detect_cuda_version.cc.o

/usr/bin/c++ -I/usr/local/cuda-11.3/include -std=c++14 -pthread -fPIC -o
CMakeFiles/cmTC_3d968.dir/detect_cuda_version.cc.o -c
/home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/build/detect_cuda_version.cc

c++: 错误：unrecognized command line option ‘-std=c++14’

gmake[1]: *** [CMakeFiles/cmTC_3d968.dir/detect_cuda_version.cc.o] 错误 1

gmake[1]:
离开目录“/home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/build/CMakeFiles/CMakeTmp”

gmake: *** [cmTC_3d968/fast] 错误 2

Call Stack (most recent call first):
fc_base/libtorch-src/share/cmake/Caffe2/Caffe2Config.cmake:88 (include)
fc_base/libtorch-src/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
cmake/libtorch.cmake:52 (find_package)
CMakeLists.txt:35 (include)

-- Configuring incomplete, errors occurred!
See also "/home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/build/CMakeFiles/CMakeOutput.log".
See also "/home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/build/CMakeFiles/CMakeError.log".

---

please what should Ido?

[robin1001]

what's the version of your GCC?

[zhaoyinjiang9825]

My gcc version is 7.3@robin1001

[zhaoyinjiang9825]

And I deployed libtorch_GPU in the provided docker, and there was also a cmake error！
dockerfile

---

FROM ubuntu:latest
MAINTAINER zhendong.peng@mobvoi.com
ENV DEBIAN_FRONTEND=noninteractive
RUN sed -i s@/archive.ubuntu.com/@/mirrors.tuna.tsinghua.edu.cn/@g /etc/apt/sources.list
RUN apt-get update && apt-get install -y git cmake wget build-essential
RUN git clone https://github.com/wenet-e2e/wenet.git /home/wenet
ARG model=20210618_u2pp_conformer_libtorch.tar.gz
RUN wget -P /home https://wenet-1256283475.cos.ap-shanghai.myqcloud.com/models/aishell2/$model
RUN tar -xzf /home/$model -C /home
ARG build=/home/wenet/runtime/server/x86/build
RUN mkdir $build && cd $build && cmake -DGPU=ON .. -DCMAKE_BUILD_TYPE=Release && cmake --build . --config Release

---

cmake error in container:

---

root@821526b736a0:/home/wenet/runtime/server/x86/build# cmake -DGPU=ON ..
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Populating libtorch
-- Configuring done
-- Generating done
-- Build files have been written to: /home/wenet/runtime/LibTorch/fc_base/libtorch-subbuild
[ 11%] Creating directories for 'libtorch-populate'
[ 22%] Performing download step (download, verify and extract) for 'libtorch-populate'
-- verifying file...
file='/home/wenet/runtime/LibTorch/fc_base/libtorch-subbuild/libtorch-populate-prefix/src/libtorch-shared-with-deps-1.10.0%2Bcu113.zip'
-- File already exists and hash match (skip download):
file='/home/wenet/runtime/LibTorch/fc_base/libtorch-subbuild/libtorch-populate-prefix/src/libtorch-shared-with-deps-1.10.0%2Bcu113.zip'
SHA256='0996a6a4ea8bbc1137b4fb0476eeca25b5efd8ed38955218dec1b73929090053'
-- extracting...
src='/home/wenet/runtime/LibTorch/fc_base/libtorch-subbuild/libtorch-populate-prefix/src/libtorch-shared-with-deps-1.10.0%2Bcu113.zip'
dst='/home/wenet/runtime/LibTorch/fc_base/libtorch-src'
-- extracting... [tar xfz]
-- extracting... [analysis]
-- extracting... [rename]
-- extracting... [clean up]
-- extracting... done
[ 33%] No patch step for 'libtorch-populate'
[ 44%] No update step for 'libtorch-populate'
[ 55%] No configure step for 'libtorch-populate'
[ 66%] No build step for 'libtorch-populate'
[ 77%] No install step for 'libtorch-populate'
[ 88%] No test step for 'libtorch-populate'
[100%] Completed 'libtorch-populate'
[100%] Built target libtorch-populate
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found CUDA: /usr/local/cuda-11.3 (found version "11.3")
-- Caffe2: CUDA detected: 11.3
-- Caffe2: CUDA nvcc is: /usr/local/cuda-11.3/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda-11.3
-- Caffe2: Header version is: 11.3
-- Found CUDNN: /usr/local/cuda-11.3/lib64/libcudnn.so
-- Found cuDNN: v? (include: /usr/local/cuda-11.3/include, library: /usr/local/cuda-11.3/lib64/libcudnn.so)
CMake Error at fc_base/libtorch-src/share/cmake/Caffe2/public/cuda.cmake:176 (message):
PyTorch requires cuDNN 7 and above.
Call Stack (most recent call first):
fc_base/libtorch-src/share/cmake/Caffe2/Caffe2Config.cmake:88 (include)
fc_base/libtorch-src/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
cmake/libtorch.cmake:52 (find_package)
CMakeLists.txt:35 (include)

-- Configuring incomplete, errors occurred!
See also "/home/wenet/runtime/server/x86/build/CMakeFiles/CMakeOutput.log".
See also "/home/wenet/runtime/server/x86/build/CMakeFiles/CMakeError.log".

---

[zhaoyinjiang9825]

@robin1001

[robin1001]

@yuekaizhang @veelion any idea on the problem?

[yuekaizhang]

Could you try one of nvidia docker image? nvcr.io/nvidia/pytorch:xx.xx-py3 For example, 22.01.

[zhaoyinjiang9825]

Hi, the compilation problem has been solved, the gcc version needs to be greater than 7.3, and when cuda is installed, copy cudnn_version.h to the /usr/local/cuda/include/ directory, and it can be compiled perfectly, but during the calling process, the got the following error:

---

export GLOG_logtostderr=1
export GLOG_v=2
wav_path=/home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/model/proper_noun/123.wav model_dir=/home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/model/20210815_unified_conformer_libtorch
./build/bin/decoder_main --chunk_size -1 --wav_path $wav_path --model_path $model_dir/final.zip --unit_path $model_dir/units.txt 2>&1 | tee log.txt
An error occurred

---

I0801 13:29:29.093011 61770 params.h:135] Reading torch model /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/model/20210815_unified_conformer_libtorch/final.zip
I0801 13:29:29.109166 61770 torch_asr_model.cc:36] Num intra-op threads: 1
I0801 13:29:29.109195 61770 torch_asr_model.cc:37] Num inter-op threads: 1
I0801 13:29:29.532994 61770 torch_asr_model.cc:47] CUDA available! Running on GPU
terminate called after throwing an instance of 'c10::NotImplementedError'
what(): The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
File "code/torch/torch/nn/quantized/modules/linear.py", line 18, in setstate
state: Tuple[Tensor, Optional[Tensor], bool, int]) -> NoneType:
self.dtype = (state)[3]
_1 = (self).set_weight_bias((state)[0], (state)[1], )
~~~~~~~~~~~~~~~~~~~~~ <--- HERE
self.training = (state)[2]
return None
File "code/torch/torch/nn/quantized/modules/linear.py", line 41, in set_weight_bias
_10 = "Unsupported dtype on dynamic quantized linear!"
if torch.eq(self.dtype, 12):
_11 = ops.quantized.linear_prepack(weight, bias)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
self._packed_params = _11
else:

Traceback of TorchScript, original code (most recent call last):
File "/home/binbzha/miniconda3/envs/wenet/lib/python3.8/site-packages/torch/nn/quantized/modules/linear.py", line 94, in setstate
def setstate(self, state):
self.dtype = state[3]
self.set_weight_bias(state[0], state[1])
~~~~~~~~~~~~~~~~~~~~ <--- HERE
self.training = state[2]
File "/home/binbzha/miniconda3/envs/wenet/lib/python3.8/site-packages/torch/nn/quantized/modules/linear.py", line 24, in set_weight_bias
def set_weight_bias(self, weight: torch.Tensor, bias: Optional[torch.Tensor]) -> None:
if self.dtype == torch.qint8:
self._packed_params = torch.ops.quantized.linear_prepack(weight, bias)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
elif self.dtype == torch.float16:
self._packed_params = torch.ops.quantized.linear_prepack_fp16(weight, bias)
RuntimeError: Could not run 'quantized::linear_prepack' with arguments from the 'QuantizedCUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'quantized::linear_prepack' is only available for these backends: [QuantizedCPU, BackendSelect, Python, Named, Conjugate, Negative, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradLazy, AutogradXPU, AutogradMLC, Tracer, UNKNOWN_TENSOR_TYPE_ID, Autocast, Batched, VmapMode].

QuantizedCPU: registered at ../aten/src/ATen/native/quantized/cpu/qlinear_prepack.cpp:324 [kernel]
BackendSelect: fallthrough registered at ../aten/src/ATen/core/BackendSelectFallbackKernel.cpp:3 [backend fallback]
Python: registered at ../aten/src/ATen/core/PythonFallbackKernel.cpp:47 [backend fallback]
Named: registered at ../aten/src/ATen/core/NamedRegistrations.cpp:7 [backend fallback]
Conjugate: registered at ../aten/src/ATen/ConjugateFallback.cpp:18 [backend fallback]
Negative: registered at ../aten/src/ATen/native/NegateFallback.cpp:18 [backend fallback]
ADInplaceOrView: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:64 [backend fallback]
AutogradOther: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:35 [backend fallback]
AutogradCPU: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:39 [backend fallback]
AutogradCUDA: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:47 [backend fallback]
AutogradXLA: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:51 [backend fallback]
AutogradLazy: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:55 [backend fallback]
AutogradXPU: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:43 [backend fallback]
AutogradMLC: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:59 [backend fallback]
Tracer: registered at ../torch/csrc/autograd/TraceTypeManual.cpp:291 [backend fallback]
UNKNOWN_TENSOR_TYPE_ID: fallthrough registered at ../aten/src/ATen/autocast_mode.cpp:466 [backend fallback]
Autocast: fallthrough registered at ../aten/src/ATen/autocast_mode.cpp:305 [backend fallback]
Batched: registered at ../aten/src/ATen/BatchingRegistrations.cpp:1016 [backend fallback]
VmapMode: fallthrough registered at ../aten/src/ATen/VmapModeRegistrations.cpp:33 [backend fallback]

Exception raised from reportError at ../aten/src/ATen/core/dispatch/OperatorEntry.cpp:431 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x42 (0x7f77049f1d62 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libc10.so)
frame #1: c10::impl::OperatorEntry::reportError(c10::DispatchKey) const + 0x6f2 (0x7f77060f5fb2 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #2: + 0x3e88796 (0x7f7708cdc796 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #3: + 0x3bd0a59 (0x7f7708a24a59 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #4: torch::jit::InterpreterState::run(std::vector<c10::IValue, std::allocatorc10::IValue >&) + 0x1c (0x7f7708a136fc in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #5: + 0x3bb7492 (0x7f7708a0b492 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #6: torch::jit::GraphFunction::operator()(std::vector<c10::IValue, std::allocatorc10::IValue >, std::unordered_map<std::string, c10::IValue, std::hashstd::string, std::equal_tostd::string, std::allocator<std::pair<std::string const, c10::IValue> > > const&) + 0x39 (0x7f7708730509 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #7: + 0x3c5f1b5 (0x7f7708ab31b5 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #8: + 0x3c60342 (0x7f7708ab4342 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #9: + 0x3c96b20 (0x7f7708aeab20 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #10: torch::jit::Unpickler::readInstruction() + 0x78f (0x7f7708af321f in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #11: torch::jit::Unpickler::run() + 0x90 (0x7f7708af4b20 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #12: torch::jit::Unpickler::parse_ivalue() + 0x18 (0x7f7708af4c98 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #13: torch::jit::readArchiveAndTensors(std::string const&, std::string const&, std::string const&, c10::optional<std::function<c10::StrongTypePtr (c10::QualifiedName const&)> >, c10::optional<std::function<c10::intrusive_ptr<c10::ivalue::Object, c10::detail::intrusive_target_default_null_typec10::ivalue::Object > (c10::StrongTypePtr, c10::IValue)> >, c10::optionalc10::Device, caffe2::serialize::PyTorchStreamReader&, std::shared_ptrtorch::jit::DeserializationStorageContext) + 0x4a0 (0x7f7708ab5ca0 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #14: + 0x3c5b5fb (0x7f7708aaf5fb in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #15: + 0x3c5d08a (0x7f7708ab108a in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #16: torch::jit::load(std::shared_ptrcaffe2::serialize::ReadAdapterInterface, c10::optionalc10::Device, std::unordered_map<std::string, std::string, std::hashstd::string, std::equal_tostd::string, std::allocator<std::pair<std::string const, std::string> > >&) + 0x1af (0x7f7708ab244f in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #17: torch::jit::load(std::string const&, c10::optionalc10::Device, std::unordered_map<std::string, std::string, std::hashstd::string, std::equal_tostd::string, std::allocator<std::pair<std::string const, std::string> > >&) + 0xad (0x7f7708ab441d in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #18: torch::jit::load(std::string const&, c10::optionalc10::Device) + 0x54 (0x7f7708ab44f4 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #19: ./build/bin/decoder_main() [0x5b34a7]
frame #20: ./build/bin/decoder_main() [0x46fbf8]
frame #21: ./build/bin/decoder_main() [0x4710d8]
frame #22: __libc_start_main + 0xf5 (0x7f76c0803555 in /lib64/libc.so.6)
frame #23: ./build/bin/decoder_main() [0x46f739]

---

I think it's still a libtorch problem, do you have any ideas?

[zhaoyinjiang9825]

@robin1001@yuekaizhang

[zhaoyinjiang9825]

@yuekaizhang

[robin1001]

please do not use quantized model for GPU.

[zhaoyinjiang9825]

I don't use the quantitative model, use the Checkpoint Model test, it also got an error:

---

(wenet) [ZYJ@localhost LibTorch]$ model_dir=./model/20220506_u2pp_conformer_exp
(wenet) [ZYJ@localhost LibTorch]$ ./build/bin/decoder_main --chunk_size -1 --wav_path $wav_path --model_path $model_dir/final.zip --unit_path $model_dir/units.txt 2>&1 | tee log.txt
I0801 16:32:03.666874 33850 params.h:135] Reading torch model ./model/20220506_u2pp_conformer_exp/final.zip
I0801 16:32:03.683038 33850 torch_asr_model.cc:36] Num intra-op threads: 1
I0801 16:32:03.683090 33850 torch_asr_model.cc:37] Num inter-op threads: 1
I0801 16:32:03.919312 33850 torch_asr_model.cc:47] CUDA available! Running on GPU
terminate called after throwing an instance of 'c10::Error'
what(): open file failed because of errno 2 on fopen: , file path: ./model/20220506_u2pp_conformer_exp/final.zip
Exception raised from RAIIFile at ../caffe2/serialize/file_adapter.cc:27 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x42 (0x7f36f84fdd62 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::string const&) + 0x5b (0x7f36f84fa68b in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libc10.so)
frame #2: caffe2::serialize::FileAdapter::RAIIFile::RAIIFile(std::string const&) + 0xde (0x7f36fb0f937e in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #3: caffe2::serialize::FileAdapter::FileAdapter(std::string const&) + 0x31 (0x7f36fb0f9901 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #4: torch::jit::load(std::string const&, c10::optionalc10::Device, std::unordered_map<std::string, std::string, std::hashstd::string, std::equal_tostd::string, std::allocator<std::pair<std::string const, std::string> > >&) + 0x30 (0x7f36fc5c03a0 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #5: torch::jit::load(std::string const&, c10::optionalc10::Device) + 0x54 (0x7f36fc5c04f4 in /home/ZYJ/WeNet/wenet_gpu/wenet/runtime/LibTorch/fc_base/libtorch-src/lib/libtorch_cpu.so)
frame #6: ./build/bin/decoder_main() [0x5b34a7]
frame #7: ./build/bin/decoder_main() [0x46fbf8]
frame #8: ./build/bin/decoder_main() [0x4710d8]
frame #9: __libc_start_main + 0xf5 (0x7f36b430f555 in /lib64/libc.so.6)
frame #10: ./build/bin/decoder_main() [0x46f739]

---

how can i solve it？@robin1001

[robin1001]

decode_main requires runtime model. If GPU is compiled in, it requires float runtime model. We only give checkpoint model and quantized runtime model in pretrained models, both of them can not be used in decode_main with GPU compiled in.

[zhaoyinjiang9825]

Can the pre-trained models you released use GPU inference in websocket_server_main and grpc_server_main?

I want to use GPU for inference test now, besides tritonserver_GPU, is there any good way? Please let me know, thanks!
@robin1001

[yuekaizhang]

Can the pre-trained models you released use GPU inference in websocket_server_main and grpc_server_main?

I want to use GPU for inference test now, besides tritonserver_GPU, is there any good way? Please let me know, thanks! @robin1001

https://github.com/wenet-e2e/wenet/blob/main/examples/aishell/s0/run.sh#L204 You could export them from checkpoint models. Or if you just would like test, you could modify wenet/bin/recognize.py

[zhaoyinjiang9825]

ok, i'll try it! thank you very much

[zhaoyinjiang9825]

As you suggested:
"https://github.com/wenet-e2e/wenet/blob/main/examples/aishell/s0/run.sh#L204 You could export them from checkpoint models.
"
I refer to this and export the floating-point runtime model, so I can use the GPU to compile inference, right?
@robin1001