Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The compiler image deployed using kubernetes does not support the compilation of checkpoint to TensorRT #491

Closed
KellyZhang2020 opened this issue Jul 7, 2021 · 0 comments
Assignees

Comments

@KellyZhang2020
Copy link
Contributor

When compile the checkpoint model to TensorRT, the error message is as follows:

[TensorRT] ERROR: ../rtSafe/safeContext.cpp (110) - cuBLAS Error in initializeCommonContext: 1 (Could not initialize cublas, please check cuda installation.)
[TensorRT] ERROR: ../rtSafe/safeContext.cpp (110) - cuBLAS Error in initializeCommonContext: 1 (Could not initialize cublas, please check cuda installation.)
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/model_compiler/__init__.py", line 13, in compile_model
    'path': compiler.compile_from_json(config)
  File "/usr/local/lib/python3.6/dist-packages/model_compiler/compiler.py", line 50, in compile_from_json
    target_model = compiler(source=source_type.from_json(value), config=compiler_config_type.from_json(value))
  File "/usr/local/lib/python3.6/dist-packages/model_compiler/compilers/repository.py", line 118, in _compiler
    result = edge.compiler(result, inner_config)
  File "/usr/local/lib/python3.6/dist-packages/model_compiler/compilers/onnx_model_to_tensorrt_model.py", line 90, in compile_source
    raise ValueError('Unable to build CUDA engine')
ValueError: Unable to build CUDA engine
{'status': 'failure', 'error_msg': 'Unable to build CUDA engine'}
KellyZhang2020 added a commit that referenced this issue Jul 7, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
bors bot pushed a commit that referenced this issue Jul 7, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 8, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 8, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 9, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 9, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 9, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 10, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 13, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 19, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 19, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 19, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 20, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 20, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 20, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 20, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 20, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 20, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 20, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 20, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 20, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 20, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 20, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 20, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 20, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 20, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 21, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 21, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 22, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 23, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 23, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 23, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 23, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 23, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 23, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
KellyZhang2020 added a commit that referenced this issue Jul 23, 2021
Closes #491

Signed-off-by: zhangkaili <zhang.kaili@zte.com.cn>
@bors bors bot closed this as completed in de18b26 Jul 29, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant