Skip to content

[WIP] add support for ultra-infer with cuda12.6 and cuda12.9 #4217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: develop
Choose a base branch
from

Conversation

zhang-prog
Copy link
Collaborator

No description provided.

@@ -258,7 +259,25 @@ def _install_hpi_deps(device_type):
package = "ultra-infer-npu-python"

with importlib.resources.path("paddlex", "hpip_links.html") as f:
install_packages([package], pip_install_opts=["--find-links", str(f)])
try:
version = importlib.metadata.version(package)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

建议用paddlex.utils.deps.get_dep_version

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

response = input(
f"The package '{package}' (version {version}) is already installed. Do you want to reinstall it? (y/n): "
)
if response.lower() == "y":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

也支持一下yes

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

else:
return
except importlib.metadata.PackageNotFoundError:
install_packages([package], pip_install_opts=["--find-links", str(f)])

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

建议这里可以再加一个检查和提示:

if not is_paddle2onnx_plugin_available():
    logging.info("The Paddle2ONNX plugin is not available. It is recommended to run `paddlex --install paddle2onnx` to install the Paddle2ONNX plugin to use the full functionality of high-performance inference.")

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

try:
version = importlib.metadata.version(package)
response = input(
f"The package '{package}' (version {version}) is already installed. Do you want to reinstall it? (y/n): "
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我们对外统一暴露plugin的概念吧,就不说package了

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -71,7 +71,11 @@ else()
message("Cannot compile with onnxruntime-gpu while in linux-aarch64 platform, fallback to onnxruntime-cpu")
set(ONNXRUNTIME_FILENAME "onnxruntime-linux-aarch64-${ONNXRUNTIME_VERSION}.tgz")
else()
set(ONNXRUNTIME_FILENAME "onnxruntime-linux-x64-gpu-${ONNXRUNTIME_VERSION}.tgz")
if(CUDA_VERSION MATCHES "12.6" OR CUDA_VERSION MATCHES "12.9")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里是否应该只匹配大版本号呢,还是说必须要匹配到minor version?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

好像没有修改?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

install_packages([package], pip_install_opts=["--find-links", str(f)])
else:
response = input(
f"The plugin '{package}' (version {version}) is already installed. Do you want to reinstall it? (y/n): "
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里建议:

The high-performance inference plugin is already installed (version {repr(version)}). Do you want to reinstall it? (y/n) 

@@ -258,7 +260,30 @@ def _install_hpi_deps(device_type):
package = "ultra-infer-npu-python"

with importlib.resources.path("paddlex", "hpip_links.html") as f:
install_packages([package], pip_install_opts=["--find-links", str(f)])
version = get_dep_version(package)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

待添加检查其他variant是否存在的逻辑,以及适配CUDA 12

@zhang-prog zhang-prog added the wip label Jun 16, 2025
@zhang-prog zhang-prog changed the title add support for ultra-infer with cuda12.6 and cuda12.9 [WIP] add support for ultra-infer with cuda12.6 and cuda12.9 Jun 16, 2025
install_packages(
[package],
pip_install_opts=[
"--force-reinstall",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

建议用标准的先卸载、再安装的方式,防止本地cache影响行为。此外,不应该--no-deps吧?不同版本依赖是可能会改变的

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants