Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix provider options when several providers are passed #653

Merged
merged 2 commits into from Jan 2, 2023

Conversation

fxmarty
Copy link
Collaborator

@fxmarty fxmarty commented Dec 29, 2022

When several providers are passed to the InferenceSession, which is the case when TensorrtExecutionProvider is chosen, the provider_options argument needs to be of the same length than providers, otherwise raising:

EP Error using ['TensorrtExecutionProvider', 'CUDAExecutionProvider']
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.

Reference: https://onnxruntime.ai/docs/api/python/api_summary.html#inferencesession

This was untested up to now. Still need to add a test for this PR.

In a next PR: remove all the code duplication for load_model() in modeling_ort.py, modeling_decoder.py, modeling_seq2seq.py. But I won't do it in this PR.

This should fix #606

@fxmarty fxmarty changed the title fix provider options when several providers are passed Fix provider options when several providers are passed Dec 29, 2022
@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Dec 29, 2022

The documentation is not available anymore as the PR was closed or merged.

Copy link
Contributor

@regisss regisss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@fxmarty fxmarty merged commit 8d1fe87 into huggingface:main Jan 2, 2023
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] tensorrt engine cannot store locally
3 participants