[inference] Flipped usage of wait_for_model flag #649
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mishig25
requested review from
SBrandeis,
Wauplin,
coyotte508,
gary149 and
julien-c
Wauplin
approved these changes
May 3, 2024
Contributor
Wauplin
left a comment
Wauplin
left a comment
There was a problem hiding this comment.
Looks good at first glance
Collaborator
Author
|
will wait for others to confirm |
coyotte508
reviewed
May 3, 2024
| const response = await (options?.fetch ?? fetch)(url, info); | ||
|
|
||
| if (options?.retry_on_error !== false && response.status === 503 && !options?.wait_for_model) { | ||
| if (options?.retry_on_error !== false && response.status === 503 && options?.wait_for_model) { |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
Member
|
you should pass (but yes we could probably remove |
Collaborator
Author
|
@coyotte508 submitted #650, should I close this PR #649 then? |
Member
|
yes you can close :) |
Collaborator
Author
|
superceded by #650 |
mishig25
pushed a commit
that referenced
this pull request
May 3, 2024
from #649 (comment) > you should pass retry_on_error: false instead of wait_for_model: false in widgets, I think that would solve it > (but yes we could probably remove retry_on_error as some point and check wait_for_model for false, true or undefined) ### on main from @gary149 > 503 but no error appears in the UI? https://huggingface.co/microsoft/Phi-3-mini-128k-instruct?text=hi <img src="https://github.com/huggingface/huggingface.js/assets/11827707/c07ce78d-3e00-45e1-b07d-dfa709c2ebe5" width="200px"> ### this PR/branch https://github.com/huggingface/huggingface.js/assets/11827707/05cfd37b-fd15-482e-82ae-cdb76961707b
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I think
wait_for_modelusage is flipped incorrectly in #148. Check files tabWhat prompted this error:
from @gary149
Explanation: although widget sends
wait_for_model: false, inference.js does NOT throw error and tries to load the model and make a call to this model once it is loaded