Adding autocomplete to vllm model.py #20
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
oandreeva-nv
requested review from
dyastremsky,
tanmayv25,
rmccorm4 and
krishung5
rmccorm4
reviewed
Nov 4, 2023
rmccorm4
reviewed
Nov 4, 2023
rmccorm4
reviewed
Nov 4, 2023
dyastremsky
previously approved these changes
Nov 6, 2023
rmccorm4
previously approved these changes
Nov 6, 2023
krishung5
previously approved these changes
Nov 6, 2023
tanmayv25
previously approved these changes
Nov 8, 2023
tanmayv25
dismissed stale reviews from krishung5, rmccorm4, dyastremsky, and themself
via
70b4547
tanmayv25
force-pushed
the
oandreeva_autocomplete_vllm
branch
from
11d5c96
to
70b4547
Compare
tanmayv25
approved these changes
Nov 8, 2023
rmccorm4
approved these changes
Nov 8, 2023
|
@tanmayv25 should we merge this? |
|
I was unable to get a successful run using this change with olga's pipeline. I am in middle of investigating what is not working. |
|
Seems like there was some setup problem. Tested successfully with job id 73883232. |
tanmayv25
pushed a commit
that referenced
this pull request
Nov 9, 2023
tanmayv25
added a commit
that referenced
this pull request
Nov 9, 2023
|
Does autocomplete work? It doesn't seem to be called anywhere in the code besides the method being defined? |
|
The auto-complete feature is currently available only on triton's main branch that tracks development to a future release. The auto-complete functionality will be available with 23.11 release which is scheduled to be released by end of the month. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Moving default parameters from config to auto_complete function.
Also tests Python_backend's
set_model_transaction_policy