Autocomplete: Only ever complete a single line in single-line mode and reduce the output token limit #344
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…d reduce the output token limit
philipp-spiess
requested review from
beyang,
vovakulikov and
a team
and removed request for
beyang
vdavid
reviewed
Jul 21, 2023
| @@ -22,6 +22,7 @@ Starting from `0.2.0`, Cody is using `major.EVEN_NUMBER.patch` for release versi | |||
|
|
|||
| - Exclude context for chat input with only one word. [pull/279](https://github.com/sourcegraph/cody/pull/279) | |||
| - [Internal Only] `Custom Recipe`: Store `cody.json` file for user recipes within the `.vscode` folder located in the $HOME directory. [pull/279](https://github.com/sourcegraph/cody/pull/279) | |||
| - Various autocomplete improvements. [pull/344](https://github.com/sourcegraph/cody/pull/344) | |||
There was a problem hiding this comment.
Just curious: is there a reason we switched from the format #344 to pull/344?
There was a problem hiding this comment.
No idea, I think when we created the VS Code change log file we always had this format haha. I find it slightly more convenient because I can use multi-cursor to change both numbers at the same time when I commit before I create the PR. 🤷
vdavid
approved these changes
Jul 21, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Ensure that single-line mode completions are only ever returning a single line.
Since we no longer have two lines, I thought it's also okay to decrease the response token limit a bit. While there's still some variations, this seems to make it much more common for me when testing to get sub 800ms responses (compared to ~1sec before). I wasn't able to find an instance where 60 tokens weren't enough for single line completions yet.
Test plan
Updated test suite. Tested the latency locally.