Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
RE: #3523 & #2636
This PR adds character limit tracking for chat models in the SimpleChatModel class. It introduces a new property
maxChars
that stores the maximum number of characters allowed for the selected chat model.The
charsLeft
property is also added to calculate the remaining characters available based on the current chat messages without context.These changes allow the chat interface to track the character limit for the selected chat model and ensure that the at-mention context items are showing the correct warning for large files based on the latest available token limit instead of a standardized one that doesn't represent the latest limit.
Test plan
Try
@Simple
and look forSimpleChatModel.ts
in the cody repository following the examples belowBefore
@ a file that is showing
File too large
warning, but when you select it, it will still be included in your message because the warning size was using the incorrect limit number.You can see Cody is able to reply to you because the file was still included, meaning the warning was incorrectly displayed:
After
Type
@Simple
and look forSimpleChatModel.ts
again, you can seeSimpleChatModel.ts
doesn't have the warning anymore, and selecting it still works:However, keep asking Cody question so that the context window will become smaller, and when the messages is larger than the file, then the warning will show up correctly for the file:
Limitation
The large @-mention files can still be added by the user, which has been addressed in a separate PR.
Currently cannot dynamically calculate the token limit change for a resubmitted query, meaning users won't see the file size warning if they have selected an at-mention item that used all the limit, but they can still add more at-mention items. This will also be addressed in a future PR.
Follow-up