New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improved sliding window - trim old context #12
Conversation
to prevent exceeding token limits
hack is to trim them, recognizing the LLM answer may not be ideal...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks great! added nitpicky suggestions, feel free to reject
Jetchat/app/src/main/java/com/example/compose/jetchat/data/CustomChatMessage.kt
Outdated
Show resolved
Hide resolved
Jetchat/app/src/main/java/com/example/compose/jetchat/data/CustomChatMessage.kt
Outdated
Show resolved
Hide resolved
Jetchat/app/src/main/java/com/example/compose/jetchat/data/Tokenizer.kt
Outdated
Show resolved
Hide resolved
Jetchat/app/src/main/java/com/example/compose/jetchat/data/CustomChatMessage.kt
Outdated
Show resolved
Hide resolved
Jetchat/app/src/main/java/com/example/compose/jetchat/data/CustomChatMessage.kt
Outdated
Show resolved
Hide resolved
Jetchat/app/src/main/java/com/example/compose/jetchat/data/CustomChatMessage.kt
Outdated
Show resolved
Hide resolved
Jetchat/app/src/main/java/com/example/compose/jetchat/data/CustomChatMessage.kt
Outdated
Show resolved
Hide resolved
Jetchat/app/src/main/java/com/example/compose/jetchat/data/CustomChatMessage.kt
Outdated
Show resolved
Hide resolved
// ONLY show date/time when embeddings are empty, as it triggers the SessionsByTime function (I THINK) | ||
// ALWAYS add the date and time to every prompt |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// ONLY show date/time when embeddings are empty, as it triggers the SessionsByTime function (I THINK) | |
// ALWAYS add the date and time to every prompt | |
// ONLY show date/time when embeddings are empty, as it triggers the SessionsByTime function |
hoping the "i think" has been resolved 😜
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤔 not sure!
role = completionMessage.role, | ||
content = completionMessage.content | ||
userContent = completionMessage.content | ||
?: "", // required to not be empty in this case |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this still true with the custom chat message class? or can we remove the null check now
Co-authored-by: Kristen Halper <khalper@microsoft.com>
Co-authored-by: Kristen Halper <khalper@microsoft.com>
Co-authored-by: Kristen Halper <khalper@microsoft.com>
and also trim 'current' context if it exceeds the limit (rare but possible bug)