Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can snapkv compress kv in case different user questions are posed towards the same context? #18

Open
namespace-Pt opened this issue Jun 27, 2024 · 1 comment

Comments

@namespace-Pt
Copy link

Say there is a long document, then two users ask two different questions based on the document. These two questions are no way similar, targeting on different part of the document. In this case, can snapkv compress the context robustly?

@WendyH1108
Copy link
Collaborator

In our observation, we found out that the attention allocation depends on the nature of questions. SnapKV needs to compress individually for different turns/questions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants