-
Notifications
You must be signed in to change notification settings - Fork 25.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cache: models return input cache type #30716
Conversation
@zucchini-nlp this was the issue you pointed me to yesterday 🤗 |
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It overall a clean fix. I think we deprecated the legacy cache so at some point we can just return whatever was given!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great, thanks a lot for standardizing this!
@ArthurZucker |
What does this PR do?
Changes the
Cache
compatible models to return the same type of cache as the cache passed throughpast_key_values
. This was not happening in the case aDynamicCache
was explicitly passed as input, which was breaking a test :)