-
-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[API/feature?] make fids caching optional #32315
Conversation
@nyalldawson @m-kuhn I'm not sure if this is a feature or a preventive bugfix but this would be nice to have it in 3.10 in case some users have to do counting on big dataset. |
4ac89e2
to
b8ee3f2
Compare
@m-kuhn a quick review would be appreciated. |
Backport would be preferable, especially early in this cycle. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me, just some minor changes to wording.
And +1 for backporting.
b8ee3f2
to
cc8c2b6
Compare
@m-kuhn Thanks for the review/suggestions. I have also made the same changes to the counter in order to have constant docs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot
@m-kuhn Did I need to do anything for the backporting? I assume a manual backport is only needed when automatic backporting fails. |
Description
This is a follow up PR to #31061 that enabled caching of fids per symbol during the counting procedure.
This PR aims to make fids caching optional, as it is not necessary to have the have the fids in most cases, especially until #30297 is merged.
This should prevent for useless memory usage.
If this is merged after 3.10 is published, backporting is highly recommended to prevent extra memory usage.
Checklist
Fixes #11111
at the bottom of the commit messagescripts/prepare-commit.sh
script before each commit