Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LUCENE-10060: Ensure DrillSidewaysQuery instances never get cached #261

Merged
merged 1 commit into from
Aug 26, 2021

Conversation

gsmiller
Copy link
Contributor

Description

We need to ensure DrillSidewaysQuery instances never get cached. The process of caching the results ignores deleted docs (null acceptDocs passed to the BulkScorer). This is an issue with DrillSidewaysScorer since it has the (intentional) side-effect of populating the "sideways" FacetsCollector instances, which means it will included deleted docs in the sideways counts.

Solution

DrillSidewaysQuery#isCacheable advertises as never cacheable by returning false. Note that I also removed the older hack in DrillSideways that's no longer needed with this solution.

Tests

Added a new test that repro's the caching bug with both a CollectorManager and Collector execution path. Confirmed the test now passes.

Checklist

Please review the following and check all that apply:

  • I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
  • I have created a Jira issue and added the issue ID to my pull request title.
  • I have given Lucene maintainers access to contribute to my PR branch. (optional but recommended)
  • I have developed this patch against the main branch.
  • I have run ./gradlew check.
  • I have added tests for my changes.

@@ -467,6 +467,8 @@ Bug Fixes

* LUCENE-10008: Respect ignoreCase in CommonGramsFilterFactory (Vigya Sharma)

* LUCENE-10060: Ensure DrillSidewaysQuery instances never get cached. (Greg Miller, Zachary Chen)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll backport to 8x

Copy link
Contributor

@zacharymorn zacharymorn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I like that this is a simple and yet formal fix to turn off caching for DSQ, instead of a hack. I've run this through the nightly tests against my other branch with collectorManager changes and everything passed, so we should be all good. Thanks @gsmiller again for the quick fix!

@gsmiller gsmiller merged commit dbf7e18 into apache:main Aug 26, 2021
@gsmiller
Copy link
Contributor Author

Thanks @zacharymorn for the quick review and double-checking the fix against your PR!

@gsmiller gsmiller deleted the LUCENE-10060-ds-fix branch August 26, 2021 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants