Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing JDK-11 and HighlighterSearchIT::testPostingsHighlighter #1

Merged
merged 1 commit into from
Feb 11, 2022

Conversation

reta
Copy link

@reta reta commented Feb 8, 2022

Signed-off-by: Andriy Redko andriy.redko@aiven.io

Description

Fixing JDK-11 and HighlighterSearchIT::testPostingsHighlighter

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@reta
Copy link
Author

reta commented Feb 8, 2022

@nknize for HighlighterSearchIT::testWithNestedQuery (and very likely all other), there is more profound issue. The problem is related to org.opensearch.index.cache.bitset.BitsetFilterCache:

    private BitSet getAndLoadIfNotPresent(final Query query, final LeafReaderContext context) throws ExecutionException {
        final IndexReader.CacheHelper cacheHelper = context.reader().getCoreCacheHelper();
        if (cacheHelper == null) {
            throw new IllegalArgumentException("Reader " + context.reader() + " does not support caching");
        }
     ...
    }

The UnifiedHighlighter uses MemoryIndexOffsetStrategy with OverlaySingleDocTermsLeafReader as the LeafReaderContext, and this guy has no caching support:

public class OverlaySingleDocTermsLeafReader extends FilterLeafReader {
 ...
  @Override
  public CacheHelper getCoreCacheHelper() {
    return null;
  }

  @Override
  public CacheHelper getReaderCacheHelper() {
    return null;
  }
}

I don't have solution for now, looking into it (if you have please feel free to share). I found the solution but Opensearch Highlighters do not work properly with new HighlightFlag.WEIGHT_MATCHES, introduced by https://issues.apache.org/jira/browse/LUCENE-9431. This could be part of opensearch-project#2063

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Copy link
Owner

@nknize nknize left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice! thx for taking care of this one!

@nknize nknize merged this pull request into nknize:upgrade/lucene9 Feb 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants