Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOLR-17037 - Clone LTRScoringQuery on rewriting to preserve queryResultCache key #2022

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

softwaredoug
Copy link

@softwaredoug softwaredoug commented Oct 18, 2023

https://issues.apache.org/jira/browse/SOLR-17037

Description

Clone the LTRScoringQuery during rewriting so it benefits from the cache

Solution

Clone the LTRScoringQuery during rewriting so it benefits from the cache

Tests

See added test that repros, which is now fixed.

Checklist

Please review the following and check all that apply:

  • I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
  • I have created a Jira issue and added the issue ID to my pull request title.
  • I have given Solr maintainers access to contribute to my PR branch. (optional but recommended)
  • I have developed this patch against the main branch.
  • I have run ./gradlew check.
  • I have added tests for my changes.
  • I have added documentation for the Reference Guide

@softwaredoug softwaredoug changed the title SOLR-17073 - Clone LTRScoringQuery on rewriting to preserve queryResultCache key SOLR-17037 - Clone LTRScoringQuery on rewriting to preserve queryResultCache key Oct 18, 2023
Copy link
Member

@mkhludnev mkhludnev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test passed.

@softwaredoug
Copy link
Author

Looks like I broke some interleaving test, looking into that...

@softwaredoug
Copy link
Author

Fixed the interleaving tests :)

Copy link
Contributor

@alessandrobenedetti alessandrobenedetti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a minor comment, but the PR looks good to me in general!

.gitignore Outdated
@@ -43,3 +43,5 @@ __pycache__
# Ignore gradle wrapper
gradle/wrapper/gradle-wrapper.jar
.gradletasknamecache

.tool-versions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

anything to do with the fix? is this intended?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh you're right, this is my asdf environment. Will remove it.

@alessandrobenedetti
Copy link
Contributor

Thanks @softwaredoug for the contribution!
Pretty sure my colleague @aruggero worked on this as well, but can't remember if she wrote any blog on it or if she has any further consideration!

Copy link

This PR had no visible activity in the past 60 days, labeling it as stale. Any new activity will remove the stale label. To attract more reviewers, please tag someone or notify the dev@solr.apache.org mailing list. Thank you for your contribution!

@github-actions github-actions bot added the stale PR not updated in 60 days label Jan 26, 2024
@cpoerschke cpoerschke self-requested a review June 12, 2024 11:04

private boolean equalsTo(LTRInterleavingScoringQuery other) {
boolean superSame = (super.equalsTo(other) && other.equalsTo((LTRScoringQuery) this));
if (this.getPickedInterleavingDocIds() != null && other.getPickedInterleavingDocIds() != null) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor/style/subjective

Suggested change
if (this.getPickedInterleavingDocIds() != null && other.getPickedInterleavingDocIds() != null) {
if (this.pickedInterleavingDocIds != null && other.getPickedInterleavingDocIds() != null) {

Comment on lines +53 to +57
getScoringModel(), getExternalFeatureInfo(), getThreadModule());
cloned.setOriginalQuery(getOriginalQuery());
cloned.setFeatureLogger(getFeatureLogger());
cloned.setRequest(getRequest());
cloned.setPickedInterleavingDocIds(getPickedInterleavingDocIds());
Copy link
Contributor

@cpoerschke cpoerschke Jun 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wondering if the efi map and the pickedDocIds set should be cloned too?


String resp =
restTestHarness.adminQuery(
"/admin/metrics?group=core&&prefix=CACHE.searcher.queryResultCache");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"/admin/metrics?group=core&&prefix=CACHE.searcher.queryResultCache");
"/admin/metrics?group=core&prefix=CACHE.searcher.queryResultCache");

"/admin/metrics?group=core&&prefix=CACHE.searcher.queryResultCache");
String error =
JSONTestUtil.match(
resp, "/metrics/solr.core.collection1/CACHE.searcher.queryResultCache/hits==1");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps at the start of the test we could also test that the cache is empty hits==0 to start with?

query.add("fl", "*, score");
query.add("rows", "4");
query.add("fv", "true");
query.add("rq", "{!ltr model=6029760550880411648 reRankDocs=3}");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wondering how practical it would be to have test coverage for the interleaving case too.

@github-actions github-actions bot removed the stale PR not updated in 60 days label Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants