Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

introduce support in KnnVectorQuery for getters #12029

Merged

Conversation

alessandrobenedetti
Copy link
Contributor

Description

Knn Queries are locked currently, it would be beneficial for applications using them to have access to getters and setters.
An example is how filter queries are managed in Apache Solr:
the processing of pre-filters and post-filters could benefit from opening up the access to such variables.
Especially the pre-filter support introduced in Solr 9.1 could get great benefits from being able to set the filter, after the query has been parsed.
See:
apache/solr#1245

If there are no objections I would simply remove the final and add the getters/setters.
I may consider alternative if there's some valid concern in doing that.

@alessandrobenedetti alessandrobenedetti changed the title KnnVectorQuery introduce getters/setters introduce support in KnnVectorQuery for getters/setters Dec 21, 2022
@rmuir
Copy link
Member

rmuir commented Dec 21, 2022

queries should be immutable, see Query.java documentation. Hence I don't think we should add getter/setters or remove final keywords.

@alessandrobenedetti
Copy link
Contributor Author

Thanks @rmuir for the prompt answer, I took a look at Query.java, and couldn't find any particular reason for the Knn query to be immutable(aside from historical reasons?).
If you can elaborate I am happy to consider alternatives (I could bring back final and just add getters if any better).

Knn query has a sub-query that uses as an internal filter, and having access to that can make the Solr side of filter/post-filters processing much easier and performant.

@rmuir
Copy link
Member

rmuir commented Dec 21, 2022

All queries need to be immutable for the query cache to work correctly and consistently.

You are right, the docs need help here. Unfortunately docs on immutability were attached to deprecated methods and disappeared! https://github.com/apache/lucene-solr/blob/releases/lucene-solr/5.5.5/lucene/core/src/java/org/apache/lucene/search/Query.java#L95-L107

@rmuir
Copy link
Member

rmuir commented Dec 21, 2022

the getTarget() getters are unsafe as they return mutable things (float[], BytesRef)

@alessandrobenedetti
Copy link
Contributor Author

Thanks, @rmuir for the explanation, it seems reasonable and definitely, I won't argue with that.
For the sake of my needs, just the getters would be fine(and I'll clone the query, changing the filter).
I checked around and getters seem to be tolerated (see org.apache.lucene.search.FuzzyQuery).
Any problem with this?
I updated the Pull Request

@alessandrobenedetti
Copy link
Contributor Author

alessandrobenedetti commented Dec 21, 2022

@rmuir you are right again, I gave it another try, using copies(ArraysUtils for arrays and clone for byteref), this should be safe.
If there are still concerns I may move to some Builder/Constructors approaches, to be able to build a KnnVectorQuery starting from an old one.

@rmuir
Copy link
Member

rmuir commented Dec 21, 2022

BytesRef.clone won't do what we want here. it is a shallow clone.

@dsmiley
Copy link
Contributor

dsmiley commented Dec 21, 2022

Completely agree with Robert -- Query subclasses ought to be immutable and the javadocs ought to be updated to scream this. Nasty/hard bugs happen when a Query is mutable.

@alessandrobenedetti
Copy link
Contributor Author

Thanks again @rmuir, moved to a deep copy!

@rmuir
Copy link
Member

rmuir commented Dec 22, 2022

needs javadocs and tests, otherwise it looks like these getters are unnecessary and should be factored out.

@msokolov
Copy link
Contributor

Hi, you had asked for my review, but I think you got lots of helpful feedback already. Anyway this looks fine to me. I added one comment about a test, we could tighten it up a bit.

@alessandrobenedetti
Copy link
Contributor Author

waiting for the checks and then I'll merge tonight!

@risdenk
Copy link
Contributor

risdenk commented Jan 20, 2023

Re: immutable Query in Solr - See https://issues.apache.org/jira/browse/SOLR-16509 and apache/solr#1146

@alessandrobenedetti alessandrobenedetti linked an issue Jan 21, 2023 that may be closed by this pull request
@alessandrobenedetti alessandrobenedetti merged commit 77eca4b into apache:main Jan 23, 2023
@alessandrobenedetti alessandrobenedetti changed the title introduce support in KnnVectorQuery for getters/setters introduce support in KnnVectorQuery for getters Jan 23, 2023
asfgit pushed a commit that referenced this pull request Jan 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

introduce support in KnnVectorQuery for getters
5 participants