Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Search with k-NN filters #1814

Merged
merged 50 commits into from
Nov 11, 2022
Merged

Search with k-NN filters #1814

merged 50 commits into from
Nov 11, 2022

Conversation

alicejw1
Copy link
Contributor

@alicejw1 alicejw1 commented Nov 4, 2022 •
edited
Loading

Description

Document how to use filters with k-nn searches.

Note to reviewers: please review the new content in filter-search-knn.md only.
The other topics within the k-NN plugin only have nav_order number updates.

Issues Resolved

issue 923

Review PDF

To view the staged document as it will render on the site, look at this PDF:
Search with k-NN filters.pdf

Checklist

  • [ x] By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@alicejw1
new file for knn filter searches
1fab580 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
for knn filter queries
bf80984 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
more details and include graphic
c9ec53c 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
add graph of filtered doc set
0388530 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1 alicejw1 added 2 - In progress Issue/PR: The issue or PR is in progress. backport 2.0 PR: Backport label for v2.0.x backport 2.1 PR: Backport label for 2.1 backport 2.2 PR: Backport label for 2.2 backport 2.3 PR: Backport label for 2.3 labels Nov 4, 2022
@alicejw1 alicejw1 self-assigned this Nov 4, 2022
@alicejw1
add release label
5c0d566 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
filters are defined by Query DSL
6e70c72 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1 alicejw1 added this to the v2.4 milestone Nov 4, 2022
@alicejw1
more details about how the algorithm works and how to specify lucene …
285d198 
…as the search engine

Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
more refining sentences
6fafe53 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
for response samples
0f67a2a 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
reorg heading levels
d930ed5 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
more rewrites for clarity
9579edd 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
to add the complex filter query
2ed4b8a 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
update response for complex query
3b230b0 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
for typo
b642e2d 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1 alicejw1 changed the title Knn filters Search with k-NN filters Nov 5, 2022
@alicejw1 alicejw1 added 3 - Tech review PR: Tech review in progress and removed 2 - In progress Issue/PR: The issue or PR is in progress. labels Nov 5, 2022
@alicejw1
for rewrites to overview
da3c6f3 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
to add better request/response for the complex filter example
bd6bb3e 
Signed-off-by: alicejw <alicejw@amazon.com>
@martin-gaievski
Copy link
Member

Please use following workflow diagram to illustrate how the algorithm works:

Lucene filtering workflow

@alicejw1
for eng review update
79260e4 
Signed-off-by: alicejw <alicejw@amazon.com>
cwillum
cwillum approved these changes Nov 8, 2022
View reviewed changes
Copy link
Contributor

@cwillum cwillum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few comments. But LGTM

@alicejw1
for doc review feedback updates
59c4a29 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
minor grammar change
5359d05 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1 alicejw1 linked an issue Nov 8, 2022 that may be closed by this pull request
[DOC] Add efficient filtering capability for Lucene knn #923
Closed
4 tasks
@alicejw1 alicejw1 added v2.4.0 'Issues and PRs related to version v2.4.0' and removed backport 2.0 PR: Backport label for v2.0.x backport 2.1 PR: Backport label for 2.1 backport 2.2 PR: Backport label for 2.2 backport 2.3 PR: Backport label for 2.3 labels Nov 8, 2022
@alicejw1
removed figure and table titles, per AWS Style Guide
38f60be 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
remove table title per style guide
7bcf88d 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
update nav orders for all pages to give space for new topics in multi…
624ea21 
…ples of 5, and add links to other knn topics where appropriate

Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
small rewrite
fc436d4 
Signed-off-by: alicejw <alicejw@amazon.com>
vagimeli
vagimeli approved these changes Nov 10, 2022
View reviewed changes
Copy link
Collaborator

@vagimeli vagimeli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @alicejw-aws, I completed my review and left some comments to address before editorial review. :)

_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
@alicejw1
for second doc review comments
fcef31e 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
Copy link
Contributor Author

Thank you @vagimeli for the doc review!
I updated the files and prepared this PDF to show latest updates:
Search with k-NN filters11.10.2022.pdf

@alicejw1 alicejw1 added 5 - Editorial review PR: Editorial review in progress and removed 4 - Doc review PR: Doc review in progress labels Nov 10, 2022
natebower
natebower reviewed Nov 11, 2022
View reviewed changes
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alicejw-aws Please see my comment and changes and let me know if you have any questions. Thanks!

_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
_search-plugins/knn/filter-search-knn.md Outdated Show resolved Hide resolved
alicejw1 and others added 5 commits November 11, 2022 10:06
@alicejw1 @natebower
Update _search-plugins/knn/filter-search-knn.md
563807d 
Co-authored-by: Nate Bower <nbower@amazon.com>
@alicejw1 @natebower
Update _search-plugins/knn/filter-search-knn.md
70d5648 
Co-authored-by: Nate Bower <nbower@amazon.com>
@alicejw1
for editorial review updates
3d2a098 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
for merge changes
8b8c509 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
for editorial review updates
7a7ed44 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1 alicejw1 removed the 5 - Editorial review PR: Editorial review in progress label Nov 11, 2022
@alicejw1
fix cross-ref link
bfa1f54 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1
fix undone commit suggestions
2875939 
Signed-off-by: alicejw <alicejw@amazon.com>
@alicejw1 alicejw1 merged commit 253ae34 into opensearch-project:main Nov 11, 2022
@jmazanec15 jmazanec15 mentioned this pull request Dec 21, 2022
Closed
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@natebower natebower natebower left review comments

@cwillum cwillum cwillum approved these changes

@vagimeli vagimeli vagimeli approved these changes

@martin-gaievski martin-gaievski martin-gaievski approved these changes

Assignees

@alicejw1 alicejw1

Labels
v2.4.0 'Issues and PRs related to version v2.4.0'
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[DOC] Add efficient filtering capability for Lucene knn
5 participants
@alicejw1 @martin-gaievski @natebower @vagimeli @cwillum