Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add vector searching from lucene 9.0 using KnnVectorQuery and KnnVectorField #70

Merged
merged 1 commit into from May 23, 2022

Conversation

mdavis95
Copy link
Contributor

Replaces old, unused, and slow implementation using Superbit projects and minimum should match queries with cosine similarity

Examples searches:

Alone

search = new Search("vectorTestIndex");
search.addQuery(new VectorTopNQuery(new float[] { 1.0f, 0.0f, 0.0f, 0.0f }, 3, "v"));

Post Filtering (After the topN)

search = new Search("vectorTestIndex");
search.addQuery(new VectorTopNQuery(new float[] { 1.0f, 0.0f, 0.0f, 0.0f }, 3, "v"));
search.addQuery(new FilterQuery("red").addQueryField("description"));

Pre Filtering

search = new Search("vectorTestIndex");
VectorTopNQuery vectorTopNQuery = new VectorTopNQuery(new float[] { 1.0f, 0.0f, 0.0f, 0.0f }, 3, "v");
vectorTopNQuery.addPreFilterQuery(new FilterQuery("blue").addQueryField("description"));
search.addQuery(vectorTopNQuery);

Creating Index

ClientIndexConfig indexConfig = new ClientIndexConfig();
indexConfig.addDefaultSearchField("description");
indexConfig.addFieldConfig(FieldConfigBuilder.createString("description").indexAs(DefaultAnalyzers.STANDARD).sort());
indexConfig.addFieldConfig(FieldConfigBuilder.createUnitVector("v").index()); // use createVector if not unit normalized
indexConfig.setIndexName("vectorTestIndex");
zuliaWorkPool.createIndex(indexConfig);

Indexing

float[] vector = { 0, 0, 0.70710678f, 0.70710678f } // needs to be unit normalized if type is UNIT_VECTOR
Document mongoDocument = new Document();
mongoDocument.put("description", "great description");
mongoDocument.put("v", vector != null ? Floats.asList(vector) : null);
Store s = new Store("someId", "vectorTestIndex");

@mdavis95 mdavis95 merged commit d1cf3b5 into main May 23, 2022
@mdavis95 mdavis95 deleted the vector_search branch May 23, 2022 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant