Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

Refactor KNNVectorFieldMapper #240

Merged

Conversation

jmazanec15
Copy link
Member

Issue #, if available:
#239

Description of changes:
PR refactors KNNVectorFieldMapper in a variety of ways.

First, it refactors KNNVectorFieldMapper to utilize ParametrizedFieldMapper. This way, we do not need to parse the field parameters manually.

Second, it refactors KNNVectorFieldMapper to build a new fieldType in parseCreateField, as opposed to updating Defaults.FIELD_TYPE inside the builder logic. This will prevent race condition bugs seen in #239.

Third, it refactors the ordering of the logic inside of KNNVectorFieldMapper to follow the convention of other Elasticsearch field mappers.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@jmazanec15 jmazanec15 added the Bug Fixes Change that fixes a bug label Oct 13, 2020
@codecov
Copy link

codecov bot commented Oct 13, 2020

Codecov Report

❗ No coverage uploaded for pull request base (master@76bdfaa). Click here to learn what that means.
The diff coverage is 87.83%.

Impacted file tree graph

@@            Coverage Diff            @@
##             master     #240   +/-   ##
=========================================
  Coverage          ?   78.65%           
  Complexity        ?      336           
=========================================
  Files             ?       53           
  Lines             ?     1326           
  Branches          ?      121           
=========================================
  Hits              ?     1043           
  Misses            ?      234           
  Partials          ?       49           
Impacted Files Coverage Δ Complexity Δ
...oforelasticsearch/knn/index/util/KNNConstants.java 0.00% <ø> (ø) 0.00 <0.00> (?)
...relasticsearch/knn/index/KNNVectorFieldMapper.java 78.62% <87.83%> (ø) 15.00 <5.00> (?)
...ndistroforelasticsearch/knn/index/KNNSettings.java 83.47% <0.00%> (ø) 27.00% <0.00%> (?%)
...istroforelasticsearch/knn/index/KNNIndexCache.java 92.30% <0.00%> (ø) 35.00% <0.00%> (?%)
...roforelasticsearch/knn/plugin/KNNCodecService.java 66.66% <0.00%> (ø) 2.00% <0.00%> (?%)
...rch/knn/plugin/transport/KNNStatsNodeResponse.java 50.00% <0.00%> (ø) 4.00% <0.00%> (?%)
...csearch/knn/plugin/transport/KNNStatsResponse.java 64.00% <0.00%> (ø) 4.00% <0.00%> (?%)
...icsearch/knn/plugin/rest/RestKNNWarmupHandler.java 100.00% <0.00%> (ø) 10.00% <0.00%> (?%)
...ticsearch/knn/plugin/rest/RestKNNStatsHandler.java 87.87% <0.00%> (ø) 14.00% <0.00%> (?%)
... and 45 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 76bdfaa...ef548a5. Read the comment docs.

Comment on lines 316 to 318
fieldType.putAttribute(KNNConstants.SPACE_TYPE, spaceType);
fieldType.putAttribute(KNNConstants.HNSW_ALGO_M, String.valueOf(m));
fieldType.putAttribute(KNNConstants.HNSW_ALGO_EF_CONSTRUCTION, String.valueOf(efConstruction));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about reading these algo params from the indexSettings here? Idea is to lazy load these settings to ensure the settings are available to read from IndexSettings.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would cause index settings to be read everytime a document is indexed. I refactored to build fieldType to the KNNVectorFieldMapper constructor and read the settings from BuilderContext during Builder.build()

Comment on lines 134 to 136
if (spaceType == null) {
spaceType = getSpaceType(context.indexSettings());
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we storing state in Builder class? Why not get a fresh copy every time by reading from indexSettings?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. Updated.

Copy link
Member

@vamshin vamshin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for refactoring the KNNVectorFieldMapper and keeping it consistent with other Elasticsearch's Mapper. This will help in smooth migration for future releases.

@jmazanec15 jmazanec15 merged commit 6d6a961 into opendistro-for-elasticsearch:master Oct 13, 2020
jmazanec15 added a commit to jmazanec15/k-NN that referenced this pull request Oct 14, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Bug Fixes Change that fixes a bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants