Similarity should accept dynamic settings when possible #20339
Conversation
|
@s1monw I am very late on this issue, I was reluctant to do it since this solution has some downsides (listed in the description of the PR) but it should be enough for now especially with the move to BM25 as the default similarity (users may want to tweak the parameters k1 and b). |
…ilarities on an open index. It does not allow to change the type of an existing similarity, to change the similarity of a field or to add a new similarity on an open index. Options are updatable only if they don't change the way the similarity encodes the norm on disk. The non-updatable options can't be changed if the index is opened nor closed. This change adds validation to the similarity settings and throws an exception with a detailed message if a setting is unknown or not parseable. Since it uses affix settings to register non-concrete setting, similarity settings that share the same suffix have been renamed: `lambda` has been renamed to `collection_model` for the IBSimilarity (clashes with the lambda setting in LM Jelinek Mercer similarity). Relates #6727
| return new NormalizationH2(c); | ||
| } else if ("h3".equals(normalization)) { | ||
| float c = settings.getAsFloat("normalization.h3.c", 800f); | ||
| if (innerSettings.get("h3.c") == null) { |
|
@jimferenczi I took a look and left one concerning comment here #20339 (comment) look good in general. thanks so much for doing this. |
|
Do you have to reindex existing docs for the new parameters to take effect? |
|
Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually? |
|
Closing since the uber issue is still open and we need a different approach. |
clintongormley
added
:Search/Search
This is a spin off for #19046
This change allows to dynamically change options for any existing similarities on an open index.
It uses the affix setting to register all the similarity settings.
It does not allow to change the type of an existing similarity, to change the similarity of a field or to add a new similarity on an open index.
Options are updatable only if they don't change the way the similarity encodes the norm on disk.
The non-updatable options can't be changed if the index is opened nor closed.
This change adds validation to the similarity settings and throws an exception with a detailed message if a setting is unknown or not parseable.
Since it uses affix settings to register non-concrete setting, similarity settings that share the same suffix have been renamed:
lambdahas been renamed tocollection_modelfor the IBSimilarity (clashes with the lambda setting in LM Jelinek Mercer similarity).The downside of using the affix setting is that each setting is registered as a global setting for any similarity type even though some settings are available for a certain similarity type only. Another solution could be to change the way the settings are registered, so instead of:
... we could do:
This would solve the problem of the global settings but it would also break the settings framework which does not accept settings without value:
Relates #6727