Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow keyword_marker filter to be updateable #65355

Closed
spinscale opened this issue Nov 23, 2020 · 2 comments · Fixed by #65457
Closed

Allow keyword_marker filter to be updateable #65355

spinscale opened this issue Nov 23, 2020 · 2 comments · Fixed by #65457
Assignees
Labels
>enhancement :Search/Analysis How text is split into tokens Team:Search Meta label for search team

Comments

@spinscale
Copy link
Contributor

The keyword_marker filter allows for a keyboard_path configuration setting, but does not seem to support the updateable flag and thus cannot be reloaded via the reload search analyzers API.

To reproduce create a config/analysis/example_word_list.txt file containing

experience
sample

Then run the following in dev-tools

DELETE test_index

PUT /test_index
{
  "settings": {
    "index": {
      "analysis": {
        "analyzer": {
          "search_synonyms": {
            "tokenizer": "whitespace",
            "filter": [
              "my_custom_keyword_marker_filter",
              "porter_stem"
            ]
          }
        },
        "filter": {
          "my_custom_keyword_marker_filter": {
            "type": "keyword_marker",
            "keywords_path": "analysis/example_word_list.txt",
            "updateable": true
          }
        }
      }
    }
  }
}

# cat example_word_list.txt
# experience 
# sample

# sample and experience are untouched
GET test_index/_analyze
{
"text": "this is a experience sample",
"analyzer": "search_synonyms"
}

# Now remove the `sample` line from the file and reload
POST /test_index/_reload_search_analyzers

# retry, sample should be `sampl` now if updateable, but remains sample
GET test_index/_analyze
{
"text": "this is a experience sample",
"analyzer": "search_synonyms"
}

Another issue with this is, that anything can be configured in my_custom_keyword_marker_filter and thus setting/believing the updateable parameters works can be misleading.

@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@cbuescher cbuescher removed the needs:triage Requires assignment of a team area label label Nov 24, 2020
@cbuescher cbuescher self-assigned this Nov 24, 2020
@cbuescher
Copy link
Member

I can see the benefit of being able to update the keyword_marker list used for search analyzers without node shutdown.
In many ways this resource file is comparable to a synonym list. However it would be the first non-synonym filter that would be
updateable so I think this request needs some more inspection and discussion. As far as implementation goes it should be sufficient
to wire the "updateable" setting in the same way as we do for synonym filters.

cbuescher pushed a commit that referenced this issue Feb 25, 2021
Currently we don't allow `keyword_marker` filter file resources to be reloaded via the
`_reload_search_analyzers` API. It would make sense to allow reloading this when the
file content has changed to allow e.g. for updating stemmer exeption rules at search time
without having to close and re-open the index in question. This change adds the updateable
flag to this token filter in the same way it is used for synonym filters. Analyzers containing
updateable keyword_marker filters would not be allowed to be used at index time but at
search time only, similar to what we allow for synonym filters.

Closes #65355
cbuescher pushed a commit that referenced this issue Feb 25, 2021
Currently we don't allow `keyword_marker` filter file resources to be reloaded via the
`_reload_search_analyzers` API. It would make sense to allow reloading this when the
file content has changed to allow e.g. for updating stemmer exeption rules at search time
without having to close and re-open the index in question. This change adds the updateable
flag to this token filter in the same way it is used for synonym filters. Analyzers containing
updateable keyword_marker filters would not be allowed to be used at index time but at
search time only, similar to what we allow for synonym filters.

Closes #65355
easyice pushed a commit to easyice/elasticsearch that referenced this issue Mar 25, 2021
Currently we don't allow `keyword_marker` filter file resources to be reloaded via the
`_reload_search_analyzers` API. It would make sense to allow reloading this when the
file content has changed to allow e.g. for updating stemmer exeption rules at search time
without having to close and re-open the index in question. This change adds the updateable
flag to this token filter in the same way it is used for synonym filters. Analyzers containing
updateable keyword_marker filters would not be allowed to be used at index time but at
search time only, similar to what we allow for synonym filters.

Closes elastic#65355
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement :Search/Analysis How text is split into tokens Team:Search Meta label for search team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants