Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow word_delimiter_graph_filter to not adjust internal offsets #36699

Merged
merged 1 commit into from Dec 18, 2018

Conversation

romseygeek
Copy link
Contributor

This commit adds an adjust_offsets parameter to the word_delimiter_graph token filter, defaulting
to true. Most of the time you'd want sub-tokens emitted by this filter to have offsets that are
adjusted to their real position in the token stream; however, some token filters can change the length
or starting position of a token (eg trim) without changing their offset attributes, and this can lead to
word_delimiter_graph emitting illegal offsets. Setting adjust_offsets to false in these cases will
allow indexing again.

Fixes #34741, #33710

@romseygeek romseygeek added >enhancement :Search/Analysis How text is split into tokens v7.0.0 labels Dec 17, 2018
@romseygeek romseygeek self-assigned this Dec 17, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants