Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Highlighting: Highlighter still fails if broken analysis chains are used with fast vector highlighter #3006

Closed
s1monw opened this Issue May 7, 2013 · 0 comments

Comments

Projects
None yet
1 participant
@s1monw
Copy link
Contributor

s1monw commented May 7, 2013

TokenFilters like word_delimiter_filter might produce broken term_vectors and mess up highlighting. We still fail with StringIndexOutOfBoundsException which is no good. Even if those filters are broken we should try at least best effort to not fail with a SIOOBException

@ghost ghost assigned s1monw May 7, 2013

@s1monw s1monw closed this in e1b66b3 May 7, 2013

s1monw added a commit that referenced this issue May 7, 2013

Don't fail hard if broken analysis is used.
Today an analysis chain with broken tokenfilters or tokenizers like
WordDelimiterFilter might produce somewhat broken term vectors that cause
`StringIndexOutOfBoundsExceptions` if FastVectorHighlighter is used
since the positions / offsets contract is violated and offsets of highlight
tokens are not increasing but decreasing even if their positions are increasing.

Yet, if we detect such a situation we can resort the tokens which might cause
somewhat odd highlights but doesn't fail hard with a StringIndexOOBException.

Closes #3006

mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015

Don't fail hard if broken analysis is used.
Today an analysis chain with broken tokenfilters or tokenizers like
WordDelimiterFilter might produce somewhat broken term vectors that cause
`StringIndexOutOfBoundsExceptions` if FastVectorHighlighter is used
since the positions / offsets contract is violated and offsets of highlight
tokens are not increasing but decreasing even if their positions are increasing.

Yet, if we detect such a situation we can resort the tokens which might cause
somewhat odd highlights but doesn't fail hard with a StringIndexOOBException.

Closes elastic#3006
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.