Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Add `keep_types` for filtering by token type #7120
Well this is related to token type, which is like a "tag" for the token that is produced by the analysis chain. Typically the lucene tokenizers provide tags if they recognize different types of tokens, but there is nothing limiting it to that. For example it could contain part-of-speech or whatever is useful.
This filter just filters by tag, it doesnt do tagging itself. It couldn't meet all the possible use cases for token types :)
So to tag by "pattern", we would just need a filter that does that. Its separate from what action to do with the actual tags...