Highlighting on a wildcard field name uses the same highlighter for all fields that match

[ccw-morris]

HighlightPhase.hitExecute(SearchContext context, HitContext hitContext)

Assume that the field object represents a wildcard field (i.e. field.field() returns "*")
Assume that the index contains two fields:
field_1 with "index_options":"freq"
field_2 with "index_options":"offsets"
Assume that a highlighter has not been explicitly specified (i.e. field.highlighterType()==null)

The object fieldNamesToHighlight will contain both of the above field names.
The first pass through the for (String fieldName : fieldNamesToHighlight) loop will evaluate the statement: if (field.highlighterType() == null){} to true.
The block will evaluate the index_options of the first field (from a HashSet, so essentially random choice) to determine the appropriate highlighter. The highlighter will then be set using field.highlighterType("postings").

Subsequent executions of the fieldNamesToHighlight for loop will evaluate the statement: if (field.highlighterType() == null){} to false, based on the settings from the first execution.

Result: The highlighter chosen for the (arbitrary) first field will be used for all subsequent fields. If the selected highlighter is not the plain highlighter then there is the potential for the code to throw an IllegalArgumentException due to a mismatch between the highlighter and the indexing options of the field.

The solution is to treat the user provided configuration (the field object) as immutable. For each iteration of the fieldNamesToHighlight loop, extract field.highlighterType() to a local variable. Test that local variable for null - and set to a selected highlighter as appropriate.

[ccw-morris]

For completeness, I can reproduce it with the following commands:

curl -XPUT 'http://localhost:9202/test' -d '{ "mappings": { "document": { "properties": { "field1": { "type":"string" }, "field2": { "type":"string", "index_options":"offsets" } } } } }'
curl -XPUT 'http://localhost:9202/test/document/1' -d '{ "field2":"This is the first field", "field1":"This is the second field" }'
curl -XPOST 'http://localhost:9202/_search' -d '{ "query": { "term": { "field2": "field" } }, "highlight": { "fields": { "field*": {} } } }'

However, please note that the bug relies on a specific ordering of a HashSet of String objects (field names). To reproduce it you may have to modify which field name has postings. Also, you may have to modify the regular expression that selects the fields: "field_" and "_" seems to produce different results. On production I can get the error with "*", but not with the test scripts above.

[javanna]

Great catch @ccw-morris , will fix this!