elasticsearch highlight lose character when index_options=offsets

[tenlee2012]

Elasticsearch version (bin/elasticsearch --version): 7.7.1

Plugins installed: []

JVM version (java -version): openjdk version "11.0.7" 2020-04-14 LTS

OS version (uname -a if on a Unix-like system): GNU/Linux

Description of the problem including expected versus actual behavior:
set properties index_options=offsets,
doc title is can work?shit, serach result highlight is "<em>shit</em>".
If the search match the last word, and the last word is preceded by a tag symbol, then the highlighting will wrong when set properties index_options=offsets.

Steps to reproduce:

1. create index

PUT demo_1
{
  "mappings": {
    "properties": {
      "title": {
        "type": "text",
        "analyzer": "standard",
        "index_options": "offsets"
      }
    }
  }
}

2. put data

PUT demo_1/_doc/1
{
    "title": "can work?shit"
}

3. search

POST demo_1/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "multi_match": {
            "query": "shit"
          }
        }
      ]
    }
  },
  "highlight": {
    "fields": {
      "title": {}
    }
  }
}

search result is blow:

{
    "took":2,
    "timed_out":false,
    "_shards":{
        "total":1,
        "successful":1,
        "skipped":0,
        "failed":0
    },
    "hits":{
        "total":{
            "value":1,
            "relation":"eq"
        },
        "max_score":0.2876821,
        "hits":[
            {
                "_index":"demo_1",
                "_type":"_doc",
                "_id":"1",
                "_score":0.2876821,
                "_source":{
                    "title":"can work?shit"
                },
                "highlight":{
                    "title":[
                        "<em>shit</em>"
                    ]
                }
            }
        ]
    }
}

4. if set hightlight type=plain, it works.

POST demo_1/search
{
  "query": {
    "bool": {
      "must": [
        {
          "multi_match": {
            "query": "shit"
          }
        }
      ]
    }
  },
  "highlight": {
    "fields": {
      "title": {
        "type": "plain"
      }
    }
  }
}

result is blow:

{
    "took":2,
    "timed_out":false,
    "_shards":{
        "total":1,
        "successful":1,
        "skipped":0,
        "failed":0
    },
    "hits":{
        "total":{
            "value":1,
            "relation":"eq"
        },
        "max_score":0.2876821,
        "hits":[
            {
                "_index":"demo_1",
                "_type":"_doc",
                "_id":"1",
                "_score":0.2876821,
                "_source":{
                    "title":"can work?shit"
                },
                "highlight":{
                    "title":[
                      "can work?<em>shit</em>"
                    ]
                }
            }
        ]
    }
}

[elasticmachine]

Pinging @elastic/es-search (:Search/Highlighting)

[elasticsearchmachine]

Pinging @elastic/es-search (Team:Search)

[DHRUV6029]

Hi can i work on this issue

[mayya-sharipova]

@DHRUV6029 So far there is no PR submitted for this issue. You are welcome to submit yours.

[arshPratap]

Hi @mayya-sharipova .. I would like to work on this issue. Can this issue be assigned to me?

[mayya-sharipova]

@arshPratap You are welcome to submit a PR, we don't assign external contributors to an issue.