Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fast Vector Highlighter does no work with nested field as target #80895

Closed
pbvahlst opened this issue Nov 21, 2021 · 2 comments · Fixed by #85815
Closed

Fast Vector Highlighter does no work with nested field as target #80895

pbvahlst opened this issue Nov 21, 2021 · 2 comments · Fixed by #85815
Labels
>bug :Search/Highlighting How a query matched a document Team:Search Meta label for search team

Comments

@pbvahlst
Copy link

pbvahlst commented Nov 21, 2021

Elasticsearch version (bin/elasticsearch --version): 7.15.2

Plugins installed: []

JVM version (java -version): the one included in ES

OS version (uname -a if on a Unix-like system): Windows 10

Description of the problem including expected versus actual behavior:
When using a nested field as target for Fast Vector Highligter no result is produced. (This works fine with the Unified Highlighter).
The problem seems to be that fvh tries to read the target field literally from the source and as this would not be present in the source
it assumes that the field is absent. If the nested field has store: true set everything works as expected as it then reads the stored field instead of visiting the source-object.

The reason for using a nested field as target is that the root-field is a date-field and we then have multiple nested fields with different versions of the date-string with with_positions_offsets enabled for term_vector.

Steps to reproduce:
make a search like the following with and without store:true enabled in the mapping of the the nested target field (date.text).

{
  "query": {
    "query_string": {
      "query": "2000-10-10",
      "fields": [ "date.text" ],
      "quote_field_suffix": ".exact"
    }
  },
  "highlight": {
    "fields": {
      "date.text": {
        "matched_fields": [ "date.text", "date.text.exact" ],
        "type": "fvh"
      }
    }
  }
}
@pbvahlst pbvahlst added >bug needs:triage Requires assignment of a team area label labels Nov 21, 2021
@cbuescher cbuescher added the :Search/Highlighting How a query matched a document label Nov 22, 2021
@elasticmachine elasticmachine added the Team:Search Meta label for search team label Nov 22, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@cbuescher cbuescher removed the needs:triage Requires assignment of a team area label label Nov 22, 2021
@Daantie
Copy link

Daantie commented Feb 21, 2022

I'm running into this problem when trying to upgrade Elasticsearch 7.11 to 7.16. A lot of highlighting stuff in our project breaks due to this. The problem is not only for nested fields, or any particular query (like in #82458), but to subfields in general. I have done some investigation and it started in 7.14 and is still the same in 8.0.0.

I've reproduced it with these steps:

PUT simple-highlight-test
{
  "mappings": {
    "properties": {
      "text": {
        "type": "text",
        "analyzer": "standard",
        "term_vector": "with_positions_offsets",
        "fields": {
          "subfield": {
            "type": "text",
            "term_vector": "with_positions_offsets",
            "analyzer": "standard"
          }
        }
      }
    }
  }
}

POST simple-highlight-test/_doc
{
  "text": "some random text"
}

GET simple-highlight-test/_search
{
  "from": 0,
  "size": 10,
  "query": {
    "match": {
      "text.subfield": {
        "query": "random"
      }
    }
  },
  "highlight": {
    "fields": [
      {
        "text.subfield": {
          "type": "fvh"
        }
      }
    ]
  }
}

Like mentioned by OP adding "store": true to the subfield will make it work, but this obviously results in larger index size. Plain and unified highlighters work fine without store true though.

@cbuescher do you know if this is getting any attention? Or will be getting any attention any time soon?

romseygeek added a commit that referenced this issue Apr 12, 2022
…85815)

FVH was relying on `SourceLookup.extractRawValues()` to load its data, but this no
longer works for multifields. It should instead use value fetchers which will correctly
locate the input for multifields and/or copy fields.

Fixes #84690
Fixes #82458
Fixes #80895
Fixes #75011
romseygeek added a commit to romseygeek/elasticsearch that referenced this issue Jun 7, 2022
…lastic#85815)

FVH was relying on `SourceLookup.extractRawValues()` to load its data, but this no
longer works for multifields. It should instead use value fetchers which will correctly
locate the input for multifields and/or copy fields.

Fixes elastic#84690
Fixes elastic#82458
Fixes elastic#80895
Fixes elastic#75011
romseygeek added a commit that referenced this issue Jun 8, 2022
…87445)

FVH was relying on `SourceLookup.extractRawValues()` to load its data, but this no
longer works for multifields. It should instead use value fetchers which will correctly
locate the input for multifields and/or copy fields.

Fixes #84690
Fixes #82458
Fixes #80895
Fixes #75011
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Search/Highlighting How a query matched a document Team:Search Meta label for search team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants