Search phrase suggester collate query (ElasticSearch 2.x)

[harvzor]

Hi, I have been looking into using the collate option of a phrase suggestion to filter out suggestions that would not produce search query results. There is some documentation here: https://www.elastic.co/guide/en/elasticsearch/client/net-api/2.x/suggest-usage.html

I see that the query currently needs to be inline:

 .Collate(c => c
     .Query(q => q
         .Inline("{ \"match\": { \"{{field_name}}\": \"{{suggestion}}\" }}")
         .Params(p => p.Add("field_name", "title"))
     )
     .Prune()
)

However, this means that I lose any Intellisense, and if I change my mappings in my index, no errors will be thrown until query time.

I can see that there is another another option instead of the inline search descriptor. It appears I can use a previously indexed query by using the .Indexed() option:

 .Collate(c => c
     .Query(q => q
         .Indexed(/* what goes here? */)
     )
     .Prune()
)

Though there doesn't seem to be any documentation on this and no tests either. Can someone point me in the right direction?

Thanks in advance.

[gmarz]

I see that the query currently needs to be inline ... However, this means that I lose any Intellisense

The reason for this is that Query() is not just any QueryDescriptor as it is most everywhere else. It's actually a template query which is based on the Mustache templating system. Inline being the query template itself. We could perhaps type Inline as a Func<QueryDescriptor, IQuery>, but then it may not be obvious that it's a template.

The bigger issue here is that the collate Query option is typed incorrectly as IScript instead of ITemplateQuery. Unfortunately, we can only fix this in 5.x as it would be a breaking change. But fortunately, IScript and ITemplateQuery are pretty much identical, so we can get away with it. This is probably what caused the mix up to begin with.

This explains the confusion around Indexed, which is part of IScript and used to referenced an indexed script. It is the equivalent of referencing a stored template.

Hope that makes sense. Sorry for the confusion!

[harvzor]

Okay, thanks for the information.

Can you help with setting up stored templates with Nest?

I have managed to do it using just an ordinary PUT through Kibana Sense:

PUT /_search/template/my_template
{
  "template": {
    "multi_match": {
      "query": {
        "query": "{{suggestion}}",
        "fields": [
          "field1",
          "field2"
        ]
      }
    }
  }
}

But the following fails when I try to run it through the Nest client:

var templateRequest= new PutIndexTemplateRequest(new Name("my_template"))
{
    Template = "\"template\":{\"multi_match\":{\"query\":{\"query\":\"{{suggestion}}\",\"fields\":[\"field1\",\"field2\"]}}}"
};

var response = ElasticClient.PutIndexTemplate(templateRequest);

The error is:

"index_template [my_template] invalid, cause [Validation Failed: 1: template must not contain a ',';2: template must not container the following characters [\\, /, *, ?, \", <, >, |,  , ,];]"

I have checked what JSON data Nest is sending to Elastic, and it is as follows:

\"template\":{\"multi_match\":{\"query\":{\"query\":\"{{suggestion}}\",\"fields\":[\"field1\",\"field2\"]}}}

This feels a little bit like a bug, as the template queries appear to be able to have commas in them, but the error message specifically says that the template query cannot contain a comma.

[harvzor]

It turns out I was using the wrong type of request. I was using a PutIndexTemplateRequest whereas I should have been using a PutSearchTemplateDescriptor.

The final code block corrected is:

var templateRequest= new PutSearchTemplateDescriptor(new Id("my_template"));
templateRequest.Template("{\"multi_match\":{\"query\":{\"query\":\"{{suggestion}}\",\"fields\":[\"field1\",\"field2\"]}}}");

var response = ElasticClient.PutSearchTemplate(templateRequest);

When using the template in a suggest collate:

 .Collate(c => c
     .Query(q => q
         .Indexed("my_template")
     )
     .Prune()
)

[harvzor]

I'm doing a collate query, and I'm getting the following response from Elastic:

{
  "suggest": {
    "phrase1": [
      {
        "text": "things and items",
        "offset": 0,
        "length": 15,
        "options": [
          {
            "text": "thing and item",
            "score": 0.00081566535,
            "collate_match": false
          }
        ]
      }
    ]
  }
}

However, I can't filter out options that have "collate_match" equal to false using NEST:

searchResult.Suggest
    .Select(phrase => phrase.Value
        .Select(value => value.Options
             .Where(option => option.CollateMatch) // CollateMatch does not exist on option (of type SuggestOption), but Text and Score both do!
        )
    );

It seems that no property matches the field "collate_match" in NEST: https://github.com/elastic/elasticsearch-net/blob/2.4.2/src/Nest/Search/Suggesters/SuggestOption.cs

[gmarz]

Closed via #2193. I'll open a new issue for changing IScript to ITemplateQuery for 5.x.