Expose matching nested docs to the query and sort

[clintongormley]

Hiya

I have a use case for nested docs which, currently, can not be supported easily.

For instance, have a look at this doc:

{
  "location" : { "lat" : "8.31667", "lon" : "52.38333" },
  "parent_ids" : [ "2862926", "2921044" ],
  "contexts" : [ {
        "context" : "/en",
        "rank" : 2,
        "tokens" : [ 
            { "tokens" : [ "steinbrink" ] },
            { "tokens" : [ "steinbrink", "germany" ] }
        ],
        "label" : {
           "short" : "Steinbrink",
           "long" : "Steinbrink, Germany"
        }
     },
     {
        "context" : "/de",
        "rank" : 3,
        "tokens" : [ 
            { "tokens" : [ "steinbrink" ] },
            { "tokens" : [ "steinbrink", "deutschland" ] }
        ],
        "label" : {
           "short" : "Steinbrink",
           "long" : "Steinbrink, Deutschland"
        }
     }
  ]
}

contexts and tokens are both nested docs.

I would like to run query like:

• filter by contexts.context == '/de'
• sort by:
  • contexts.context.rank desc
  • contexts.label.short asc

but of course I have no way of pulling out the single matching context and using the values in that sub-doc.

I could use a script sort like the horribly inefficient:

{
   "sort" : [
      {
         "params" : {
            "name" : "/de"
         },
         "_script" : {
            "script" : "for (context: contexts) { if (context.context == name) { return context.rank }}",
            "order" : "desc",
            "type" : "number"
         }
      },
      {
         "_script" : {
            "params" : {
               "name" : "/de"
            },
            "script" : "for (context: contexts) { if (context.context == name) { return context.label.short }}",
            "order" : "asc",
            "type" : "string"
         }
      }
   ]
}

Or I may want to do:

• filter by context=='/de'
• find doc with tokens that best match stein*
• return the tokens that match best as a field

Or have a slightly different doc structure:

"tokens" : [ 
    { 
        "label": "Steinbrink",
        "tokens" : [ "steinbrink" ] 
    },
    { 
        "label": "Steinbrink, Germany",
        "tokens" : [ "steinbrink, germany" ] 
    },
],

and return the label associated with the tokens that best match.

Even with a script, I couldn't find the best matching tokens.

In this example, I only have two contexts, and so could have changed the doc structure to something like:

contexts: {
  en: {....},
  de: {....}
}

But I have other use cases where I have hundreds of contexts, which would result in thousands of fields.

Would it be possible to add functionality to:

• expose the best matching child (or children, with the best matching first in an array) in a _scope so we could do something like:

  "sort": [
  { "_child.myscope[0].rank": "desc" },
  { "_child.myscope[0].label.short": "asc"}
  ],
  "fields": [ "_child.otherscope[0],tokens.label" ]

• return the scores for matching children in a _scope eg:

  { "children":
  {
  "myscope": [5,3,0,2,0],
  "otherscope": [0,1,0]
  }
  }

Not sure how feasible this is, and if it is feasible, there is probably a much more elegant way of doing it.

thanks

clint

[ghost]

+1 for this.

We're having to do additional post processing on the result set to try and "guess" which of the nested documents matched so we can show relevant matches to our users. Being able to return just the matched nested documents (and be able to sort on them) would be a massive win.

Thanks,
Paul.

[bryangreen]

+1

[junckritter]

+1

[jove4015]

+1

[jnarowski]

+1

[chrisyuska]

+1 as well (assuming this hasn't been added yet)

[fbpj]

+1, any news about this ?

[0xgeert]

+1 opens up a lot of use-cases. A big general one: given a product / product-variant mapping -> return only the matched product-variant

[0xgeert]

As for a more concrete example: the following (pretty general imho) usecase would be possible.

http://stackoverflow.com/questions/13974277/modeling-parent-child-relationships-product-productvariant-in-elasticsearch

[jannikks]

+1

[bbansal]

+1

[tommymonk]

+1

[martijnvg]

I'm closing this one. Nested sorting has been implemented and I created an issue for retrieving the matching nested inner objects per hit #3022

[evandu]

+1 for this.
How to filter nested array in response?

[chrisyuska]

@evandu, if you're looking for nested filtering, read this: http://www.elasticsearch.org/guide/reference/query-dsl/nested-filter/

If you're looking for nested sorting, read this: http://www.elasticsearch.org/guide/reference/api/search/sort/ (nested sorting is down the page a little bit)