Support random sort order for Lucene queries

[ArieGato]

Add support for random sort order results of Lucene queries. I have a PR that enables this behavior.
I was wondering how to add this support to the documentation.

[ArieGato]

I suggest the following syntax:

order by DisplayText ascending, then by random

{
  "query": {
    "term": { "Content.ContentItem.ContentType": "BlogPost" }
  },
  "sort": [
    "Content.ContentItem.DisplayText",
    { "Content.ContentItem.CreatedUtc": { "order": "random" } }
  ],
  "size": 10
}

order by DisplayText descending, then by random

{
  "query": {
    "term": { "Content.ContentItem.ContentType": "BlogPost" }
  },
  "sort": [
    { "Content.ContentItem.DisplayText": { "order": "desc" } },
    { "Content.ContentItem.CreatedUtc": { "order": "random" } }
  ],
  "size": 10
}

order by random

{
  "query": {
    "term": { "Content.ContentItem.ContentType": "BlogPost" }
  },
  "sort": [
    { "Content.ContentItem.CreatedUtc": { "order": "random" } }
  ],
  "size": 10
}

order by DisplayText ascending, then by CreatedUtc ascending

{
  "query": {
    "term": { "Content.ContentItem.ContentType": "BlogPost" }
  },
  "sort": [
    "Content.ContentItem.DisplayText",
    "Content.ContentItem.CreatedUtc"
  ],
  "size": 10
}

[deanmarcussen]

Sounds sensible, perhaps you could open the pr, and we can review?

[ArieGato]

Sounds sensible, perhaps you could open the pr, and we can review?

@deanmarcussen Did you have a chance to look at the PR?

[sebastienros]

How does ElasticSearch support it? The syntax seems odd.

[ArieGato]

I wasn't aware that the queries are also used by ElasticSearch. Any idea how to implement this feature then?

[ArieGato]

ElasticSearch supports this by using a random_score leaf within a function_score:

{
  "query": {
    "function_score": {
      "random_score": {
        "seed": 10,
        "field": "_seq_no"
      }
    }
  }
}

source: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#function-random

Edit

function_score is being deprecated: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-script-score-query.html#script-score-function-score-transition

So something like this:

{
  "query": {
    "script_score": {
      "query": {
        "match": { "message": "elasticsearch" }
      },
      "script": {
        "source" : "randomScore(100, '_seq_no')"
      }
    }
  }
}

Not quite sure how to implement this in the Lucene module though.

[Skrypt]

Let's make an extended research on how this should be implemented to make sure. Else, this is just a matter of parsing the json differently with a new "script_score" Query Provider. Remember that we can nest those Queries.

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-script-score-query.html#random-score-function

elastic/elasticsearch#1170

[ArieGato]

@Skrypt Have you investigated how we can add support for random sort order?

[Skrypt]

Your solution works for a global random sort but ElasticSearch queries uses the script_score function to make this available per Query statement. So one part of your results could come randomly sorted and another part sorted by score. I've not worked on this yet but I believe the solution is to implement script_score if it's doable with Lucene.NET. And as @sebastienros suggested, the implementation should follow ElasticSearch documentation ... at least first. I'm open having your solution merged but it needs to be documented in OC.