Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seeded random ordering (Feature request) #1170

Closed
pawpro opened this Issue Jul 27, 2011 · 24 comments

Comments

Projects
None yet
@pawpro
Copy link

pawpro commented Jul 27, 2011

Implementation of random result ordering with optional seed allowing for recreation of random order i.e. across "pages" when using pagination (from, size).

@mattiassvedhem

This comment has been minimized.

Copy link

mattiassvedhem commented Mar 20, 2012

I'm really interested in this, did someone try to implement this?

@pawpro

This comment has been minimized.

Copy link
Author

pawpro commented Mar 21, 2012

I discussed this briefly with kimchy at the time and his feedback was that it wouldn't be hard to do. Definitely a great feature candidate.
I have implemented this by storing set of "random" numbers e.g. each object has say 20 additional integers (each one of these is a sorting field named as rand1, rand2, rand3 etc). For each user I select (calculate based on date and UA) number 1...20 which is used to select which "random" field will be used. When new objects are added each simply receives set of random. Umbers. You decide how many random states you need. Remember that reverse order in this case doubles the number of random states. This solution is all about perception.

@mattiassvedhem

This comment has been minimized.

Copy link

mattiassvedhem commented Mar 22, 2012

Cool! I'll try that.

@Evan-R

This comment has been minimized.

Copy link

Evan-R commented Apr 18, 2013

Has this been implemented into the current version of ES yet?

@s1monw

This comment has been minimized.

Copy link
Contributor

s1monw commented Apr 18, 2013

@uboness can we pull your latest commit in for this?

@emarthinsen

This comment has been minimized.

Copy link

emarthinsen commented May 10, 2013

This would be great if it were added.

@missinglink

This comment has been minimized.

Copy link
Contributor

missinglink commented Jul 22, 2013

BUMP++

uboness added a commit to uboness/elasticsearch that referenced this issue Aug 13, 2013

Added support for random sort:
    * Support seeds for consistent pagination. If no seed is provided, the current timestamp is used (at the "cost" of consistent pagination).
    * Note, just like normal search, the pagination will be consistent up to segment merges, for absolute consistency scroll should be used
    * order is supported for (reverse) consistent pagination
    * Enhanced the SortParser abstraction to enable sort parser with default configurations (enables the user to specify the sort as a simple string w

    Closes elastic#1170

@uboness uboness closed this in c93eae8 Aug 17, 2013

uboness added a commit that referenced this issue Aug 17, 2013

Added support for random_score function:
 * can be used to return matching results in random order

 Closes #1170
@uboness

This comment has been minimized.

Copy link
Contributor

uboness commented Aug 17, 2013

random order is now supported as part of the new function score queries. request example:

curl -XGET 'localhost:9200/_search' -d '{
  "query": {
    "function_score" : {
      "query" : { "match_all": {} },
      "random_score" : {}
    }
  }
}';

or with a seed (for near-consistent pagination):

curl -XGET 'localhost:9200/_search' -d '{
  "query": {
    "function_score" : {
      "query" : { "match_all": {} },
      "random_score" : { "seed" : 1376773391128418000 }
    }
  }
}';

Check out more on function score queries here: #3423

@emptyflask

This comment has been minimized.

Copy link

emptyflask commented Aug 20, 2013

This is immensely helpful. I hope we'll see this in v0.90.4 in a matter of days...

@s1monw

This comment has been minimized.

Copy link
Contributor

s1monw commented Aug 20, 2013

@emptyflask can you tell us a bit about your actual usecase for this? I am very curious and it would be good to have a couple of usecases on the reference documenation

@emptyflask

This comment has been minimized.

Copy link

emptyflask commented Aug 20, 2013

@s1monw I'm currently doing this with Solr, and it's the one thing that's tying up my migration to ES on that app. There are a few thousand user-submitted ideas that receive votes from other users, so to give every idea an equal chance of showing up on the first page, we want the default view to be randomly sorted, yet paginated and consistent for each user. Using the current user's id as a random seed solves this problem nicely.

@missinglink

This comment has been minimized.

Copy link
Contributor

missinglink commented Aug 20, 2013

👍 10 points for @uboness

@brupm

This comment has been minimized.

Copy link

brupm commented Aug 30, 2013

👍 Will this make it into 0.90.4?

@s1monw

This comment has been minimized.

Copy link
Contributor

s1monw commented Aug 31, 2013

@brupm it's already pushed to 0.90 branch so yes it will be in 0.90.4

@damienalexandre

This comment has been minimized.

Copy link
Contributor

damienalexandre commented Sep 2, 2013

Awesome feature, thx!

Another use-case is a Wikipedia-like random page - coupled to some search criteria (to avoid low interest pages) it's much faster than querying the DB for a random ID.

@brupm

This comment has been minimized.

Copy link

brupm commented Sep 3, 2013

@s1monw Any idea when 0.90.4 will be released? Thanks!

@kimchy

This comment has been minimized.

Copy link
Member

kimchy commented Sep 3, 2013

we plan to release 0.90.4 early next week.

@dctdct

This comment has been minimized.

Copy link

dctdct commented Mar 1, 2014

"sort": [
{
"name": {
"order": "random"
}
}
],

Is not working :-(

@emptyflask

This comment has been minimized.

@dctdct

This comment has been minimized.

Copy link

dctdct commented Mar 3, 2014

Maybe this is a bug with the php client but passing this query as array doesn't work.

$json = '{
"query": {
"function_score": {
"functions": [
{
"random_score": {

                   }
                }
             ],
             "boost_mode": "replace",
             "query": {
                "match_all": {}
             }
          }
       }
    }';

    $qry = array(
        'query' => array(
            'function_score' => array(
                'functions' => array(
                    array('random_score' => array())
                ),
                'query' => array(
                    array('match_all' => array())
                )
            )
        )
    );

    $searchParams['body'] = $qry;

    $retDoc = $elastic->search($searchParams);
@yao23

This comment has been minimized.

Copy link

yao23 commented May 16, 2014

How to iterate every items in each page with the random score function?

@emptyflask

This comment has been minimized.

Copy link

emptyflask commented May 16, 2014

Same way you would iterate over any other set of results. The function
score only affects the score attribute of each document, which reorders the
results. Everything else remains the same.
On May 16, 2014 4:28 PM, "Yao Li" notifications@github.com wrote:

How to iterate every items in each page with the random score function?


Reply to this email directly or view it on GitHubhttps://github.com//issues/1170#issuecomment-43381486
.

mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015

Added support for random_score function:
 * can be used to return matching results in random order

 Closes elastic#1170
@bato3

This comment has been minimized.

Copy link

bato3 commented Mar 21, 2018

@dctdct for PHP use new stdClass() instead Array() This is json_encode "feature"

@johnfrades

This comment has been minimized.

Copy link

johnfrades commented Dec 18, 2018

Sweet! thanks @uboness

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.