Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add scoring support to percolate api #3506

Closed
martijnvg opened this Issue Aug 14, 2013 · 2 comments

Comments

Projects
None yet
3 participants
@martijnvg
Copy link
Member

martijnvg commented Aug 14, 2013

Adding scoring support will allow the percolate matches to be sorted, or just assign a scores to percolate matches. Sorting by score can be very useful when millions of matches are being returned.

The scoring support hooks in into the percolate query option and adds two new boolean options:

  • sort - Whether to sort the matches based on the score. This will also include the score for each match. The size option is a required option when sorting percolate matches is enabled.
  • score - Whether to compute the score and include it with each match. This will not sort the matches.

For both new options the query option needs to be specified, which is used to produce the scores. The query option is normally used to control which percolate queries are evaluated. In order to give meaning to these score, the recently added function_score query in #3423 can be used to wrap these queries as is shown in the examples below.

Examples

Indexing dummy percolator queries:

First query:

curl -XPUT 'localhost:9200/my-index/_percolator/1' -d '{ 
    "query": { 
        "match_all" : {} 
    },
    "priority" : 1,
    "create_date" : "2010/01/01"
}'

Second query:

curl -XPUT 'localhost:9200/my-index/_percolator/2' -d '{ 
    "query": { 
        "match_all" : {} 
    },
    "priority" : 2,
    "create_date" : "2013/01/01"
}'

These queries have two extra fields: 'priority' and 'create_date'. These fields can be used in the percolate api during sorting.

Script score example

Percolate request using the script_score function:

{ 
    "doc": { 
        "field" : "value" 
    },
    "query" : {
        "function_score" : {
            "query" : { "match_all": {}},
            "functions" : [
                {
                    "script_score" : {
                        "script" : "doc['priority'].value"
                    }
                }
            ]
        }
    },
    "sort" : true,
    "size" : 10
}

Response:

{
   "took": 118,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "total": 2,
   "matches": [
      {
         "_index": "my-index",
         "_id": "2",
         "_score": 2
      },
      {
         "_index": "my-index",
         "_id": "1",
         "_score": 1
      }
   ]
}

Decay function example

Percolate request using the exp decay function:

{ 
    "doc": { 
        "field" : "value" 
    },
    "query" : {
        "function_score" : {
            "query" : { "match_all": {}},
            "functions" : [
                {
                    "exp" : {
                        "create_date" : {
                            "reference" : "2013/08/14",
                            "scale" : "1000d"
                        }
                    }
                }
            ]
        }
    },
    "sort" : true,
    "size" : 10
}

Response:

{
   "took": 2,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "total": 2,
   "matches": [
      {
         "_index": "my-index",
         "_id": "2",
         "_score": 0.85559505
      },
      {
         "_index": "my-index",
         "_id": "1",
         "_score": 0.4002574
      }
   ]
}
@zoellner

This comment has been minimized.

Copy link

zoellner commented Jun 30, 2015

Just in case someone comes across this (it's one of the few percolate examples showing up on google) and tries to get it to work, the percolate request needs a string or array in the sort field, not a boolean.
(In addition, due to issues with dynamic scripts, one might have to switch the language from the default groovy to expression)

{ 
    "doc": { 
        "field" : "value" 
    },
    "query" : {
        "function_score" : {
            "query" : { "match_all": {}},
            "functions" : [
                {
                    "script_score" : {
                        "script" : "doc['priority'].value",
                        "lang": "expression"
                    }
                }
            ]
        }
    },
    "sort" : "_score",
    "size" : 10
}
@Charlemagne3

This comment has been minimized.

Copy link

Charlemagne3 commented Mar 15, 2016

Confirming @zoellner, I had to use the following for my more complex filtered query:

{
    "query": {
        "filtered": {
            "filter": {
                "term": {
                    "tenant_id": 29
                }
            },
            "query": {
                "function_score": {
                    "query": {"match_all": {}},
                    "functions": [
                        {
                            "script_score": {
                                "script": "doc[\"priority\"].value",
                                "lang": "expression"
                            }
                        }
                    ]
                }
            }
        }
    },
    "sort": "_score",
    "size": 10
}

note the escaped quotes; the single quotes did not work for me, they gave an exception:

{"script": "doc['priority'].value","lang": "expression"}}]}},"sort": "_score","size": 10}}}'
{"took":11,"_shards":{"total":5,"successful":0,"failed":5,"failures":[{"shard":0,"index":"pulley.c","status":"BAD_REQUEST","reason":{"type":"parse_exception","reason":"failed to parse request","caused_by":{"type":"query_parsing_exception","reason":"script_score the script could not be loaded","index":"pulley.c","line":1,"col":205,"caused_by":{"type":"script_exception","reason":"Failed to compile inline script [doc[priority].value] using lang [expression]","caused_by":{"type":"script_exception","reason":"Failed to parse expression: doc[priority].value","caused_by":{"type":"parse_exception","reason":"unexpected character '[' on line (1) position (3)","caused_by":{"type":"lexer_no_viable_alt_exception","reason":null}}}}}}}]},"total":0,"matches":[]}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.