Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add scoring support to percolate api #3506

Closed
martijnvg opened this issue Aug 14, 2013 · 2 comments
Closed

Add scoring support to percolate api #3506

martijnvg opened this issue Aug 14, 2013 · 2 comments
Assignees
@martijnvg
Labels
>feature v1.0.0.Beta1

Comments

@martijnvg
Copy link
Member

Adding scoring support will allow the percolate matches to be sorted, or just assign a scores to percolate matches. Sorting by score can be very useful when millions of matches are being returned.

The scoring support hooks in into the percolate query option and adds two new boolean options:

  • sort - Whether to sort the matches based on the score. This will also include the score for each match. The size option is a required option when sorting percolate matches is enabled.
  • score - Whether to compute the score and include it with each match. This will not sort the matches.

For both new options the query option needs to be specified, which is used to produce the scores. The query option is normally used to control which percolate queries are evaluated. In order to give meaning to these score, the recently added function_score query in #3423 can be used to wrap these queries as is shown in the examples below.

Examples

Indexing dummy percolator queries:

First query:

curl -XPUT 'localhost:9200/my-index/_percolator/1' -d '{ 
    "query": { 
        "match_all" : {} 
    },
    "priority" : 1,
    "create_date" : "2010/01/01"
}'

Second query:

curl -XPUT 'localhost:9200/my-index/_percolator/2' -d '{ 
    "query": { 
        "match_all" : {} 
    },
    "priority" : 2,
    "create_date" : "2013/01/01"
}'

These queries have two extra fields: 'priority' and 'create_date'. These fields can be used in the percolate api during sorting.

Script score example

Percolate request using the script_score function:

{ 
    "doc": { 
        "field" : "value" 
    },
    "query" : {
        "function_score" : {
            "query" : { "match_all": {}},
            "functions" : [
                {
                    "script_score" : {
                        "script" : "doc['priority'].value"
                    }
                }
            ]
        }
    },
    "sort" : true,
    "size" : 10
}

Response:

{
   "took": 118,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "total": 2,
   "matches": [
      {
         "_index": "my-index",
         "_id": "2",
         "_score": 2
      },
      {
         "_index": "my-index",
         "_id": "1",
         "_score": 1
      }
   ]
}

Decay function example

Percolate request using the exp decay function:

{ 
    "doc": { 
        "field" : "value" 
    },
    "query" : {
        "function_score" : {
            "query" : { "match_all": {}},
            "functions" : [
                {
                    "exp" : {
                        "create_date" : {
                            "reference" : "2013/08/14",
                            "scale" : "1000d"
                        }
                    }
                }
            ]
        }
    },
    "sort" : true,
    "size" : 10
}

Response:

{
   "took": 2,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "total": 2,
   "matches": [
      {
         "_index": "my-index",
         "_id": "2",
         "_score": 0.85559505
      },
      {
         "_index": "my-index",
         "_id": "1",
         "_score": 0.4002574
      }
   ]
}
@ghost ghost assigned martijnvg Aug 14, 2013
@martijnvg martijnvg mentioned this issue Sep 3, 2013
Closed
@zoellner
Copy link

Just in case someone comes across this (it's one of the few percolate examples showing up on google) and tries to get it to work, the percolate request needs a string or array in the sort field, not a boolean.
(In addition, due to issues with dynamic scripts, one might have to switch the language from the default groovy to expression)

{ 
    "doc": { 
        "field" : "value" 
    },
    "query" : {
        "function_score" : {
            "query" : { "match_all": {}},
            "functions" : [
                {
                    "script_score" : {
                        "script" : "doc['priority'].value",
                        "lang": "expression"
                    }
                }
            ]
        }
    },
    "sort" : "_score",
    "size" : 10
}

@ncolomer ncolomer mentioned this issue Sep 28, 2015
Closed
@Charlemagne3
Copy link

Confirming @zoellner, I had to use the following for my more complex filtered query:

{
    "query": {
        "filtered": {
            "filter": {
                "term": {
                    "tenant_id": 29
                }
            },
            "query": {
                "function_score": {
                    "query": {"match_all": {}},
                    "functions": [
                        {
                            "script_score": {
                                "script": "doc[\"priority\"].value",
                                "lang": "expression"
                            }
                        }
                    ]
                }
            }
        }
    },
    "sort": "_score",
    "size": 10
}

note the escaped quotes; the single quotes did not work for me, they gave an exception:

{"script": "doc['priority'].value","lang": "expression"}}]}},"sort": "_score","size": 10}}}'
{"took":11,"_shards":{"total":5,"successful":0,"failed":5,"failures":[{"shard":0,"index":"pulley.c","status":"BAD_REQUEST","reason":{"type":"parse_exception","reason":"failed to parse request","caused_by":{"type":"query_parsing_exception","reason":"script_score the script could not be loaded","index":"pulley.c","line":1,"col":205,"caused_by":{"type":"script_exception","reason":"Failed to compile inline script [doc[priority].value] using lang [expression]","caused_by":{"type":"script_exception","reason":"Failed to parse expression: doc[priority].value","caused_by":{"type":"parse_exception","reason":"unexpected character '[' on line (1) position (3)","caused_by":{"type":"lexer_no_viable_alt_exception","reason":null}}}}}}}]},"total":0,"matches":[]}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@martijnvg martijnvg

Labels
>feature v1.0.0.Beta1
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@martijnvg @zoellner @Charlemagne3