Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Script based sorting is applied only after pagination #3309

Closed
peschlowp opened this Issue Jul 10, 2013 · 2 comments

Comments

Projects
None yet
5 participants
@peschlowp
Copy link
Contributor

peschlowp commented Jul 10, 2013

When a query requests both sorting and pagination, the expected behavior is that sorting is executed before pagination. And indeed this is the actual behavior when the query requests sorting on a field level. However, when doing script based sorting, it seems that sorting is only executed after pagination. This is behavior is unexpected and especially surprising because the same sort may yield different results depending on whether it is specified on a field level or via a script.

Example

Suppose we have 100 files indexed, named "file001", "file002", ..., "file100", with the file name being stored in a "filename" field. Then the following query

{
  "from" : 0,
  "size" : 10,
  "query" : {
    "match_all" : { }
  },
  "sort" : [ {
    "filename" : {
      "order" : "asc"
    }
  } ]
}

yields the expected ordering "file001", "file002", ..., "file010". The seemingly equivalent query

{
  "from" : 0,
  "size" : 10,
  "query" : {
    "match_all" : { }
  },
  "sort" : [ {
    "_script" : {
      "script" : "doc['filename'].value",
      "type" : "string"
    }
  } ]
}

may return, for example, "file005", "file007", "file020", "file027", "file035", "file050", "file067", "file080", "file092", "file097". Which is pretty useless and certainly not what the client would expect.

This behavior was observed in ElasticSearch 0.90.2, but I didn't test previous versions.

@kimchy

This comment has been minimized.

Copy link
Member

kimchy commented Jul 11, 2013

script sorting doesn't sort after pagination, can you provide a full curl recreation so we can check it out?

@imotov

This comment has been minimized.

Copy link
Member

imotov commented Jul 12, 2013

It looks like a bug that was introduced between 0.90.0RC2 and 0.90.0. Here is a repro. I didn't have a chance to dig into it yet but it looks like it got broken after this commit.

@ghost ghost assigned javanna Jul 12, 2013

s1monw added a commit to s1monw/elasticsearch that referenced this issue Jul 15, 2013

Set spare becore comparing comparator bottom value
The actual documents value was never calculated if setSpare wasn't called
before compareBottom was called on a certain document.

Closes elastic#3309

@ghost ghost assigned s1monw Jul 15, 2013

@s1monw s1monw closed this in 37edfe0 Jul 15, 2013

s1monw added a commit that referenced this issue Jul 15, 2013

Set spare becore comparing comparator bottom value
The actual documents value was never calculated if setSpare wasn't called
before compareBottom was called on a certain document.

Closes #3309

mute pushed a commit to mute/elasticsearch that referenced this issue Jul 29, 2015

Set spare becore comparing comparator bottom value
The actual documents value was never calculated if setSpare wasn't called
before compareBottom was called on a certain document.

Closes elastic#3309

talevy added a commit to talevy/elasticsearch that referenced this issue Apr 25, 2018

Adds ForceMerge action to Index Lifecycle (elastic#3309)
add forcemerge index-lifecycle action

talevy added a commit to talevy/elasticsearch that referenced this issue May 14, 2018

Adds ForceMerge action to Index Lifecycle (elastic#3309)
add forcemerge index-lifecycle action

jasontedor added a commit that referenced this issue Aug 17, 2018

Adds ForceMerge action to Index Lifecycle (#3309)
add forcemerge index-lifecycle action
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.