Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty index corrupts sort order #8226

Closed
matthughes opened this Issue Oct 25, 2014 · 5 comments

Comments

Projects
None yet
3 participants
@matthughes
Copy link
Contributor

matthughes commented Oct 25, 2014

I've been trying to track down some strange sorting results and have narrowed it down to having an empty index. I have multiple indexes all aliased to the same value. Sort order works fine, but once you add an empty index, the order is corrupted. Reproduction below:

#!/bin/bash

# Create index with no docs with alias test
curl -s -XPOST "http://localhost:9200/test-1"     -d '{ "aliases" : { "test" : {} }}' > /dev/null

# Then create a bunch of indices with alias test
curl -s XPOST "http://localhost:9200/test-2"     -d '{ "aliases" : { "test" : {} }}' > /dev/null
curl -s -XPOST "http://localhost:9200/test-2/doc" -d "{ \"entry\": 1 }"

curl -s -XPOST "http://localhost:9200/test-3"     -d '{ "aliases" : { "test" : {} }}' > /dev/null
curl -s -XPOST "http://localhost:9200/test-3/doc" -d "{ \"entry\": 2 }"

curl -s -XPOST "http://localhost:9200/test-4"     -d '{ "aliases" : { "test" : {} }}' > /dev/null
curl -s -XPOST "http://localhost:9200/test-4/doc" -d "{ \"entry\": 3 }"

curl -s -XPOST "http://localhost:9200/test-5"     -d '{ "aliases" : { "test" : {} }}' > /dev/null
curl -s -XPOST "http://localhost:9200/test-5/doc" -d "{ \"entry\": 4 }"

curl -s -XPOST "http://localhost:9200/test-6"     -d '{ "aliases" : { "test" : {} }}' > /dev/null
curl -s -XPOST "http://localhost:9200/test-6/doc" -d "{ \"entry\": 5 }"

curl -s -XPOST "http://localhost:9200/test-7"     -d '{ "aliases" : { "test" : {} }}' > /dev/null
curl -s -XPOST "http://localhost:9200/test-7/doc" -d "{ \"entry\": 6 }"

sleep 2

# Perform a sorted query, descending on field 'entry'
curl -XPOST 'http://localhost:9200/test/_search?pretty' -d '{
  "query": {
    "filtered": {
      "query": {
        "bool": {
          "should": [
            {
              "query_string": {
                "query": "*"
              }
            }
          ]
        }
      },
      "filter": {
        "bool": {
          "must": [
            {
              "match_all": {}
            }
          ]
        }
      }
    }
  },
  "highlight": {
    "fields": {},
    "fragment_size": 2147483647,
    "pre_tags": [
      "@start-highlight@"
    ],
    "post_tags": [
      "@end-highlight@"
    ]
  },
  "size": 500,
  "sort": [
    {
      "entry": {
        "order": "desc",
        "ignore_unmapped": true
      }
    }
  ]
}' | jq ".hits.hits[] | ._source.entry"

Results:

5
1
3
2
6
4

I can fix sort order in two ways: 1) remove the empty index or 2) change "ignore_unmapped" to false.

But IMO an empty index should no affect sort results. I'm also confused on why "ignore_unmapped": false, fixes things. I would think you would want the reverse, but it fails when true.

@s1monw

This comment has been minimized.

Copy link
Contributor

s1monw commented Oct 27, 2014

I can reproduce this on 1.3 branch with:

public void testIssue8226() {
        int numIndices = between(5, 10);
        for (int i = 0; i < numIndices; i++) {
            assertAcked(prepareCreate("test_" + i).addAlias(new Alias("test")));
            if (i > 0) {
                client().prepareIndex("test_" + i, "foo", "" + i).setSource("{\"entry\": " + i + "}").get();
            }
        }
        ensureYellow();
        refresh();
        SearchResponse searchResponse = client().prepareSearch()
                .addSort(new FieldSortBuilder("entry").order(SortOrder.DESC).ignoreUnmapped(true))
                .setSize(10).get();
        assertSearchResponse(searchResponse);

        for (int j = 1; j < searchResponse.getHits().hits().length; j++) {
            assertThat(searchResponse.toString(), searchResponse.getHits().hits()[j].getId(), lessThan(searchResponse.getHits().hits()[j-1].getId()));
        }
    }

@s1monw s1monw added the >bug label Oct 27, 2014

@s1monw s1monw self-assigned this Oct 27, 2014

@s1monw

This comment has been minimized.

Copy link
Contributor

s1monw commented Oct 27, 2014

It seems this doesn't happen in master, 1.x and neither on 1.4 but the test fails on 1.3 on every run so either we forgot to backport a fix or it's been fixed due to the cut-over to lucene comparators or so?

s1monw added a commit that referenced this issue Oct 27, 2014

s1monw added a commit that referenced this issue Oct 27, 2014

s1monw added a commit that referenced this issue Oct 27, 2014

@s1monw

This comment has been minimized.

Copy link
Contributor

s1monw commented Oct 27, 2014

alright I found the issue - nasty... will open a PR soon

@jpountz

This comment has been minimized.

Copy link
Contributor

jpountz commented Oct 27, 2014

It seems this doesn't happen in master, 1.x and neither on 1.4 but the test fails on 1.3 on every run so either we forgot to backport a fix or it's been fixed due to the cut-over to lucene comparators or so?

I believe it is because this change #7039 is only on 1.4+

s1monw added a commit to s1monw/elasticsearch that referenced this issue Oct 27, 2014

@s1monw s1monw added the v1.3.5 label Oct 27, 2014

@s1monw

This comment has been minimized.

Copy link
Contributor

s1monw commented Oct 27, 2014

fixed in 1.3.5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.