Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Facets incorrect when scrolling #1642

Closed
clintongormley opened this issue Jan 26, 2012 · 11 comments
Closed

Facets incorrect when scrolling #1642

clintongormley opened this issue Jan 26, 2012 · 11 comments
Assignees

Comments

@clintongormley
Copy link

When requesting facets on a scrolled search, the counts just keep rising, incorrectly.

I'd say it makes more sense to only return the facets on the first request, not on subsequent scroll requests

curl -XPUT 'http://127.0.0.1:9200/test/?pretty=1' 
curl -XPOST 'http://127.0.0.1:9200/test/foo?pretty=1'  -d ' { "num" : 111 }'
curl -XPOST 'http://127.0.0.1:9200/test/foo?pretty=1'  -d ' { "num" : 112 }'
curl -XPOST 'http://127.0.0.1:9200/test/foo?pretty=1'  -d ' { "num" : 113 }'
curl -XPOST 'http://127.0.0.1:9200/test/foo?pretty=1'  -d ' { "num" : 114 }'
curl -XPOST 'http://127.0.0.1:9200/test/foo?pretty=1'  -d ' { "num" : 115 }'
curl -XPOST 'http://127.0.0.1:9200/test/foo?pretty=1'  -d ' { "num" : 116 }'
curl -XPOST 'http://127.0.0.1:9200/test/foo?pretty=1'  -d ' { "num" : 117 }'
curl -XPOST 'http://127.0.0.1:9200/test/foo?pretty=1'  -d ' { "num" : 118 }'
curl -XPOST 'http://127.0.0.1:9200/test/foo?pretty=1'  -d ' { "num" : 119 }'
curl -XPOST 'http://127.0.0.1:9200/test/foo?pretty=1'  -d ' { "num" : 120 }'

Then run facets:

curl -XGET 'http://127.0.0.1:9200/test/foo/_search?scroll=1m&pretty=1'  -d '
{
   "facets" : {
      "num" : {
         "terms" : {
            "field" : "num"
         }
      }
   },
   "size" : 1
}
'

# [Thu Jan 26 19:58:12 2012] Response:
# {
#    "hits" : {
#       "hits" : [
#          {
#             "_source" : {
#                "num" : 113
#             },
#             "_score" : 1,
#             "_index" : "test",
#             "_id" : "ApvhoFAnTTayEyB4NHdMGw",
#             "_type" : "foo"
#          }
#       ],
#       "max_score" : 1,
#       "total" : 10
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUz
# >    JlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80Yl
# >    VRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OT
# >    pXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 1,
#                "term" : 120
#             },
#             {
#                "count" : 1,
#                "term" : 119
#             },
#             {
#                "count" : 1,
#                "term" : 118
#             },
#             {
#                "count" : 1,
#                "term" : 117
#             },
#             {
#                "count" : 1,
#                "term" : 116
#             },
#             {
#                "count" : 1,
#                "term" : 115
#             },
#             {
#                "count" : 1,
#                "term" : 114
#             },
#             {
#                "count" : 1,
#                "term" : 113
#             },
#             {
#                "count" : 1,
#                "term" : 112
#             },
#             {
#                "count" : 1,
#                "term" : 111
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 10
#       }
#    },
#    "took" : 4
# }

# [Thu Jan 26 19:58:21 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XGET 'http://127.0.0.1:9200/_search/scroll?scroll=1m&pretty=1&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OTpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7' 

# [Thu Jan 26 19:58:21 2012] Response:
# {
#    "hits" : {
#       "hits" : [
#          {
#             "_source" : {
#                "num" : 116
#             },
#             "_score" : 1,
#             "_index" : "test",
#             "_id" : "bJUOIFylTBqyTS4kSzeIcw",
#             "_type" : "foo"
#          }
#       ],
#       "max_score" : 1,
#       "total" : 10
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUz
# >    JlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80Yl
# >    VRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OT
# >    pXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 2,
#                "term" : 120
#             },
#             {
#                "count" : 2,
#                "term" : 118
#             },
#             {
#                "count" : 2,
#                "term" : 115
#             },
#             {
#                "count" : 2,
#                "term" : 114
#             },
#             {
#                "count" : 2,
#                "term" : 112
#             },
#             {
#                "count" : 2,
#                "term" : 111
#             },
#             {
#                "count" : 1,
#                "term" : 119
#             },
#             {
#                "count" : 1,
#                "term" : 117
#             },
#             {
#                "count" : 1,
#                "term" : 116
#             },
#             {
#                "count" : 1,
#                "term" : 113
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 16
#       }
#    },
#    "took" : 1
# }

# [Thu Jan 26 19:58:21 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XGET 'http://127.0.0.1:9200/_search/scroll?scroll=1m&pretty=1&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OTpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7' 

# [Thu Jan 26 19:58:21 2012] Response:
# {
#    "hits" : {
#       "hits" : [
#          {
#             "_source" : {
#                "num" : 117
#             },
#             "_score" : 1,
#             "_index" : "test",
#             "_id" : "QVlLmSz6QUm7zi3WthUntQ",
#             "_type" : "foo"
#          }
#       ],
#       "max_score" : 1,
#       "total" : 10
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUz
# >    JlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80Yl
# >    VRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OT
# >    pXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 3,
#                "term" : 120
#             },
#             {
#                "count" : 3,
#                "term" : 118
#             },
#             {
#                "count" : 3,
#                "term" : 115
#             },
#             {
#                "count" : 3,
#                "term" : 114
#             },
#             {
#                "count" : 3,
#                "term" : 112
#             },
#             {
#                "count" : 3,
#                "term" : 111
#             },
#             {
#                "count" : 1,
#                "term" : 119
#             },
#             {
#                "count" : 1,
#                "term" : 117
#             },
#             {
#                "count" : 1,
#                "term" : 116
#             },
#             {
#                "count" : 1,
#                "term" : 113
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 22
#       }
#    },
#    "took" : 1
# }

# [Thu Jan 26 19:58:21 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XGET 'http://127.0.0.1:9200/_search/scroll?scroll=1m&pretty=1&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OTpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7' 

# [Thu Jan 26 19:58:21 2012] Response:
# {
#    "hits" : {
#       "hits" : [
#          {
#             "_source" : {
#                "num" : 119
#             },
#             "_score" : 1,
#             "_index" : "test",
#             "_id" : "OdDkJXgRTPOsvbYA4PjXtg",
#             "_type" : "foo"
#          }
#       ],
#       "max_score" : 1,
#       "total" : 10
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUz
# >    JlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80Yl
# >    VRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OT
# >    pXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 4,
#                "term" : 120
#             },
#             {
#                "count" : 4,
#                "term" : 118
#             },
#             {
#                "count" : 4,
#                "term" : 115
#             },
#             {
#                "count" : 4,
#                "term" : 114
#             },
#             {
#                "count" : 4,
#                "term" : 112
#             },
#             {
#                "count" : 4,
#                "term" : 111
#             },
#             {
#                "count" : 1,
#                "term" : 119
#             },
#             {
#                "count" : 1,
#                "term" : 117
#             },
#             {
#                "count" : 1,
#                "term" : 116
#             },
#             {
#                "count" : 1,
#                "term" : 113
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 28
#       }
#    },
#    "took" : 0
# }

# [Thu Jan 26 19:58:21 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XGET 'http://127.0.0.1:9200/_search/scroll?scroll=1m&pretty=1&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OTpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7' 

# [Thu Jan 26 19:58:21 2012] Response:
# {
#    "hits" : {
#       "hits" : [
#          {
#             "_source" : {
#                "num" : 114
#             },
#             "_score" : 1,
#             "_index" : "test",
#             "_id" : "f24bgaz7ROC1jTldU6UGiQ",
#             "_type" : "foo"
#          }
#       ],
#       "max_score" : 1,
#       "total" : 10
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUz
# >    JlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80Yl
# >    VRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OT
# >    pXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 5,
#                "term" : 120
#             },
#             {
#                "count" : 5,
#                "term" : 118
#             },
#             {
#                "count" : 5,
#                "term" : 115
#             },
#             {
#                "count" : 5,
#                "term" : 114
#             },
#             {
#                "count" : 5,
#                "term" : 112
#             },
#             {
#                "count" : 5,
#                "term" : 111
#             },
#             {
#                "count" : 1,
#                "term" : 119
#             },
#             {
#                "count" : 1,
#                "term" : 117
#             },
#             {
#                "count" : 1,
#                "term" : 116
#             },
#             {
#                "count" : 1,
#                "term" : 113
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 34
#       }
#    },
#    "took" : 1
# }

# [Thu Jan 26 19:58:21 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XGET 'http://127.0.0.1:9200/_search/scroll?scroll=1m&pretty=1&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OTpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7' 

# [Thu Jan 26 19:58:21 2012] Response:
# {
#    "hits" : {
#       "hits" : [
#          {
#             "_source" : {
#                "num" : 111
#             },
#             "_score" : 1,
#             "_index" : "test",
#             "_id" : "B0u6QEKZSKCwaV5-uqkoFQ",
#             "_type" : "foo"
#          }
#       ],
#       "max_score" : 1,
#       "total" : 10
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUz
# >    JlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80Yl
# >    VRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OT
# >    pXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 6,
#                "term" : 120
#             },
#             {
#                "count" : 6,
#                "term" : 118
#             },
#             {
#                "count" : 6,
#                "term" : 115
#             },
#             {
#                "count" : 6,
#                "term" : 114
#             },
#             {
#                "count" : 6,
#                "term" : 112
#             },
#             {
#                "count" : 6,
#                "term" : 111
#             },
#             {
#                "count" : 1,
#                "term" : 119
#             },
#             {
#                "count" : 1,
#                "term" : 117
#             },
#             {
#                "count" : 1,
#                "term" : 116
#             },
#             {
#                "count" : 1,
#                "term" : 113
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 40
#       }
#    },
#    "took" : 0
# }

# [Thu Jan 26 19:58:21 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XGET 'http://127.0.0.1:9200/_search/scroll?scroll=1m&pretty=1&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OTpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7' 

# [Thu Jan 26 19:58:21 2012] Response:
# {
#    "hits" : {
#       "hits" : [
#          {
#             "_source" : {
#                "num" : 112
#             },
#             "_score" : 1,
#             "_index" : "test",
#             "_id" : "3Gho6ITESYmIDdqCIpWWPg",
#             "_type" : "foo"
#          }
#       ],
#       "max_score" : 1,
#       "total" : 10
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUz
# >    JlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80Yl
# >    VRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OT
# >    pXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 7,
#                "term" : 120
#             },
#             {
#                "count" : 7,
#                "term" : 118
#             },
#             {
#                "count" : 7,
#                "term" : 115
#             },
#             {
#                "count" : 7,
#                "term" : 114
#             },
#             {
#                "count" : 7,
#                "term" : 112
#             },
#             {
#                "count" : 7,
#                "term" : 111
#             },
#             {
#                "count" : 1,
#                "term" : 119
#             },
#             {
#                "count" : 1,
#                "term" : 117
#             },
#             {
#                "count" : 1,
#                "term" : 116
#             },
#             {
#                "count" : 1,
#                "term" : 113
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 46
#       }
#    },
#    "took" : 1
# }

# [Thu Jan 26 19:58:21 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XGET 'http://127.0.0.1:9200/_search/scroll?scroll=1m&pretty=1&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OTpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7' 

# [Thu Jan 26 19:58:21 2012] Response:
# {
#    "hits" : {
#       "hits" : [
#          {
#             "_source" : {
#                "num" : 118
#             },
#             "_score" : 1,
#             "_index" : "test",
#             "_id" : "gwtal_yVRhqx8ACAOU2Dog",
#             "_type" : "foo"
#          }
#       ],
#       "max_score" : 1,
#       "total" : 10
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUz
# >    JlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80Yl
# >    VRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OT
# >    pXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 8,
#                "term" : 120
#             },
#             {
#                "count" : 8,
#                "term" : 118
#             },
#             {
#                "count" : 8,
#                "term" : 115
#             },
#             {
#                "count" : 8,
#                "term" : 114
#             },
#             {
#                "count" : 8,
#                "term" : 112
#             },
#             {
#                "count" : 8,
#                "term" : 111
#             },
#             {
#                "count" : 1,
#                "term" : 119
#             },
#             {
#                "count" : 1,
#                "term" : 117
#             },
#             {
#                "count" : 1,
#                "term" : 116
#             },
#             {
#                "count" : 1,
#                "term" : 113
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 52
#       }
#    },
#    "took" : 1
# }

# [Thu Jan 26 19:58:21 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XGET 'http://127.0.0.1:9200/_search/scroll?scroll=1m&pretty=1&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OTpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7' 

# [Thu Jan 26 19:58:21 2012] Response:
# {
#    "hits" : {
#       "hits" : [
#          {
#             "_source" : {
#                "num" : 120
#             },
#             "_score" : 1,
#             "_index" : "test",
#             "_id" : "_tcnUkiFR2mxw6Puzll14A",
#             "_type" : "foo"
#          }
#       ],
#       "max_score" : 1,
#       "total" : 10
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUz
# >    JlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80Yl
# >    VRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OT
# >    pXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 9,
#                "term" : 120
#             },
#             {
#                "count" : 9,
#                "term" : 118
#             },
#             {
#                "count" : 9,
#                "term" : 115
#             },
#             {
#                "count" : 9,
#                "term" : 114
#             },
#             {
#                "count" : 9,
#                "term" : 112
#             },
#             {
#                "count" : 9,
#                "term" : 111
#             },
#             {
#                "count" : 1,
#                "term" : 119
#             },
#             {
#                "count" : 1,
#                "term" : 117
#             },
#             {
#                "count" : 1,
#                "term" : 116
#             },
#             {
#                "count" : 1,
#                "term" : 113
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 58
#       }
#    },
#    "took" : 0
# }

# [Thu Jan 26 19:58:21 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XGET 'http://127.0.0.1:9200/_search/scroll?scroll=1m&pretty=1&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OTpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7' 

# [Thu Jan 26 19:58:21 2012] Response:
# {
#    "hits" : {
#       "hits" : [
#          {
#             "_source" : {
#                "num" : 115
#             },
#             "_score" : 1,
#             "_index" : "test",
#             "_id" : "G0Te7qRFRWSq9XzyLNb74g",
#             "_type" : "foo"
#          }
#       ],
#       "max_score" : 1,
#       "total" : 10
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUz
# >    JlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80Yl
# >    VRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OT
# >    pXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 10,
#                "term" : 120
#             },
#             {
#                "count" : 10,
#                "term" : 118
#             },
#             {
#                "count" : 10,
#                "term" : 115
#             },
#             {
#                "count" : 10,
#                "term" : 114
#             },
#             {
#                "count" : 10,
#                "term" : 112
#             },
#             {
#                "count" : 10,
#                "term" : 111
#             },
#             {
#                "count" : 1,
#                "term" : 119
#             },
#             {
#                "count" : 1,
#                "term" : 117
#             },
#             {
#                "count" : 1,
#                "term" : 116
#             },
#             {
#                "count" : 1,
#                "term" : 113
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 64
#       }
#    },
#    "took" : 1
# }

# [Thu Jan 26 19:58:21 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XGET 'http://127.0.0.1:9200/_search/scroll?scroll=1m&pretty=1&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OTpXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7' 

# [Thu Jan 26 19:58:21 2012] Response:
# {
#    "hits" : {
#       "hits" : [],
#       "max_score" : 1,
#       "total" : 10
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTs5NjpXSWkxTzRiVVFIS2EtUz
# >    JlVnFXdXFnOzk3OldJaTFPNGJVUUhLYS1TMmVWcVd1cWc7OTg6V0lpMU80Yl
# >    VRSEthLVMyZVZxV3VxZzsxMDA6V0lpMU80YlVRSEthLVMyZVZxV3VxZzs5OT
# >    pXSWkxTzRiVVFIS2EtUzJlVnFXdXFnOzA7",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 11,
#                "term" : 120
#             },
#             {
#                "count" : 11,
#                "term" : 118
#             },
#             {
#                "count" : 11,
#                "term" : 115
#             },
#             {
#                "count" : 11,
#                "term" : 114
#             },
#             {
#                "count" : 11,
#                "term" : 112
#             },
#             {
#                "count" : 11,
#                "term" : 111
#             },
#             {
#                "count" : 1,
#                "term" : 119
#             },
#             {
#                "count" : 1,
#                "term" : 117
#             },
#             {
#                "count" : 1,
#                "term" : 116
#             },
#             {
#                "count" : 1,
#                "term" : 113
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 70
#       }
#    },
#    "took" : 1
# }

# [Thu Jan 26 20:02:12 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XPUT 'http://127.0.0.1:9200/test/?pretty=1' 

# [Thu Jan 26 20:02:12 2012] Response:
# {
#    "ok" : true,
#    "acknowledged" : true
# }

# [Thu Jan 26 20:02:15 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XPOST 'http://127.0.0.1:9200/test/foo?pretty=1'  -d '
{
   "num" : 1
}
'

# [Thu Jan 26 20:02:15 2012] Response:
# {
#    "ok" : true,
#    "_index" : "test",
#    "_id" : "4kjF8ezXQUSTdm0wqHCDmg",
#    "_type" : "foo",
#    "_version" : 1
# }

# [Thu Jan 26 20:02:15 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XPOST 'http://127.0.0.1:9200/test/foo?pretty=1'  -d '
{
   "num" : 2
}
'

# [Thu Jan 26 20:02:15 2012] Response:
# {
#    "ok" : true,
#    "_index" : "test",
#    "_id" : "0AmRRD58QnWE-hyeRF11Kw",
#    "_type" : "foo",
#    "_version" : 1
# }

# [Thu Jan 26 20:02:15 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XPOST 'http://127.0.0.1:9200/test/foo?pretty=1'  -d '
{
   "num" : 3
}
'

# [Thu Jan 26 20:02:15 2012] Response:
# {
#    "ok" : true,
#    "_index" : "test",
#    "_id" : "whjOB26XQFq5eiYiThNk-Q",
#    "_type" : "foo",
#    "_version" : 1
# }

# [Thu Jan 26 20:02:15 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XPOST 'http://127.0.0.1:9200/test/foo?pretty=1'  -d '
{
   "num" : 4
}
'

# [Thu Jan 26 20:02:15 2012] Response:
# {
#    "ok" : true,
#    "_index" : "test",
#    "_id" : "Ax7l0JEWTgWZUYsUcl4oog",
#    "_type" : "foo",
#    "_version" : 1
# }

# [Thu Jan 26 20:02:18 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XGET 'http://127.0.0.1:9200/test/foo/_search?scroll=1m&pretty=1'  -d '
{
   "facets" : {
      "num" : {
         "terms" : {
            "field" : "num"
         }
      }
   },
   "size" : 1
}
'

# [Thu Jan 26 20:02:18 2012] Response:
# {
#    "hits" : {
#       "hits" : [
#          {
#             "_source" : {
#                "num" : 3
#             },
#             "_score" : 1,
#             "_index" : "test",
#             "_id" : "whjOB26XQFq5eiYiThNk-Q",
#             "_type" : "foo"
#          }
#       ],
#       "max_score" : 1,
#       "total" : 4
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTsxMTE6V0lpMU80YlVRSEthLV
# >    MyZVZxV3VxZzsxMTI6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTM6V0lpMU
# >    80YlVRSEthLVMyZVZxV3VxZzsxMTU6V0lpMU80YlVRSEthLVMyZVZxV3VxZz
# >    sxMTQ6V0lpMU80YlVRSEthLVMyZVZxV3VxZzswOw==",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 1,
#                "term" : 4
#             },
#             {
#                "count" : 1,
#                "term" : 3
#             },
#             {
#                "count" : 1,
#                "term" : 2
#             },
#             {
#                "count" : 1,
#                "term" : 1
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 4
#       }
#    },
#    "took" : 2
# }

# [Thu Jan 26 20:02:20 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XGET 'http://127.0.0.1:9200/_search/scroll?scroll=1m&pretty=1&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTsxMTE6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTI6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTM6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTU6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTQ6V0lpMU80YlVRSEthLVMyZVZxV3VxZzswOw%3D%3D' 

# [Thu Jan 26 20:02:20 2012] Response:
# {
#    "hits" : {
#       "hits" : [
#          {
#             "_source" : {
#                "num" : 2
#             },
#             "_score" : 1,
#             "_index" : "test",
#             "_id" : "0AmRRD58QnWE-hyeRF11Kw",
#             "_type" : "foo"
#          }
#       ],
#       "max_score" : 1,
#       "total" : 4
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTsxMTE6V0lpMU80YlVRSEthLV
# >    MyZVZxV3VxZzsxMTI6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTM6V0lpMU
# >    80YlVRSEthLVMyZVZxV3VxZzsxMTU6V0lpMU80YlVRSEthLVMyZVZxV3VxZz
# >    sxMTQ6V0lpMU80YlVRSEthLVMyZVZxV3VxZzswOw==",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 2,
#                "term" : 4
#             },
#             {
#                "count" : 2,
#                "term" : 2
#             },
#             {
#                "count" : 2,
#                "term" : 1
#             },
#             {
#                "count" : 1,
#                "term" : 3
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 7
#       }
#    },
#    "took" : 1
# }

# [Thu Jan 26 20:02:20 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XGET 'http://127.0.0.1:9200/_search/scroll?scroll=1m&pretty=1&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTsxMTE6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTI6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTM6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTU6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTQ6V0lpMU80YlVRSEthLVMyZVZxV3VxZzswOw%3D%3D' 

# [Thu Jan 26 20:02:20 2012] Response:
# {
#    "hits" : {
#       "hits" : [
#          {
#             "_source" : {
#                "num" : 4
#             },
#             "_score" : 1,
#             "_index" : "test",
#             "_id" : "Ax7l0JEWTgWZUYsUcl4oog",
#             "_type" : "foo"
#          }
#       ],
#       "max_score" : 1,
#       "total" : 4
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTsxMTE6V0lpMU80YlVRSEthLV
# >    MyZVZxV3VxZzsxMTI6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTM6V0lpMU
# >    80YlVRSEthLVMyZVZxV3VxZzsxMTU6V0lpMU80YlVRSEthLVMyZVZxV3VxZz
# >    sxMTQ6V0lpMU80YlVRSEthLVMyZVZxV3VxZzswOw==",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 3,
#                "term" : 4
#             },
#             {
#                "count" : 3,
#                "term" : 2
#             },
#             {
#                "count" : 3,
#                "term" : 1
#             },
#             {
#                "count" : 1,
#                "term" : 3
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 10
#       }
#    },
#    "took" : 1
# }

# [Thu Jan 26 20:02:20 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XGET 'http://127.0.0.1:9200/_search/scroll?scroll=1m&pretty=1&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTsxMTE6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTI6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTM6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTU6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTQ6V0lpMU80YlVRSEthLVMyZVZxV3VxZzswOw%3D%3D' 

# [Thu Jan 26 20:02:20 2012] Response:
# {
#    "hits" : {
#       "hits" : [
#          {
#             "_source" : {
#                "num" : 1
#             },
#             "_score" : 1,
#             "_index" : "test",
#             "_id" : "4kjF8ezXQUSTdm0wqHCDmg",
#             "_type" : "foo"
#          }
#       ],
#       "max_score" : 1,
#       "total" : 4
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTsxMTE6V0lpMU80YlVRSEthLV
# >    MyZVZxV3VxZzsxMTI6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTM6V0lpMU
# >    80YlVRSEthLVMyZVZxV3VxZzsxMTU6V0lpMU80YlVRSEthLVMyZVZxV3VxZz
# >    sxMTQ6V0lpMU80YlVRSEthLVMyZVZxV3VxZzswOw==",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 4,
#                "term" : 4
#             },
#             {
#                "count" : 4,
#                "term" : 2
#             },
#             {
#                "count" : 4,
#                "term" : 1
#             },
#             {
#                "count" : 1,
#                "term" : 3
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 13
#       }
#    },
#    "took" : 1
# }

# [Thu Jan 26 20:02:20 2012] Protocol: http, Server: 192.168.5.10:9200
curl -XGET 'http://127.0.0.1:9200/_search/scroll?scroll=1m&pretty=1&scroll_id=cXVlcnlUaGVuRmV0Y2g7NTsxMTE6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTI6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTM6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTU6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTQ6V0lpMU80YlVRSEthLVMyZVZxV3VxZzswOw%3D%3D' 

# [Thu Jan 26 20:02:20 2012] Response:
# {
#    "hits" : {
#       "hits" : [],
#       "max_score" : 1,
#       "total" : 4
#    },
#    "timed_out" : false,
#    "_shards" : {
#       "failed" : 0,
#       "successful" : 5,
#       "total" : 5
#    },
#    "_scroll_id" : "cXVlcnlUaGVuRmV0Y2g7NTsxMTE6V0lpMU80YlVRSEthLV
# >    MyZVZxV3VxZzsxMTI6V0lpMU80YlVRSEthLVMyZVZxV3VxZzsxMTM6V0lpMU
# >    80YlVRSEthLVMyZVZxV3VxZzsxMTU6V0lpMU80YlVRSEthLVMyZVZxV3VxZz
# >    sxMTQ6V0lpMU80YlVRSEthLVMyZVZxV3VxZzswOw==",
#    "facets" : {
#       "num" : {
#          "other" : 0,
#          "terms" : [
#             {
#                "count" : 5,
#                "term" : 4
#             },
#             {
#                "count" : 5,
#                "term" : 2
#             },
#             {
#                "count" : 5,
#                "term" : 1
#             },
#             {
#                "count" : 1,
#                "term" : 3
#             }
#          ],
#          "missing" : 0,
#          "_type" : "terms",
#          "total" : 16
#       }
#    },
#    "took" : 1
# }
@kimchy
Copy link
Member

kimchy commented Feb 1, 2012

Mmm, interesting.... I did not cover the case of using faceting with scrolling since I did not think it makes sense, but, it should at least be consistent... .

@clintongormley
Copy link
Author

We should probably only return facets on the first request, and not on scrolls

@s1monw
Copy link
Contributor

s1monw commented Apr 10, 2013

+1 to only return facets on the first request

@dadoonet
Copy link
Member

I'm wondering if it's a common use case to ask for facets when doing scrolling... I would vote to not compute facets at all when doing scroll or to reject the query with a clear error message.
That way we won't have to answer to questions such as "why do I get facets on the first round trip and facets disappear after?"

@clintongormley
Copy link
Author

@dadoonet i've just had pecke01 in IRC saying that he has customers who are using facets with scrolling, and I don't see any reason to remove them on the first request. but it is meaningless to calculate them on subsequent requests

@s1monw
Copy link
Contributor

s1monw commented Apr 10, 2013

well even if it's not a common usecase it seems pretty straight forward form an API perspective to just return the facets on the first response no?

@seiflotfy
Copy link

What is the status of this issue?

@clintongormley
Copy link
Author

I don't think anything further has happened, but I think a reasonable resolution would be to disable facets/aggs on subsequent scroll requests.

/cc @uboness

@clintongormley
Copy link
Author

In fact, in master, the call to _scroll with facets now throws an NPE:

Failed to execute query phase
org.elasticsearch.search.query.QueryPhaseExecutionException: [test][2]: query[ConstantScore(cache(_type:foo))],from[1],size[1]: Query Failed [Failed to execute main query]
    at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:162)
    at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:282)
    at org.elasticsearch.search.action.SearchServiceTransportAction$9.call(SearchServiceTransportAction.java:274)
    at org.elasticsearch.search.action.SearchServiceTransportAction$9.call(SearchServiceTransportAction.java:271)
    at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:517)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.NullPointerException
    at org.elasticsearch.search.facet.terms.longs.TermsLongFacetExecutor$StaticAggregatorValueProc.onValue(TermsLongFacetExecutor.java:220)
    at org.elasticsearch.search.facet.LongFacetAggregatorBase.onDoc(LongFacetAggregatorBase.java:35)
    at org.elasticsearch.search.facet.terms.longs.TermsLongFacetExecutor$Collector.collect(TermsLongFacetExecutor.java:157)
    at org.elasticsearch.common.lucene.MultiCollector.collect(MultiCollector.java:60)
    at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:193)
    at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:163)
    at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)
    at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:175)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:491)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:448)
    at org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:243)
    at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:135)
    ... 7 more

Aggregations return the correct counts on each call to scroll, but it still makes more sense to me to only return them on the first request.

@jpountz
Copy link
Contributor

jpountz commented Jul 4, 2014

Aggregations return the correct counts on each call to scroll, but it still makes more sense to me to only return them on the first request.

Not sure if the bug still exists in recent releases (#5799 might have fixed it), but I think this behavior has probably been the cause of #3801.

@jpountz jpountz removed the adoptme label Aug 28, 2014
@jpountz jpountz self-assigned this Aug 28, 2014
jpountz added a commit to jpountz/elasticsearch that referenced this issue Sep 3, 2014
Aggregations are collection-wide statistics so they would always be the same.
In order to save CPU/bandwidth, we can just return them on the first page.

Same as elastic#1642 but for aggregations.
jpountz added a commit that referenced this issue Sep 3, 2014
Aggregations are collection-wide statistics so they would always be the same.
In order to save CPU/bandwidth, we can just return them on the first page.

Same as #1642 but for aggregations.
jpountz added a commit that referenced this issue Sep 3, 2014
Aggregations are collection-wide statistics so they would always be the same.
In order to save CPU/bandwidth, we can just return them on the first page.

Same as #1642 but for aggregations.
@jpountz
Copy link
Contributor

jpountz commented Sep 3, 2014

Closing this issue as facets are deprecated and the issue has been fixed for aggregations via #7497.

jpountz added a commit that referenced this issue Sep 8, 2014
Aggregations are collection-wide statistics so they would always be the same.
In order to save CPU/bandwidth, we can just return them on the first page.

Same as #1642 but for aggregations.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants