Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow_partial_search_results setting must be passed as query-string parameter #41223

Closed
danielsnider opened this Issue Apr 15, 2019 · 10 comments

Comments

Projects
None yet
4 participants
@danielsnider
Copy link

commented Apr 15, 2019

The allow_partial_search_results setting must be passed as query-string parameter but this isn't possible with the tool that I'm using (ReactiveSearch). A global config option would be better for me or an option in the request body.

@danielsnider danielsnider changed the title allow_partial_search_results setting must be passed as query-string parameters allow_partial_search_results setting must be passed as query-string parameter Apr 15, 2019

@matriv

This comment has been minimized.

Copy link
Contributor

commented Apr 16, 2019

Hello @danielsnider,

The default value for this param can also be said globally through a cluster setting: search.default_allow_partial_results. Please check the documentation here.

@elasticmachine

This comment has been minimized.

Copy link

commented Apr 16, 2019

@danielsnider

This comment has been minimized.

Copy link
Author

commented Apr 16, 2019

@matriv Thanks so much! Do you know why it's not working?

Setting is set:

$ curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
{
    "persistent" : {
        "search.default_allow_partial_results" : "false"
    }
}
'
$ curl -X GET "localhost:9200/_cluster/settings"
{"persistent":{"search":{"default_allow_partial_results":"false"}},"transient":{}}

Then I GET string query with intentional syntax error "heart(":

{"query_string":{"fields":["*"],"query":"heart("}}

But I receive response 200 with even though the body is a clear error 😢:

{
  "responses": [
    {
      "error": {
        "root_cause": [
          {
            "type": "query_shard_exception",
            "reason": "Failed to parse query [heart(]",
            "index_uuid": "QrWhvbq2RIWijCAoXfFKhw",
            "index": "image"
          }
        ],
        "type": "search_phase_execution_exception",
        "reason": "all shards failed",
        "phase": "query",
        "grouped": true,
        "failed_shards": [
          {
            "shard": 0,
...

How can we make the 200 response be the 400 or 500 error that is should be?
Many thanks! This is a major blocker for me.

@jimczi

This comment has been minimized.

Copy link
Member

commented Apr 16, 2019

@danielsnider this looks like a bug but I cannot reproduce, can you post the entire response and the version that you're using ?

@danielsnider

This comment has been minimized.

Copy link
Author

commented Apr 16, 2019

@jimczi Thanks for taking a look.

I've tried ElasticSearch 6.7.1 and 6.6.0. This is how I start elastic:

sudo docker run -d --name elasticsearch -p 9200:9200 -p 9300:9300 -e ES_JAVA_OPTS="-Xms16g -Xmx16g" -v `pwd`/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /home/ubuntu/esdata:/usr/share/elasticsearch/data docker.elastic.co/elasticsearch/elasticsearch:6.6.0

Full search request:

Request URL: http://192.168.136.128:9200/image/_msearch?
Request Method: POST
Status Code: 200 OK
Remote Address: 192.168.136.128:9200
Referrer Policy: no-referrer-when-downgrade
accept: application/json
content-type: application/x-ndjson
Origin: http://192.168.136.128:3000
Referer: http://192.168.136.128:3000/
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
x-requested-with: 771100

{"preference":"modality-list"}
{"query":{"bool":{"must":[{"bool":{"must":[{"query_string":{"fields":["*"],"query":"heart("}}]}}]}},"highlight":{"pre_tags":["<mark>"],"post_tags":["</mark>"],"fields":{}},"size":0,"_source":{"includes":["*"],"excludes":[]},"from":0,"aggs":{"Modality.raw":{"terms":{"field":"Modality.raw","size":20,"order":{"_count":"desc"}}}}}
{"preference":"bodypart-list"}
{"query":{"bool":{"must":[{"bool":{"must":[{"query_string":{"fields":["*"],"query":"heart("}}]}}]}},"highlight":{"pre_tags":["<mark>"],"post_tags":["</mark>"],"fields":{}},"size":0,"_source":{"includes":["*"],"excludes":[]},"from":0,"aggs":{"BodyPartExamined.raw":{"terms":{"field":"BodyPartExamined.raw","size":100,"order":{"_count":"desc"}}}}}
{"preference":"gender-list"}
{"query":{"bool":{"must":[{"bool":{"must":[{"query_string":{"fields":["*"],"query":"heart("}}]}}]}},"highlight":{"pre_tags":["<mark>"],"post_tags":["</mark>"],"fields":{}},"size":0,"_source":{"includes":["*"],"excludes":[]},"from":0,"aggs":{"PatientSex.raw":{"terms":{"field":"PatientSex.raw","size":20,"order":{"_count":"desc"}}}}}
{"preference":"tagCloud"}
{"query":{"bool":{"must":[{"bool":{"must":[{"query_string":{"fields":["*"],"query":"heart("}}]}}]}},"highlight":{"pre_tags":["<mark>"],"post_tags":["</mark>"],"fields":{}},"size":0,"_source":{"includes":["*"],"excludes":[]},"from":0,"aggs":{"descriptions.raw":{"terms":{"field":"descriptions.raw","size":200,"order":{"_term":"asc"}}}}}
{"preference":"results"}
{"query":{"bool":{"must":[{"bool":{"must":[{"query_string":{"fields":["*"],"query":"heart("}}]}}]}},"highlight":{"pre_tags":["<mark>"],"post_tags":["</mark>"],"fields":{}},"size":12,"_source":{"includes":["*"],"excludes":[]},"from":0}

Full 200 response:

access-control-allow-credentials: true
access-control-allow-origin: *
content-encoding: gzip
content-length: 538
content-type: application/json; charset=UTF-8
Warning: 299 Elasticsearch-6.7.1-2f32220 "Deprecated aggregation order key [_term] used, replaced by [_key]"

{"responses":[{"error":{"root_cause":[{"type":"query_shard_exception","reason":"Failed to parse query [heart(]","index_uuid":"FqZfR415QIuajKOY32gXYg","index":"image"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"image","node":"X6uClAg5SimP2fxx3L872g","reason":{"type":"query_shard_exception","reason":"Failed to parse query [heart(]","index_uuid":"FqZfR415QIuajKOY32gXYg","index":"image","caused_by":{"type":"parse_exception","reason":"Cannot parse 'heart(': Encountered \"<EOF>\" at line 1, column 6.\nWas expecting one of:\n    <NOT> ...\n    \"+\" ...\n    \"-\" ...\n    <BAREOPER> ...\n    \"(\" ...\n    \"*\" ...\n    <QUOTED> ...\n    <TERM> ...\n    <PREFIXTERM> ...\n    <WILDTERM> ...\n    <REGEXPTERM> ...\n    \"[\" ...\n    \"{\" ...\n    <NUMBER> ...\n    <TERM> ...\n    ","caused_by":{"type":"parse_exception","reason":"Encountered \"<EOF>\" at line 1, column 6.\nWas expecting one of:\n    <NOT> ...\n    \"+\" ...\n    \"-\" ...\n    <BAREOPER> ...\n    \"(\" ...\n    \"*\" ...\n    <QUOTED> ...\n    <TERM> ...\n    <PREFIXTERM> ...\n    <WILDTERM> ...\n    <REGEXPTERM> ...\n    \"[\" ...\n    \"{\" ...\n    <NUMBER> ...\n    <TERM> ...\n    "}}}}]},"status":400},{"error":{"root_cause":[{"type":"query_shard_exception","reason":"Failed to parse query [heart(]","index_uuid":"FqZfR415QIuajKOY32gXYg","index":"image"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"image","node":"X6uClAg5SimP2fxx3L872g","reason":{"type":"query_shard_exception","reason":"Failed to parse query [heart(]","index_uuid":"FqZfR415QIuajKOY32gXYg","index":"image","caused_by":{"type":"parse_exception","reason":"Cannot parse 'heart(': Encountered \"<EOF>\" at line 1, column 6.\nWas expecting one of:\n    <NOT> ...\n    \"+\" ...\n    \"-\" ...\n    <BAREOPER> ...\n    \"(\" ...\n    \"*\" ...\n    <QUOTED> ...\n    <TERM> ...\n    <PREFIXTERM> ...\n    <WILDTERM> ...\n    <REGEXPTERM> ...\n    \"[\" ...\n    \"{\" ...\n    <NUMBER> ...\n    <TERM> ...\n    ","caused_by":{"type":"parse_exception","reason":"Encountered \"<EOF>\" at line 1, column 6.\nWas expecting one of:\n    <NOT> ...\n    \"+\" ...\n    \"-\" ...\n    <BAREOPER> ...\n    \"(\" ...\n    \"*\" ...\n    <QUOTED> ...\n    <TERM> ...\n    <PREFIXTERM> ...\n    <WILDTERM> ...\n    <REGEXPTERM> ...\n    \"[\" ...\n    \"{\" ...\n    <NUMBER> ...\n    <TERM> ...\n    "}}}}]},"status":400},{"error":{"root_cause":[{"type":"query_shard_exception","reason":"Failed to parse query [heart(]","index_uuid":"FqZfR415QIuajKOY32gXYg","index":"image"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"image","node":"X6uClAg5SimP2fxx3L872g","reason":{"type":"query_shard_exception","reason":"Failed to parse query [heart(]","index_uuid":"FqZfR415QIuajKOY32gXYg","index":"image","caused_by":{"type":"parse_exception","reason":"Cannot parse 'heart(': Encountered \"<EOF>\" at line 1, column 6.\nWas expecting one of:\n    <NOT> ...\n    \"+\" ...\n    \"-\" ...\n    <BAREOPER> ...\n    \"(\" ...\n    \"*\" ...\n    <QUOTED> ...\n    <TERM> ...\n    <PREFIXTERM> ...\n    <WILDTERM> ...\n    <REGEXPTERM> ...\n    \"[\" ...\n    \"{\" ...\n    <NUMBER> ...\n    <TERM> ...\n    ","caused_by":{"type":"parse_exception","reason":"Encountered \"<EOF>\" at line 1, column 6.\nWas expecting one of:\n    <NOT> ...\n    \"+\" ...\n    \"-\" ...\n    <BAREOPER> ...\n    \"(\" ...\n    \"*\" ...\n    <QUOTED> ...\n    <TERM> ...\n    <PREFIXTERM> ...\n    <WILDTERM> ...\n    <REGEXPTERM> ...\n    \"[\" ...\n    \"{\" ...\n    <NUMBER> ...\n    <TERM> ...\n    "}}}}]},"status":400},{"error":{"root_cause":[{"type":"query_shard_exception","reason":"Failed to parse query [heart(]","index_uuid":"FqZfR415QIuajKOY32gXYg","index":"image"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"image","node":"X6uClAg5SimP2fxx3L872g","reason":{"type":"query_shard_exception","reason":"Failed to parse query [heart(]","index_uuid":"FqZfR415QIuajKOY32gXYg","index":"image","caused_by":{"type":"parse_exception","reason":"Cannot parse 'heart(': Encountered \"<EOF>\" at line 1, column 6.\nWas expecting one of:\n    <NOT> ...\n    \"+\" ...\n    \"-\" ...\n    <BAREOPER> ...\n    \"(\" ...\n    \"*\" ...\n    <QUOTED> ...\n    <TERM> ...\n    <PREFIXTERM> ...\n    <WILDTERM> ...\n    <REGEXPTERM> ...\n    \"[\" ...\n    \"{\" ...\n    <NUMBER> ...\n    <TERM> ...\n    ","caused_by":{"type":"parse_exception","reason":"Encountered \"<EOF>\" at line 1, column 6.\nWas expecting one of:\n    <NOT> ...\n    \"+\" ...\n    \"-\" ...\n    <BAREOPER> ...\n    \"(\" ...\n    \"*\" ...\n    <QUOTED> ...\n    <TERM> ...\n    <PREFIXTERM> ...\n    <WILDTERM> ...\n    <REGEXPTERM> ...\n    \"[\" ...\n    \"{\" ...\n    <NUMBER> ...\n    <TERM> ...\n    "}}}}]},"status":400},{"error":{"root_cause":[{"type":"query_shard_exception","reason":"Failed to parse query [heart(]","index_uuid":"FqZfR415QIuajKOY32gXYg","index":"image"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"image","node":"X6uClAg5SimP2fxx3L872g","reason":{"type":"query_shard_exception","reason":"Failed to parse query [heart(]","index_uuid":"FqZfR415QIuajKOY32gXYg","index":"image","caused_by":{"type":"parse_exception","reason":"Cannot parse 'heart(': Encountered \"<EOF>\" at line 1, column 6.\nWas expecting one of:\n    <NOT> ...\n    \"+\" ...\n    \"-\" ...\n    <BAREOPER> ...\n    \"(\" ...\n    \"*\" ...\n    <QUOTED> ...\n    <TERM> ...\n    <PREFIXTERM> ...\n    <WILDTERM> ...\n    <REGEXPTERM> ...\n    \"[\" ...\n    \"{\" ...\n    <NUMBER> ...\n    <TERM> ...\n    ","caused_by":{"type":"parse_exception","reason":"Encountered \"<EOF>\" at line 1, column 6.\nWas expecting one of:\n    <NOT> ...\n    \"+\" ...\n    \"-\" ...\n    <BAREOPER> ...\n    \"(\" ...\n    \"*\" ...\n    <QUOTED> ...\n    <TERM> ...\n    <PREFIXTERM> ...\n    <WILDTERM> ...\n    <REGEXPTERM> ...\n    \"[\" ...\n    \"{\" ...\n    <NUMBER> ...\n    <TERM> ...\n    "}}}}]},"status":400}]}

Any ideas?

@danielsnider

This comment has been minimized.

Copy link
Author

commented Apr 16, 2019

Same problem on both my laptop and production server. I can see in the elasticsearch log that the setting is getting set, but it's still not working.

[2019-04-16T17:14:02,049][INFO ][o.e.c.s.ClusterSettings  ] [1D5Dwct] updating [search.default_allow_partial_results] from [true] to [false]
@jimczi

This comment has been minimized.

Copy link
Member

commented Apr 16, 2019

Ok I understand now, this is because you look at the status of the _msearch response. As you can see in the response array, the statuses for the internal requests are all set to 400. We discussed here if we should convey the error status from the internal responses to the main _msearch response but agreed that we should keep the current behavior of returning an error only if the parsing of the _msearch fails and not if the actual parsing of the internal queries failed in shards. We have the mechanism for _bulk queries where it is required to check the status of the individual requests to find the errors.

This is a major blocker for me.

Can you explain why ? The errors are correctly reported in the response. Also note that setting search.default_allow_partial_results has nothing to do with this behavior. If query parsing fails on all shards, the response for the individual request will always be a 400.

@danielsnider

This comment has been minimized.

Copy link
Author

commented Apr 16, 2019

Crazy, so there's no way option to get an error HTTP status even when the response body is full of 400?
This is a major blocker for me because these errors then do not bubble up to users, they see a loading screen forever. The tool I build on ReactiveSearch doesn't know what to do when it gets a 200 HTTP response who's body is all errors and no data. Could there be another setting? Standards say that 200 means "the response will contain an entity corresponding to the requested resource" and ElasticSearch is not behaving that way.

@jimczi

This comment has been minimized.

Copy link
Member

commented Apr 16, 2019

Well the arguments why we return a 200 are all in the issue I linked. Your example is simple because you have the same query repeated multiple times but what should be the behavior if you have different types of errors in the response ? The reasoning for returning a 200 is that the problem is not on the _msearch but on the individual requests, you'd get an error if the _msearch query is malformed or if the dispatching of the queries failed in some way.
I hope you don't mind if I close this issue as a duplicate of #29169 and don't hesitate to add your thoughts there if you have counter arguments to the ones already exposed.

@jimczi jimczi closed this Apr 16, 2019

@danielsnider

This comment has been minimized.

Copy link
Author

commented Apr 16, 2019

Thanks for helping me!! ❤

I agree with your explanation. I still think a new option would be very useful to enable an HTTP error status if any query had an error in a _msearch. ElasticSearch is breaking the HTTP standard! 🤮

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.