Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High Level REST client fails to parse query results with certain script fields #28380

Closed
wfhartford opened this issue Jan 25, 2018 · 1 comment
Closed

Comments

@wfhartford
Copy link

@wfhartford wfhartford commented Jan 25, 2018

Elasticsearch version (bin/elasticsearch --version):
Elasticsearch Version: Version: 6.1.2, Build: 5b1fea5/2018-01-10T02:35:59.208Z, JVM: 1.8.0_151
REST High Level Java Client Verstion: 6.1.2

Plugins installed: X-Pack

JVM version (java -version):
Client Version:

java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

Server Version:

openjdk version "1.8.0_151"
OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

OS version (uname -a if on a Unix-like system):
Client Version:

Linux wrk-it-303 4.13.0-31-generic #34~16.04.1-Ubuntu SMP Fri Jan 19 17:11:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Server Version:

Linux rd-es-02 4.4.0-112-generic #135-Ubuntu SMP Fri Jan 19 11:48:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Description of the problem including expected versus actual behavior:
When using the high level rest client to query and retrieve script field values containing an array of anything other than strings, the HTTP response is 200, but an exception is thrown parsing the response. A typical error log is:

Steps to reproduce:

  1. Create or use any index with at least one document,
  2. Construct a query which will produce at least one result,
  3. Execute that query include a scripted field which produces an object, an array, or null.

This sample class shows the exceptions encountered using several trivial scripts; modify the constants at the top so that any document is found.

import java.io.IOException;
import java.util.HashMap;

import org.apache.http.HttpHost;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.script.Script;
import org.elasticsearch.script.ScriptType;

public final class HighLevelClientScripts {
  private static final String ELASTICSEARCH_HOST = "localhost";
  private static final String TARGET_INDEX = "time-series";
  private static final String DOCUMENT_ID = "NDcnKHrpsDNKN5pbJZGl6SOidqlrtDQ5IEKjdAd2p1E=";
  private static final String[] TEST_SCRIPTS = new String[]{
      "null",
      "new HashMap()",
      "new String[]{}"
  };

  public static void main(String[] args) throws IOException {
    try (final RestHighLevelClient client = new RestHighLevelClient(
        RestClient.builder(new HttpHost(ELASTICSEARCH_HOST, 9200))
    )) {
      for (final String script : TEST_SCRIPTS) {
        final SearchRequest request = new SearchRequest(TARGET_INDEX);

        request.source()
            .query(QueryBuilders.idsQuery().addIds(DOCUMENT_ID))
            .scriptField("result", new Script(ScriptType.INLINE, "painless", script, new HashMap<>()));

        try {
          client.search(request);
          System.out.println("Script '" + script + "' succeeded!");
        }
        catch (IOException e) {
          System.out.println("Script '" + script + "' caused an exception");
          e.printStackTrace(System.out);
        }
      }
    }
  }
}

Output of the sample class:

Script 'null' caused an exception
java.io.IOException: Unable to parse response body for Response{requestLine=GET /time-series/_search?typed_keys=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&search_type=query_then_fetch&batched_reduce_size=512 HTTP/1.1, host=http://rd-es-01:9200, response=HTTP/1.1 200 OK}
	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:462)
	at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:429)
	at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:368)
	at com.ze.zemart.elasticsearchclient.HighLevelClientScripts.main(HighLevelClientScripts.java:36)
Caused by: ParsingException[[innerHitParser] failed to parse field [fields]]; nested: ParsingException[Failed to parse object: unexpected token [VALUE_NULL] found];
	at org.elasticsearch.common.xcontent.ObjectParser.parseValue(ObjectParser.java:316)
	at org.elasticsearch.common.xcontent.ObjectParser.parseSub(ObjectParser.java:325)
	at org.elasticsearch.common.xcontent.ObjectParser.parse(ObjectParser.java:169)
	at org.elasticsearch.common.xcontent.ObjectParser.apply(ObjectParser.java:183)
	at org.elasticsearch.search.SearchHit.fromXContent(SearchHit.java:500)
	at org.elasticsearch.search.SearchHits.fromXContent(SearchHits.java:150)
	at org.elasticsearch.action.search.SearchResponse.fromXContent(SearchResponse.java:281)
	at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:573)
	at org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAndParseEntity$2(RestHighLevelClient.java:429)
	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:460)
	... 3 more
Caused by: ParsingException[Failed to parse object: unexpected token [VALUE_NULL] found]
	at org.elasticsearch.common.xcontent.XContentParserUtils.throwUnknownToken(XContentParserUtils.java:67)
	at org.elasticsearch.common.xcontent.XContentParserUtils.parseStoredFieldsValue(XContentParserUtils.java:108)
	at org.elasticsearch.common.document.DocumentField.fromXContent(DocumentField.java:142)
	at org.elasticsearch.search.SearchHit.parseFields(SearchHit.java:610)
	at org.elasticsearch.search.SearchHit.lambda$declareInnerHitsParseFields$13(SearchHit.java:522)
	at org.elasticsearch.common.xcontent.AbstractObjectParser.lambda$declareObject$1(AbstractObjectParser.java:148)
	at org.elasticsearch.common.xcontent.ObjectParser.lambda$declareField$1(ObjectParser.java:214)
	at org.elasticsearch.common.xcontent.ObjectParser.parseValue(ObjectParser.java:314)
	... 12 more
Script 'new HashMap()' caused an exception
java.io.IOException: Unable to parse response body for Response{requestLine=GET /time-series/_search?typed_keys=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&search_type=query_then_fetch&batched_reduce_size=512 HTTP/1.1, host=http://rd-es-01:9200, response=HTTP/1.1 200 OK}
	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:462)
	at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:429)
	at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:368)
	at com.ze.zemart.elasticsearchclient.HighLevelClientScripts.main(HighLevelClientScripts.java:36)
Caused by: ParsingException[[innerHitParser] failed to parse field [fields]]; nested: ParsingException[Failed to parse object: unexpected token [START_OBJECT] found];
	at org.elasticsearch.common.xcontent.ObjectParser.parseValue(ObjectParser.java:316)
	at org.elasticsearch.common.xcontent.ObjectParser.parseSub(ObjectParser.java:325)
	at org.elasticsearch.common.xcontent.ObjectParser.parse(ObjectParser.java:169)
	at org.elasticsearch.common.xcontent.ObjectParser.apply(ObjectParser.java:183)
	at org.elasticsearch.search.SearchHit.fromXContent(SearchHit.java:500)
	at org.elasticsearch.search.SearchHits.fromXContent(SearchHits.java:150)
	at org.elasticsearch.action.search.SearchResponse.fromXContent(SearchResponse.java:281)
	at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:573)
	at org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAndParseEntity$2(RestHighLevelClient.java:429)
	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:460)
	... 3 more
Caused by: ParsingException[Failed to parse object: unexpected token [START_OBJECT] found]
	at org.elasticsearch.common.xcontent.XContentParserUtils.throwUnknownToken(XContentParserUtils.java:67)
	at org.elasticsearch.common.xcontent.XContentParserUtils.parseStoredFieldsValue(XContentParserUtils.java:108)
	at org.elasticsearch.common.document.DocumentField.fromXContent(DocumentField.java:142)
	at org.elasticsearch.search.SearchHit.parseFields(SearchHit.java:610)
	at org.elasticsearch.search.SearchHit.lambda$declareInnerHitsParseFields$13(SearchHit.java:522)
	at org.elasticsearch.common.xcontent.AbstractObjectParser.lambda$declareObject$1(AbstractObjectParser.java:148)
	at org.elasticsearch.common.xcontent.ObjectParser.lambda$declareField$1(ObjectParser.java:214)
	at org.elasticsearch.common.xcontent.ObjectParser.parseValue(ObjectParser.java:314)
	... 12 more
Script 'new String[]{}' caused an exception
java.io.IOException: Unable to parse response body for Response{requestLine=GET /time-series/_search?typed_keys=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&search_type=query_then_fetch&batched_reduce_size=512 HTTP/1.1, host=http://rd-es-01:9200, response=HTTP/1.1 200 OK}
	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:462)
	at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:429)
	at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:368)
	at com.ze.zemart.elasticsearchclient.HighLevelClientScripts.main(HighLevelClientScripts.java:36)
Caused by: ParsingException[[innerHitParser] failed to parse field [fields]]; nested: ParsingException[Failed to parse object: unexpected token [START_ARRAY] found];
	at org.elasticsearch.common.xcontent.ObjectParser.parseValue(ObjectParser.java:316)
	at org.elasticsearch.common.xcontent.ObjectParser.parseSub(ObjectParser.java:325)
	at org.elasticsearch.common.xcontent.ObjectParser.parse(ObjectParser.java:169)
	at org.elasticsearch.common.xcontent.ObjectParser.apply(ObjectParser.java:183)
	at org.elasticsearch.search.SearchHit.fromXContent(SearchHit.java:500)
	at org.elasticsearch.search.SearchHits.fromXContent(SearchHits.java:150)
	at org.elasticsearch.action.search.SearchResponse.fromXContent(SearchResponse.java:281)
	at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:573)
	at org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAndParseEntity$2(RestHighLevelClient.java:429)
	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:460)
	... 3 more
Caused by: ParsingException[Failed to parse object: unexpected token [START_ARRAY] found]
	at org.elasticsearch.common.xcontent.XContentParserUtils.throwUnknownToken(XContentParserUtils.java:67)
	at org.elasticsearch.common.xcontent.XContentParserUtils.parseStoredFieldsValue(XContentParserUtils.java:108)
	at org.elasticsearch.common.document.DocumentField.fromXContent(DocumentField.java:142)
	at org.elasticsearch.search.SearchHit.parseFields(SearchHit.java:610)
	at org.elasticsearch.search.SearchHit.lambda$declareInnerHitsParseFields$13(SearchHit.java:522)
	at org.elasticsearch.common.xcontent.AbstractObjectParser.lambda$declareObject$1(AbstractObjectParser.java:148)
	at org.elasticsearch.common.xcontent.ObjectParser.lambda$declareField$1(ObjectParser.java:214)
	at org.elasticsearch.common.xcontent.ObjectParser.parseValue(ObjectParser.java:314)
	... 12 more
@wfhartford wfhartford changed the title High Level REST client can only parse strings in arrays in field results High Level REST client fails to parse query results with certain script fields Jan 25, 2018
@javanna javanna self-assigned this Jan 26, 2018
javanna added a commit to javanna/elasticsearch that referenced this issue Jan 26, 2018
Script fields can get a bit more complicated than just stored fields. A script can return null, an object and also an array. Extended parsing to support such valid values. Also renamed util method from `parseStoredFieldsValue` to `parseFieldsValue` given that it can parse stored fields but also script fields, anything that's returned as `fields`.

Closes elastic#28380
javanna added a commit that referenced this issue Jan 30, 2018
Script fields can get a bit more complicated than just stored fields. A script can return null, an object and also an array. Extended parsing to support such valid values. Also renamed util method from `parseStoredFieldsValue` to `parseFieldsValue` given that it can parse stored fields but also script fields, anything that's returned as `fields`.

Closes #28380
javanna added a commit to javanna/elasticsearch that referenced this issue Jan 31, 2018
Script fields can get a bit more complicated than just stored fields. A script can return null, an object and also an array. Extended parsing to support such valid values. Also renamed util method from `parseStoredFieldsValue` to `parseFieldsValue` given that it can parse stored fields but also script fields, anything that's returned as `fields`.

Closes elastic#28380
@javanna

This comment has been minimized.

Copy link
Member

@javanna javanna commented Jan 31, 2018

thanks a lot for raising this and for the clear recreation @wfhartford !

javanna added a commit that referenced this issue Jan 31, 2018
Script fields can get a bit more complicated than just stored fields. A script can return null, an object and also an array. Extended parsing to support such valid values. Also renamed util method from `parseStoredFieldsValue` to `parseFieldsValue` given that it can parse stored fields but also script fields, anything that's returned as `fields`.

Closes #28380
javanna added a commit that referenced this issue Jan 31, 2018
Script fields can get a bit more complicated than just stored fields. A script can return null, an object and also an array. Extended parsing to support such valid values. Also renamed util method from `parseStoredFieldsValue` to `parseFieldsValue` given that it can parse stored fields but also script fields, anything that's returned as `fields`.

Closes #28380
javanna added a commit that referenced this issue Feb 12, 2018
Script fields can get a bit more complicated than just stored fields. A script can return null, an object and also an array. Extended parsing to support such valid values. Also renamed util method from `parseStoredFieldsValue` to `parseFieldsValue` given that it can parse stored fields but also script fields, anything that's returned as `fields`.

Closes #28380
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.