Randomize Content-Type in REST tests #23245

javanna · 2017-02-17T22:15:52Z

This PR introduces randomization of the content type for requests sent as part of our REST tests. Requests can be sent in any supported format, and responses can be read from any of the supported formats too, according to the returned Content-Type header.

This new randomization revealed a couple of problems:

update indices settings only supports yaml or json (Update indices settings api to support CBOR and SMILE format #23309)
a yaml tests for filters aggregations using wrapper query builder would only work with json or yaml content types, not with binary formats (which don't write binary inner objects as base64)
Scripts and templates should be converted to json while parsing them (Convert script/template objects to json format internally #23308)

KodrAus · 2017-02-19T21:03:35Z

So... Why randomise it instead of always testing all content types?

javanna · 2017-02-20T08:33:16Z

hi @KodrAus , this suite runs around 700 tests and takes a few minutes to run. The content type that is most important is json, the one that we've been using up until now. Running all the same tests with yaml, cbor and smile too would take too long, that is why randomizing helps. I think I will also go back and give a bit more importance to json though, given that it is the most widely used format.

KodrAus · 2017-02-20T09:01:17Z

@javanna I thought that might be the case :) Thanks for taking the time to clarify!

jaymode

This is great; thanks for doing this @javanna. I left a few small comments but other than that LGTM

jaymode · 2017-02-22T13:48:47Z

core/src/main/java/org/elasticsearch/rest/action/admin/indices/RestUpdateSettingsAction.java

-                String key = entry.getKey();
-                String value = entry.getValue();
+        Map<String, Object> settings = new HashMap<>();
+        if (request.hasContent()) {


this is a nit IMO, but might be cleaner if the request#contentOrSourceParamParserOrNull method is used?

jaymode · 2017-02-22T13:52:09Z

modules/lang-painless/src/test/resources/rest-api-spec/test/painless/50_script_doc_values.yaml

-                            binary:
-                                type: binary
-                                doc_values: true
+                            #binary:


should we remove all of these commented lines since it seems they have been moved to another test?

leftover, will remove

jaymode · 2017-02-22T13:55:31Z

...framework/src/main/java/org/elasticsearch/test/rest/yaml/ClientYamlTestExecutionContext.java

-        if (bodies.size() == 1) {
-            return bodyAsString(stash.replaceStashedValues(bodies.get(0)));
+        XContentType xContentType = null;
+        String contentType = headers.get("Content-Type");


I think we may want to document that the headers used in the yaml tests are case sensitive or fix the map to be case insensitive?

I was hoping you wouldn't catch this :)

javanna · 2017-02-22T14:06:39Z

heya @jaymode thanks for the review, FYI I took out the script changes and opened #23308 . I am contemplating taking also out other changes outside of tests to their own PRs.

…in REST tests

This allows to set content-type together with the body itself. At the moment it is always json, but this change allows makes it easier to randomize it later

…g tests This test makes little sense when sent from the REST layer, as WrapperQueryBuilder is supposed to be used from the Java api. Also, providing the inner query as base64 string will work only for string formats and break for binary formats like SMILE and CBOR, whcih doesn't play well with randomizing content type in our REST tests

…tent-Type header explicitly

Rather test that the size is present and greather than zero. The actual size depends on the content-type, which is randomized.

…rmats

javanna added review >test Issues or PRs that are addressing/adding tests v5.4.0 v6.0.0-alpha1 labels Feb 17, 2017

javanna requested a review from jaymode February 17, 2017 22:15

javanna changed the title ~~Test/randomize xcontent type~~ Randomize Content-Type in REST tests Feb 17, 2017

javanna force-pushed the test/randomize_xcontent_type branch from a5b829d to 407457c Compare February 18, 2017 09:28

javanna added WIP and removed review labels Feb 19, 2017

javanna force-pushed the test/randomize_xcontent_type branch 6 times, most recently from 7904c42 to fc2d893 Compare February 21, 2017 14:08

jaymode approved these changes Feb 22, 2017

View reviewed changes

This was referenced Feb 22, 2017

Convert script/template objects to json format internally #23308

Merged

Update indices settings api to support CBOR and SMILE format #23309

Merged

javanna force-pushed the test/randomize_xcontent_type branch from fc2d893 to 830464f Compare February 22, 2017 14:24

javanna added review and removed WIP labels Feb 22, 2017

javanna force-pushed the test/randomize_xcontent_type branch 5 times, most recently from c983838 to 832ef6b Compare February 23, 2017 14:53

javanna force-pushed the test/randomize_xcontent_type branch 4 times, most recently from 0a8b356 to d4d8cec Compare February 27, 2017 09:57

javanna added 8 commits February 27, 2017 10:57

[TEST] Remove content type auto-detection while parsing request body …

c665c10

…in REST tests

[TEST] create HttpEntity earlier in REST tests

d194ef7

This allows to set content-type together with the body itself. At the moment it is always json, but this change allows makes it easier to randomize it later

[TEST] move test for binary field to specific test file that sets Con…

5132f42

…tent-Type header explicitly

[TEST] don't check exact size in mapper-size yaml test

13c52e4

Rather test that the size is present and greather than zero. The actual size depends on the content-type, which is randomized.

[TEST] add support for binary responses to REST tests infra

848f128

[TEST] randomize request content_type between all of the supported fo…

16056fd

…rmats

[TEST] make headers case-insensitive when running yaml tests

0025724

javanna force-pushed the test/randomize_xcontent_type branch from d4d8cec to 0025724 Compare February 27, 2017 09:57

javanna merged commit 756e26c into elastic:master Feb 27, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Randomize Content-Type in REST tests #23245

Randomize Content-Type in REST tests #23245

javanna commented Feb 17, 2017 •

edited

Loading

KodrAus commented Feb 19, 2017

javanna commented Feb 20, 2017

KodrAus commented Feb 20, 2017

jaymode left a comment

jaymode Feb 22, 2017

jaymode Feb 22, 2017

javanna Feb 22, 2017

jaymode Feb 22, 2017

javanna Feb 22, 2017

javanna commented Feb 22, 2017

Randomize Content-Type in REST tests #23245

Randomize Content-Type in REST tests #23245

Conversation

javanna commented Feb 17, 2017 • edited Loading

KodrAus commented Feb 19, 2017

javanna commented Feb 20, 2017

KodrAus commented Feb 20, 2017

jaymode left a comment

Choose a reason for hiding this comment

jaymode Feb 22, 2017

Choose a reason for hiding this comment

jaymode Feb 22, 2017

Choose a reason for hiding this comment

javanna Feb 22, 2017

Choose a reason for hiding this comment

jaymode Feb 22, 2017

Choose a reason for hiding this comment

javanna Feb 22, 2017

Choose a reason for hiding this comment

javanna commented Feb 22, 2017

javanna commented Feb 17, 2017 •

edited

Loading