Make the high-level REST client skip types for document deletion. #34041

jpountz · 2018-09-25T10:22:27Z

I'm starting with delete since it is a bit simpler than other APIs, but we
should eventually do the same with all other APIs that are being replaced with
a typeless version (get, put_mapping, search, etc.).

Relates #15613

I'm starting with delete since it is a bit simpler than other APIs, but we should eventually do the same with all other APIs that are being replaced with a typeless version (get, put_mapping, search, etc.). Relates elastic#15613

elasticmachine · 2018-09-25T10:22:29Z

Pinging @elastic/es-core-infra

javanna · 2018-09-25T11:56:53Z

Is the plan to migrate to new requests/responses as part of this change, or is that part necessary to not also affect transport client? I wonder because that's a breaking change for users, and I wonder what the plan is for all other requests/responses. Also, if we go ahead this way, we may want to remove response parsing code from DeleteResponse and add unit tests for the new response class, as well as unit test the request, its validation bits etc.

jpountz · 2018-09-25T13:18:02Z

Thanks for the feedback @javanna, this is the kind of comments that I was looking for. :)

Is the plan to migrate to new requests/responses as part of this change, or is that part necessary to not also affect transport client?

It would be best if we could migrate the existing request/response classes, but unfortunately this is hard to do because types are only deprecated on the REST layer, not removed yet so the transport client needs to retain a way to set types on request objects, and there is already an IndexRequest(String,String) constructor where the second string is the type and the id autogenerated, while we'd want this constructor to take the id as a second parameter, so we can't nicely migrate users from the old API to the new one in a backward-compatible way with nice deprecation warnings. My understanding is that we are generally more keen to break the Java API than the REST layer since the compiler can help figure out what is wrong. Since decoupling the client from the server seems to be something we would like to do in general, creating new request/response classes felt like the right thing to do.

if we go ahead this way, we may want to remove response parsing code from DeleteResponse and add unit tests for the new response class, as well as unit test the request, its validation bits etc.

+1 We can't remove parsing logic from the old DeleteResponse class yet because of bulk requests, but I agree we should do this eventually. I'll add tests to the request/response classes.

javanna · 2018-09-25T13:33:23Z

My understanding is that we are generally more keen to break the Java API than the REST layer since the compiler can help figure out what is wrong.

True, but if breaking was almost required for the transport client given the internal nature of the exposed classes, I hope at some point the REST client becomes stable. Obviously we are not there yet. Let's document this clearly and make sure that users know what to expect, it is true that the compiler helps but it's also good for users to know what they have to do to migrate before they go and do it. In a way it's good to show the direction, the delete API was added with 5.6 or earlier, and it will be updated to its own classes in 7.0 which is not bad at all.

jpountz · 2018-09-25T16:20:48Z

@javanna I pushed one more commit to add tests and document the breaking change.

javanna · 2018-09-25T18:11:26Z

client/rest-high-level/src/test/java/org/elasticsearch/client/DeleteResponseTests.java

+import java.io.IOException;
+import java.util.Collections;
+
+public class DeleteResponseTests extends ESTestCase {


testing these new classes requires to repeat a lot of similar checks, that would be avoided with a test class like AbstractXContentTestCase. But this new class does not implement toXContent hence it's hard to do that. I have faced something similar in #33176 and added a possible approach. The bigger problem though is that we need to make sure that all the response parser support unknown fields (for forward compatibility) and test that, which is something that is very easy to forget if we extend ESTestCase. @hub-cap said he needs to take some time to think about this testing problem caused by using different objects in the client.

#33176

Thanks for linking.

The bigger problem though is that we need to make sure that all the response parser support unknown fields

Indeed I actually explicitly disabled support for unknown fields, which is a mistake now that I read your comment.

javanna · 2018-09-25T18:12:01Z

docs/reference/migration/migrate_7_0/java.asciidoc

+// Modify this statement once all document APIs have been migrated
+`DeleteRequest` and `DeleteResponse` moved to the `org.elasticsearch.client` package
+and do not reference types anymore. Under the hood, the high-level REST client


We also renamed all of their methods to make them proper getters I think.

javanna · 2018-09-25T18:13:33Z

Thanks for updating! I am good with this but there's a question mark on testing which requires @hub-cap 's thoughts, and his opinion on the overall direction would be great to have too.

jpountz · 2018-09-26T09:20:07Z

Thanks again for the feedback @javanna. No hurry on my end, I'm actually happy we're trying to think about how things should work since we'll have to do something similar for all other APIs that we are making type-less (put_mapping, get_mapping, index, get, put, bulk, update, etc.) and delete is probably one of the simpler ones.

hub-cap · 2018-09-28T18:33:04Z

WRT the testing, we will have a conclusion very soon. Stay tuned.

hub-cap · 2018-09-28T18:35:35Z

WRT breaking the existing API, im ok also with this until we get to 7.0, since its not technically released. Ill let others tell me this is a wrong approach tho cc @jasontedor for weighing in.

hub-cap · 2018-09-28T18:33:56Z

client/rest-high-level/src/test/java/org/elasticsearch/client/RequestConvertersTests.java

+     * @deprecated Use {@link RefreshPolicy} rather than {@link WriteRequest.RefreshPolicy}.
+     */
+    @Deprecated
+    private static void setRandomLegacyRefreshPolicy(Consumer<WriteRequest.RefreshPolicy> setter, Map<String, String> expectedParams) {


Did this get deprecated and renamed? Should we be doing both?

javanna · 2018-10-01T08:33:56Z

WRT breaking the existing API, im ok also with this until we get to 7.0, since its not technically released.

@hub-cap just one small question, what do you mean with "its not technically released"?

hub-cap · 2018-10-02T15:13:00Z

@javanna correct me if im wrong, but I assumed that we were "releasing" this in 7.0, and things prior to 7.0 were ok to change. If this is not the case then I definitely need to adjust my way of thinking!!! :)

javanna · 2018-10-02T15:15:23Z

You should try and tell users that @hub-cap :) I guess for them , what they can use is released?

hub-cap · 2018-10-02T15:43:47Z

haha, yea thats a fair point. I guess usable vs guaranteed back compat are 2 different things. Ill figure out what is the actual answer here.

javanna · 2018-10-02T15:46:39Z

I think we can and have to break things, but let's be aware of when we do so and try to document such changes and help users to migrate? After all they have just made the effort of moving to our new client :)

jtibshirani · 2018-11-28T22:13:59Z

@jpountz would it be okay to close this, as we've decided to take a different approach to types deprecation and the Java HLRC? In particular, we are preferring to add deprecations to existing methods, as opposed to creating new requests + responses for every affected API.

jpountz · 2019-01-15T16:53:53Z

Absolutely.

jpountz added 2 commits September 25, 2018 12:19

Remove equals/hashcode.

76aff80

jpountz added >enhancement v7.0.0 :Core/Java High Level REST Client labels Sep 25, 2018

jpountz requested a review from hub-cap September 25, 2018 10:23

Fix javadocs.

667d8f3

Add docs and tests.

b6873d2

javanna reviewed Sep 25, 2018

View reviewed changes

hub-cap reviewed Sep 28, 2018

View reviewed changes

jtibshirani mentioned this pull request Nov 9, 2018

Deprecate types in count and msearch. #35421

Merged

jpountz closed this Jan 15, 2019

jpountz deleted the typeless_rest_delete branch January 15, 2019 16:54

jpountz removed the v7.0.0 label Jan 15, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make the high-level REST client skip types for document deletion. #34041

Make the high-level REST client skip types for document deletion. #34041

jpountz commented Sep 25, 2018

elasticmachine commented Sep 25, 2018

javanna commented Sep 25, 2018

jpountz commented Sep 25, 2018

javanna commented Sep 25, 2018

jpountz commented Sep 25, 2018

javanna Sep 25, 2018

jpountz Sep 26, 2018

javanna Sep 25, 2018

javanna commented Sep 25, 2018

jpountz commented Sep 26, 2018

hub-cap commented Sep 28, 2018

hub-cap commented Sep 28, 2018

hub-cap Sep 28, 2018

javanna commented Oct 1, 2018

hub-cap commented Oct 2, 2018

javanna commented Oct 2, 2018

hub-cap commented Oct 2, 2018

javanna commented Oct 2, 2018

jtibshirani commented Nov 28, 2018

jpountz commented Jan 15, 2019

Make the high-level REST client skip types for document deletion. #34041

Make the high-level REST client skip types for document deletion. #34041

Conversation

jpountz commented Sep 25, 2018

elasticmachine commented Sep 25, 2018

javanna commented Sep 25, 2018

jpountz commented Sep 25, 2018

javanna commented Sep 25, 2018

jpountz commented Sep 25, 2018

javanna Sep 25, 2018

Choose a reason for hiding this comment

jpountz Sep 26, 2018

Choose a reason for hiding this comment

javanna Sep 25, 2018

Choose a reason for hiding this comment

javanna commented Sep 25, 2018

jpountz commented Sep 26, 2018

hub-cap commented Sep 28, 2018

hub-cap commented Sep 28, 2018

hub-cap Sep 28, 2018

Choose a reason for hiding this comment

javanna commented Oct 1, 2018

hub-cap commented Oct 2, 2018

javanna commented Oct 2, 2018

hub-cap commented Oct 2, 2018

javanna commented Oct 2, 2018

jtibshirani commented Nov 28, 2018

jpountz commented Jan 15, 2019