Allow request-coordinating Processor implementations to pass the TCK by akarnokd · Pull Request #414 · reactive-streams/reactive-streams-jvm

akarnokd · 2017-11-23T18:31:14Z

This PR fixes the TCK to support request-coordinating Processor implementations. These kinds of Processors may either

coordinate the request amounts of their Subscribers and request only from upstream when all Subscribers requested something; or
coordinate emissions, requesting a bounded amount upfront from the upstream and then emitting only when all Subscribers have requested something.

From the downstream Subscribers' perspective, both manifest as lack of emissions.

The IdentityProcessorVerification.doesCoordinatedEmission() added will affect

required_spec104_mustCallOnErrorOnAllItsSubscribersIfItEncountersANonRecoverableError
required_mustRequestFromUpstreamForElementsThatHaveBeenRequestedLongAgo

By adjusting the test request pattern to request on both Subscribers before asserting on events received.

The LockstepProcessorTest contains the LockstepProcessor implementation which coordinates emissions when there are multiple Subscribers.

Replaces #287.
Original issue #284.

viktorklang · 2017-11-24T14:53:48Z

+   * request amounts and only delivers onNext signals if all Subscribers have
+   * indicated (via their Subscription#request(long)) they are ready to receive elements.
+   */
+  public boolean doesCoordinatedEmission() {


Since most devs probably won't ever notice this method, or know that they should enable/disable it—what do you think about including a hint about it in case the tests which use it fail?

(Sorry, this comment was on the wrong line before)

Documenting it more in the README of the TCK could be a good step?

I can add some description to it in a couple of hours.

viktorklang · 2017-11-26T14:27:28Z

+            sub1.request(1);
+            sub2.request(1);
+
+            sendError(ex);


Perhaps put sendError(ex); after the branch (since it is the last statement in each "leg" of the branch.

viktorklang · 2017-11-26T14:28:14Z

+            expectNextElement(sub1, z);
+            expectNextElement(sub2, z);
+
+            if (totalRequests == 3) {


Since this is the same behavior on each side of the branch, perhaps put it after the branch?

viktorklang · 2017-11-27T08:54:37Z

+   * <p>
+   * The test is not executed if {@link IdentityProcessorVerification#maxSupportedSubscribers()} is less than 2.
+   * <p>
+   * If this test fails, the following could be checked within the {@code Publisher} implementation:


If we wanted to include a hint about doesCoordinatedEmission either when running the test case, or when the testcase fails, what would you propose?

We could introduce an overload to expectNextElement that takes an error message that hints about that method.

That sounds very good! (the hint override)

viktorklang · 2017-11-27T08:56:17Z

+            expectNextElement(sub2, x);
+
+            sub1.request(1);
+            sub2.request(1);


Shouldn't expectRequest() work after these two?

Yes, it should. I'll update.

viktorklang · 2017-11-27T09:02:28Z

@akarnokd Thanks for taking a stab at this issue!

@ktoso, @reactive-streams/contributors Anyone else have feedback on this?

viktorklang · 2017-11-27T09:26:12Z

+            final T z = sendNextTFromUpstream();
+            expectNextElement(sub1, z,
+                    "If the Processor coordinates requests/emissions when having multiple Subscribers"
+                            + " at once, please override doesCoordinatedEmission() in this "


"please override doesCoordinatedEmission() to return true in this"

ktoso · 2017-11-27T09:30:40Z

I do want to review this, please don't merge yet.
Esp. I'm worried about the use of coordinated omission -- I don't think this is an instance of that, and the term is used a bit misleadingly. Coordinated omission is about a measurement problem where the measuring-party is not measuring the real latencies due to backpressure from stressed system, I don't think that a Processor that handles many subscribers sis the same thing.

I want to review the thing in depth to be able to reply more precisely though

viktorklang · 2017-11-27T09:31:58Z

@ktoso Are you conflating "coordinated omission" with "coordinated emission"?

In any case, perhaps there's a better term, I'm all for improvements when it comes to names :)

ktoso · 2017-11-27T09:35:30Z

Not conflating, but misreading what was in the PR it seems 😉
Yeah emission is fine, but perhaps a bit too close to that one...

Anyway, want to review this but completely under-water today, hope to get to it tomorrow

viktorklang · 2017-11-27T09:38:20Z

@ktoso No worries! :-)

ktoso · 2017-11-29T13:47:02Z

+      if (!elem.equals(expected)) {
+        env.flop(String.format("Received `onNext(%s)` on downstream but expected `onNext(%s)`", elem, expected));
+      }
+    }


viktorklang · 2017-11-29T14:16:40Z

@ktoso @reactive-streams/contributors Ready for merge?

ktoso

Looks good as far as I can see, thanks for implementing it @akarnokd 👍

viktorklang · 2017-11-30T15:43:45Z

VERY nice, @akarnokd! Merging!

akarnokd added 2 commits November 23, 2017 19:20

Allow request-coordinating Processor impls to pass the TCK

2b1c7e2

Add test expl. for the use of doesCoordinatedEmission

09fe154

viktorklang reviewed Nov 24, 2017

View reviewed changes

viktorklang reviewed Nov 26, 2017

View reviewed changes

Remove code duplications

62d385c

viktorklang reviewed Nov 27, 2017

View reviewed changes

Error message about doesCoordinatedEmission.

984063e

viktorklang reviewed Nov 27, 2017

View reviewed changes

Adjust doesCoordinatedEmission error message

721002d

ktoso reviewed Nov 29, 2017

View reviewed changes

viktorklang approved these changes Nov 29, 2017

View reviewed changes

ktoso approved these changes Nov 29, 2017

View reviewed changes

Fix javadoc, add Readme.md entry

14e821b

viktorklang merged commit 43da5a9 into reactive-streams:master Nov 30, 2017

viktorklang added this to the 1.0.2 milestone Nov 30, 2017

viktorklang mentioned this pull request Dec 1, 2017

Clarification on multiple subscribers needed .. #284

Closed

akarnokd mentioned this pull request Dec 4, 2017

+tck #284 support "demand when all downstreams demand" Processor in TCK #287

Closed

Conversation

akarnokd commented Nov 23, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

viktorklang Nov 27, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

viktorklang commented Nov 27, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ktoso commented Nov 27, 2017

Uh oh!

viktorklang commented Nov 27, 2017

Uh oh!

ktoso commented Nov 27, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

viktorklang commented Nov 27, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

viktorklang commented Nov 29, 2017

Uh oh!

ktoso left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

viktorklang commented Nov 30, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

akarnokd commented Nov 23, 2017 •

edited

Loading

viktorklang Nov 27, 2017 •

edited

Loading

ktoso commented Nov 27, 2017 •

edited

Loading

ktoso left a comment •

edited

Loading