Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incomplete documentation on stochastic tests in TCK #278

Closed
jroper opened this issue Jun 29, 2015 · 3 comments · Fixed by #443

Comments

@jroper
Copy link
Contributor

commented Jun 29, 2015

From the code:

  /**
   * Override and return {@code true} in order to skip executing tests marked as {@code Stochastic}.
   * Such tests MAY sometimes fail even though the impl
   */
  public boolean skipStochasticTests() {
    return false;
  }

Even though the impl what? Under what circumstances may they fail? I'm never seeing them fail locally, but I see them fail quite frequently in CI, is there anything that should be documented here about the TCK in order for me to determine if the failures are expected or not?

@ktoso

This comment has been minimized.

Copy link
Contributor

commented Jun 29, 2015

Ouch, seems we've cut the comment some time during the development, thanks for noticing! I'll dig out the full comment in a bit, but it wasn't anything very precise.

The only stochastic test we have is stochastic_spec103_mustSignalOnMethodsSequentially, it tries to see if any of the subscriber on* methods is called concurrently. Can you point me to a failure and impl you're getting?

AFAIR if it fails your impl most likely has a problem, just that it does not manifest on your machine.
It's a stochastic test because if this test passes it does not prove correctness of "does not call concurrently", it may have just been lucky.

@viktorklang

This comment has been minimized.

Copy link
Contributor

commented Dec 1, 2017

@jroper Ping

@viktorklang

This comment has been minimized.

Copy link
Contributor

commented Nov 12, 2018

@jroper Annual ping ;)

jroper added a commit to jroper/reactive-streams-jvm that referenced this issue Nov 19, 2018
Fixed javadocs for skipStochasticTests
Fixes reactive-streams#278.

Clarified what the stochastic tests are, and what the flag to skip them
is provided for.
jroper added a commit to jroper/reactive-streams-jvm that referenced this issue Nov 19, 2018
Fixed javadocs for skipStochasticTests
Fixes reactive-streams#278.

This was already fixed in a previous PR, but that PR failed to fix the
javadocs for the IdentityProcessorVerification.
viktorklang added a commit that referenced this issue Nov 19, 2018
Fixed javadocs for skipStochasticTests (#443)
Fixes #278.

This was already fixed in a previous PR, but that PR failed to fix the
javadocs for the IdentityProcessorVerification.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.