Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incomplete documentation on stochastic tests in TCK #278

Closed
jroper opened this issue Jun 29, 2015 · 3 comments
Closed

Incomplete documentation on stochastic tests in TCK #278

jroper opened this issue Jun 29, 2015 · 3 comments

Comments

@jroper
Copy link
Contributor

@jroper jroper commented Jun 29, 2015

From the code:

  /**
   * Override and return {@code true} in order to skip executing tests marked as {@code Stochastic}.
   * Such tests MAY sometimes fail even though the impl
   */
  public boolean skipStochasticTests() {
    return false;
  }

Even though the impl what? Under what circumstances may they fail? I'm never seeing them fail locally, but I see them fail quite frequently in CI, is there anything that should be documented here about the TCK in order for me to determine if the failures are expected or not?

@ktoso
Copy link
Contributor

@ktoso ktoso commented Jun 29, 2015

Ouch, seems we've cut the comment some time during the development, thanks for noticing! I'll dig out the full comment in a bit, but it wasn't anything very precise.

The only stochastic test we have is stochastic_spec103_mustSignalOnMethodsSequentially, it tries to see if any of the subscriber on* methods is called concurrently. Can you point me to a failure and impl you're getting?

AFAIR if it fails your impl most likely has a problem, just that it does not manifest on your machine.
It's a stochastic test because if this test passes it does not prove correctness of "does not call concurrently", it may have just been lucky.

Loading

@viktorklang
Copy link
Contributor

@viktorklang viktorklang commented Dec 1, 2017

@jroper Ping

Loading

@viktorklang
Copy link
Contributor

@viktorklang viktorklang commented Nov 12, 2018

@jroper Annual ping ;)

Loading

jroper added a commit to jroper/reactive-streams-jvm that referenced this issue Nov 19, 2018
Fixes reactive-streams#278.

Clarified what the stochastic tests are, and what the flag to skip them
is provided for.
jroper added a commit to jroper/reactive-streams-jvm that referenced this issue Nov 19, 2018
Fixes reactive-streams#278.

This was already fixed in a previous PR, but that PR failed to fix the
javadocs for the IdentityProcessorVerification.
viktorklang added a commit that referenced this issue Nov 19, 2018
Fixes #278.

This was already fixed in a previous PR, but that PR failed to fix the
javadocs for the IdentityProcessorVerification.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

3 participants