Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stack test should continue when a test suite fails #478

Closed
mboes opened this issue Jul 1, 2015 · 2 comments
Closed

stack test should continue when a test suite fails #478

mboes opened this issue Jul 1, 2015 · 2 comments
Assignees
Milestone

Comments

@mboes
Copy link
Contributor

@mboes mboes commented Jul 1, 2015

Rationale:

  • without this users have to manually specify test targets in order to skip the package with the offending test suite. It gets tiring.
  • sometimes test suites are badly written, in which case a non-deterministic failure appears in some package. At this point it's not so helpful for stack to refuse to process the dependent test suites, when in previous invocations it would have.
  • it's consistent with the behaviour of most test frameworks, which run all tests in a test suite regardless of failures.

Variations:

  • don't make this the default, but provide-k and --keep-going flags, à la make.
  • don't process dependent packages, but do continue processing test suites for packages that don't depend on the package whose tests don't pass. Then -k would mean blindly test everything no matter what (provided it builds).
@snoyberg snoyberg self-assigned this Jul 2, 2015
@snoyberg snoyberg added this to the 0.2.0.0 milestone Jul 2, 2015
@snoyberg snoyberg added the ready label Jul 2, 2015
snoyberg added a commit that referenced this issue Jul 3, 2015
@snoyberg
Copy link
Contributor

@snoyberg snoyberg commented Jul 3, 2015

@mboes I've implemented --keep-going with default behavior as described for build, test, and bench. Confirmed?

Loading

@mboes
Copy link
Contributor Author

@mboes mboes commented Jul 3, 2015

Nice!

Loading

@mboes mboes closed this Jul 3, 2015
@snoyberg snoyberg removed the ready label Jul 3, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants