Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Spec lib improvements] Overhaul Spec execution pipeline #140

Open
faultyserver opened this issue Jan 31, 2018 · 0 comments
Open

[Spec lib improvements] Overhaul Spec execution pipeline #140

faultyserver opened this issue Jan 31, 2018 · 0 comments
Labels
feature-request Any request for a new feature of the language. Includes both syntax and library features. stdlib Any issue relating to the standard library (Myst code) of Myst.

Comments

@faultyserver
Copy link
Member

This is part of a family of issues relating to Spec library improvements.

Currently, when an AssertionFailure is raised, the Spec library immediately halts, shows the failure, then exits with a non-zero exit status.

While this can be useful in some cases (e.g., long running suites), in most cases, it's nice to continue running the remaining specs and then show all of the failures at the end.

I think the proper way to implement this change would be to split spec execution into two phases. The first phase is the "definition" phase, where the Spec code is read, and each it block is saved as a SingleSpec instance that captures the block to execute, and added to a list of Specs to run.

The second phase iterates that list, executing each block and tracking the result as either a pass or failure. Specs that fail would get added to a separate list of failing specs, and after all specs have finished executing, the failed specs would be printed out in a failure summary.

Using this approach will eventually allow us to filter Specs based on command line options (line numbers, tags, previously-failed specs, etc.).

@faultyserver faultyserver added stdlib Any issue relating to the standard library (Myst code) of Myst. feature-request Any request for a new feature of the language. Includes both syntax and library features. labels Jan 31, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request Any request for a new feature of the language. Includes both syntax and library features. stdlib Any issue relating to the standard library (Myst code) of Myst.
Projects
None yet
Development

No branches or pull requests

1 participant