Skip to content

[Spec lib improvements] Overhaul Spec execution pipeline #140

Open
@faultyserver

Description

@faultyserver

This is part of a family of issues relating to Spec library improvements.

Currently, when an AssertionFailure is raised, the Spec library immediately halts, shows the failure, then exits with a non-zero exit status.

While this can be useful in some cases (e.g., long running suites), in most cases, it's nice to continue running the remaining specs and then show all of the failures at the end.

I think the proper way to implement this change would be to split spec execution into two phases. The first phase is the "definition" phase, where the Spec code is read, and each it block is saved as a SingleSpec instance that captures the block to execute, and added to a list of Specs to run.

The second phase iterates that list, executing each block and tracking the result as either a pass or failure. Specs that fail would get added to a separate list of failing specs, and after all specs have finished executing, the failed specs would be printed out in a failure summary.

Using this approach will eventually allow us to filter Specs based on command line options (line numbers, tags, previously-failed specs, etc.).

Metadata

Metadata

Assignees

No one assigned

    Labels

    feature-requestAny request for a new feature of the language. Includes both syntax and library features.stdlibAny issue relating to the standard library (Myst code) of Myst.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions