Add --fail-first flag

[markusa380]

No description provided.

[LukasKalbertodt]

Hey and thanks for the PR! If I'm not mistaken, --fail-first is not a flag in the built-in libtest right? So this would be an additional feature of libtest-mimic, right? Generally I try to avoid those, as then there is no end to what useful features one could add to this library. Do you have a special reason to add this? What's your use case?

[markusa380]

I understand your point, yes.
We are using this feature in our company's project, as we use libtest-mimic to run a large test suite for our query engine.
We want this test suite to stop running on the first error in CI to save considerable time and resources.

[LukasKalbertodt]

I have to reject this feature request at this time. As I said: this crate aims to be a direct replacement for the built-in test harness, with basically no additional features. Otherwise, the scope could explode, as there are always interesting features one can add.

As for your use case: you can fairly easily implement a fail-first approach on top of this crate. You pass runner functions to libtest_mimic::run, so you can simply have some boolean flag failed and check that before actually running the test. It's not as nice as if it was built in, but it's still doable.