Concurrent scenarios?

[kleijnweb]

I realize this was already discussed (#75), but I just want to re-raise the issue because I think this is missing option. I understand the rationale behind feature-level concurrency, but there are (IMO) even more reasons to do it on the scenario level. If you need things so spin up you can do it before running features. Running scenarios in parallel reduces test time and may uncover any concurrency issues, specifically those that come from sharing resources between scenarios (ie: it reflects real-world usage more).

Can we not have it optional?

[l3pp4rd]

this is concerning problem, and though I agree it should be on scenario level, but before feature event is commonly used to start something for the whole feature runtime. That would break backward compatibility and having an option is confusing and not consistent.

[kleijnweb]

Having a command line option would not break BC and has to be supplied explicitly. Your scenarios will just fine without the option, they may not with it, but that is be expected when you explicitly tell godog to run your scenarios in parallel. Assuming the option is aptly named, I don't see that leading to confusion.

So start goroutine for every feature, in each: run before hooks, run each scenario (with flag: in a goroutine, then wait for all to finish), run after hooks.

[l3pp4rd]

the parallelization this way is possible indeed. at the moment scenarios of the feature are order randomized using random option, that option could be used instead, though in order to replicate the run, in case if there was a problem with state between scenarios, would be difficult. At the moment --random option accepts the seed which can be used in order to replicate the sequence. Have you seen the random option? maybe that is enough, otherwise it may make sense to have parallelization at scenario level as you describe, but in that case execution order cannot be known and replicated easily. The random option, allows to have the replication seed.

[l3pp4rd]

Hi, that may be useful for many users, if you have questions, let me know

[lonnblad]

@mpkorstanje @jaysonesmith how do you feel regarding this one?

I'm thinking mostly regarding how we want to handle BeforeFeature and AfterFeature.

[mxygem]

I think how kleijnweb mentioned it in their last comment sounds good to me. Is that doable?

[lonnblad]

yep, it would be doable

would it be consistent with other Cucumber implementations? do we care about that? :)

[mpkorstanje]

I don't think any other Cucumber implementation supports feature hooks. The other implementations also don't share state between scenarios so there is little utility.

I do know that JUnit 5 has a model that allows tests to be organized in a hierarchy. This supports state isolation, parallel execution, and feature hooks. But it's anything but simple.

https://junit.org/junit5/docs/5.3.0/api/org/junit/platform/engine/support/hierarchical/package-summary.html

[lonnblad]

I would be okey with removing feature hooks, but I'm not sure how popular it is, l3pp4rd commented that it was "commonly used".

[mxygem]

I don't like the idea of continuing to support feature hooks, especially since the other implementations don't. We're working to bring the go implementation more in line and this seems counter to that. I think feature hooks also probably exist almost as a crutch for big heavy setup which can enable things on the automation side of implementation?

[mxygem]

I have an old baseline example of a godog suite here and similarly in another suite would do larger setup in the Feature Context.

[lonnblad]

I have branches for concurrency support of the pretty formatter and the events formatter, will probably work on Cucumber next week. (depends on gherkin-go v9.2.0)

Then all the provided formatters would have concurrency support on Feature level.