Test output gets mixed up when run in parallel

[f1ames]

👓 What did you see?

When tests are run with --parallel flag, the output (for example, steps description) gets mixed.

When run without --parallel option:

With --parallel you can see that scenarios (and steps) are printed as they come from different workers and get mixed:

✅ What did you expect to see?

I would expect that each Scenario (or entire Feature, so could be multiple scenarios) would be printed as a whole, one after another.

📦 Which version are you using?

"@cucumber/cucumber": "^7.3.2",
"@cucumber/pretty-formatter": "^1.0.0-alpha.1",

🔬 How could we reproduce it?

I'm coming from Scramjet and we are using Cucumber for running BDD tests in multiple repos. The screenshots are based on tests from https://github.com/scramjetorg/transform-hub. You can reproduce it by running tests from this repo:

git clone https://github.com/scramjetorg/transform-hub.git
cd transform-hub
yarn install && yarn build:all
yarn test:bdd-ci --name="E2E" --parallel 4

Please keep in mind you may need some stuff from setup steps too.

🤔 Anything else?

There is already a similar issue reported - #859 but it's related to nightwatch-* 3rd-party which we are not using, so, for us, it seems to be Cucumber (or it's formatter) related issue.

I'm happy to provide more details/context if needed.

[aurelien-reeves]

How does look the output without the pretty-formatter, but with a built-in formatter?
If the output of built-in formatters seems correct, then I suggest to open an issue in the pretty-formatter here: https://github.com/cucumber/cucumber-js-pretty-formatter/

[mpkorstanje]

I would expect that each Scenario (or entire Feature, so could be multiple scenarios) would be printed as a whole, one after another.

That's pretty much impossible to combine with realtime feedback. If you don't need realtime feedback have you considered using other formatters such as the html formatter?

[f1ames]

How does look the output without the pretty-formatter, but with a built-in formatter?
If the output of built-in formatters seems correct, then I suggest to open an issue in the pretty-formatter here: https://github.com/cucumber/cucumber-js-pretty-formatter/

@aurelien-reeves It seems to be working fine from what I see. And from what I understand, there is no built-in formatter which simply prints out step by step so it seems to confirm that it's related to pretty-formatter. And HTML formatter has valid output for --parallel > 1 too 👍

I'm closing this issue then and moving it to https://github.com/cucumber/cucumber-js-pretty-formatter/. Thanks for your help!

I would expect that each Scenario (or entire Feature, so could be multiple scenarios) would be printed as a whole, one after another.

That's pretty much impossible to combine with realtime feedback.

@mpkorstanje In realtime (meaning print as fast as something is logged) no, but I can imagine with some kind of buffering like, wait for all test steps and print when test is done it would be doable and kind of close to realtime. But then it would be printing not step by step but entire scenarios 🤔 I get your point on this 👍

If you don't need realtime feedback have you considered using other formatters such as the html formatter?

That's a reasonable suggestion 👍 I mean, we need some text-like output on CI so HTML won't work here, but maybe something which shows only failed tests at the end of a run.