Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runner: Test run output on console/terminal is not really useful #78

Open
jenisys opened this issue Jun 8, 2023 · 1 comment
Open

Comments

@jenisys
Copy link

jenisys commented Jun 8, 2023

Describe the bug
The output of the parallel test run is not really useful.
It contains:

  • Diagnostics: Not really readable
  • Unreadable JSON output of failures (on scenarios marked with @xfail ), not beautified
  • Partial result summaries that can not be related to concrete features/scenarios, like:
...
Took 0m0.000s
1 feature passed, 0 failed, 0 skipped
6 scenarios passed, 0 failed, 2 skipped
0 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.000s
1 feature passed, 0 failed, 0 skipped
6 scenarios passed, 0 failed, 2 skipped
0 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.000s
1 feature passed, 0 failed, 0 skipped
6 scenarios passed, 0 failed, 2 skipped
0 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.000s
...

To Reproduce
Steps to reproduce the behavior:

  1. Run 'behavex' against the features/ directory of the behave repo

Expected behavior

  • Show a progress-bar how many files/features/scenarios are processed (and which feature failed w/ how many scenarios)
EXAMPLE OUTPUT:

  -- STATE: In progress during test run by using a progress-bar
   25% <########...............> features/some.feature  (duration: 32s)

  -- HINT: On completed features of this test-run on files
  PASSED  features/good.feature  (6 scenarios: 6 passed, 0 failed)
  FAILED  features/bad_1.feature (5 scenarios: 4 passed, 1 failed)

   -- ALTERNATIVE: Both parts combined 
   --   Finished feature-file status overrides the progress-bar, and progress-bar uses new/next-line
  PASSED  features/good.feature  (6 scenarios: 6 passed, 0 failed)
  FAILED  features/bad_1.feature (5 scenarios: 4 passed, 1 failed)
   25% <########...............> features/some.feature  (duration: 32s)

... final summary in the end (when everything is finished; already provided)
  • OPTIONAL: Use greed/red coloring on result on the progress-bar and completed results if possible
  • Hide the garbage output that is unreadable
  • OR: Make the garbage output readable

REASON:

  • The test reports should contain this kind of output already

Version Info:

  • behavex v2.0.2 (current HEAD of repository)
@etherfurnace
Copy link

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants