New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show total progress in terminal during testing #2657

Closed
nicoddemus opened this Issue Aug 4, 2017 · 13 comments

Comments

Projects
None yet
5 participants
@nicoddemus
Member

nicoddemus commented Aug 4, 2017

This is a proposal to change the terminal output while running tests to include the total tests ran so far and the total tests still to run, or percentage of the total tests executed so far.

Currently here's the normal pytest output (minus header):

============================= test session starts =============================
collected 1909 items

testing\acceptance_test.py ...............s................................x.............
testing\deprecated_test.py ......
testing\test_argcomplete.py ss
testing\test_assertion.py ........................................................................s.......
<snip>

What I propose is:

============================= test session starts ===============================
collected 1909 items

testing\acceptance_test.py ...............s............................ [44/1909]
                           ....x.............                           [62/1909]
testing\deprecated_test.py ......                                       [68/1909]
testing\test_argcomplete.py ss                                          [70/1909]
testing\test_assertion.py ............................................ [114/1909]
                          ............................s.......         [150/1909] 
<snip>

This would be a better view of the overall progress and how much tests are still left on the run.

Alternatively instead of showing test counts we could just show a percentage of the total run so far:

============================= test session starts ===============================
collected 1909 items

testing\acceptance_test.py ...............s................................ [ 2%]
                           ....x.........                                   [ 3%]
testing\deprecated_test.py ......                                           [ 3%]
testing\test_argcomplete.py ss                                              [ 3%]
testing\test_assertion.py ................................................. [ 5%]
                          ............................s.                    [ 7%] 
<snip>

Thoughts? cc @RonnyPfannschmidt @hpk42 @The-Compiler @hackebrot

Side note

This idea came to be because it Jenkins (it seems) only updates its console view when the process outputs a new line. When running with xdist, pytest only prints a . per test, only printing a new line at the end of the session, so the output windows doesn't update until the end of the run.

Changing the output to the one proposed here would fix that and also provide a nice user experience (IMO).

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Aug 4, 2017

Note: this would work with xdist outside of the box as well. 😉

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Aug 4, 2017

doesnt pytest-sugar already implement a much nicer version of this

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Aug 4, 2017

Indeed, but pytest-sugar only works on Linux. 😬

@The-Compiler

This comment has been minimized.

Member

The-Compiler commented Aug 4, 2017

I like it! pytest-sugar also has a lot of other problems (it being very fancy is also causing issues), and good defaults matter.

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Aug 4, 2017

according to Frozenball/pytest-sugar#49 (comment) it only a codepage issue and solvable by a single command

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Aug 4, 2017

well, im fine with enhancing terminal output, but i do wonder how much harder it will make the removal of the current terminalwriter mess

also note that currently the terminal width determination of pylib is completely broken for output capture situation

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Aug 4, 2017

according to Frozenball/pytest-sugar#49 (comment) it only a codepage issue and solvable by a single command

Didn't know that, thanks for the link!

well, im fine with enhancing terminal output, but i do wonder how much harder it will make the removal of the current terminalwriter mess

AFAIK it won't make things worse (it will depend on tw.write as does all the other code which depends on terminalwriter) but I'm of course willing to abort the idea if the implementation gets too messy.

@nicoddemus nicoddemus self-assigned this Aug 11, 2017

@ApaDoctor

This comment has been minimized.

Contributor

ApaDoctor commented Oct 5, 2017

does anybody works on this issue?
i want to take it

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Oct 5, 2017

@nicoddemus do you have any plans/attempts already started, if thats not the case, i propose we let @ApaDoctor try

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Oct 5, 2017

I have a work in progress at home actually, I don't think it is pushed to me fork though. @ApaDoctor let me push that branch when I get home so you can take a look. This is a little tricky than thought at first because we have to "hook" into terminal's writer to ensure we are tracking the correct number of chars written to the current line so far.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Oct 5, 2017

@Frozenball

This comment has been minimized.

Frozenball commented Dec 12, 2017

This is pretty cool 👍

I think if this existed 4 years ago I would have never started pytest-sugar project.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Dec 12, 2017

Which would be a shame because pytest-sugar terminal is awesome 😁

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment