Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test summary during watch mode run #3732

Open
4 tasks done
sethreidnz opened this issue Jul 4, 2023 · 9 comments
Open
4 tasks done

Test summary during watch mode run #3732

sethreidnz opened this issue Jul 4, 2023 · 9 comments
Labels
p2-nice-to-have Not breaking anything but nice to have (priority) pr welcome

Comments

@sethreidnz
Copy link

sethreidnz commented Jul 4, 2023

Clear and concise description of the problem

The proposal is to show a live-updating summary of the current test run instead of the list of tests that have ran and are running. This would provide a consistent UI that the user can see the progress of the run toward finishing while also outputting the error messages from failed tests as fast as possible. I believe this will both improve workflow of developers and improve the perceived performance of Vitest.

I'm happy to contribute and expand on anything here that you would like.

Suggested solution

Instead of showing the list of all tests suites as they are ran as is the current UI, Vitest could instead show a live-updating summary of the current run of tests. This would look somewhat similar to the Jest watch UI and could show things such as:

  • Total tests run
  • Total failures
  • Total passes
  • Total skipped
  • Total tests to run
  • Total time
  • Current status (pass or fail overall)

While making this change it will also be important to ensure that:

  • The UI is interruptible and responsive so that the user can pause execution and change things such as filename pattern
  • Test failures are printed above the summary as soon as they happen so that the user can scroll up and see the first failed test as soon as possible when running multiple test suites.
  • Test console output is also included in the correct order along with the test failures so that the messages can be read for debugging

Alternative

I'm aware there is Vitest UI which I am yet to properly check out. On my current project I have had issues getting it going so haven't had a chance to look at it. Although it may improve on the current watch experience it would still be great to improve the default cli experience.

Additional context

I would love to contribute to this and if you have other better ideas or plans that I've missed then let me know and I could help! I've just migrated a project onto Vite that has more than 10k tests and so far I'm really enjoying Vite!

Validations

@sheremet-va
Copy link
Member

sheremet-va commented Jul 4, 2023

Is it a duplicate of #3549? (but better written)

@sethreidnz
Copy link
Author

Is it a duplicate of #3549? (but better written)

Yes indeed it does look like it. Are you interested in contribution in the area?

@sheremet-va
Copy link
Member

Is it a duplicate of #3549? (but better written)

Yes indeed it does look like it. Are you interested in contribution in the area?

Yes, PR is welcome.

@sethreidnz
Copy link
Author

sethreidnz commented Jul 9, 2023

@sheremet-va I'll have a look at this! I have been given the go ahead to spend a little of my day job time on this so I will first have to upskill on how the project works and then hopefully be able to contribute something! Do you have any suggstions or guidance on learning about the codebase?

@sheremet-va
Copy link
Member

@sheremet-va I'll have a look at this! I have been given the go ahead to spend a little of my day job time on this so I will first have to upskill on how the project works and then hopefully be able to contribute something! Do you have any suggstions or guidance on learning about the codebase?

Renderer that prints test results is located here:

export function renderTree(tasks: Task[], options: ListRendererOptions, level = 0, maxRows?: number): string {

Test summary is printed after all tests finished running, so you will need to figure out a way to print it alongside running tests:

async reportSummary(files: File[], errors: unknown[]) {

@AriPerkkio
Copy link
Member

Test summary during watch mode run

I think this should be based on isTTY instead of --watch/--run.

Even in --run mode this would be really useful, just like Jest does it.

@sheremet-va sheremet-va added the p2-nice-to-have Not breaking anything but nice to have (priority) label Oct 2, 2023
@sethreidnz
Copy link
Author

Hi yall! I really would like to work on this, however we've been finding since moving to vitest that our tests are running about 3x slower (from 25min with jest, to 1h 30m with vitest). We have been investigating all the possible options and what not but we may end up moving away from vitest (which I'm sad about) but that means it's hard for me to justify spending the time on this...

@sheremet-va
Copy link
Member

Hi yall! I really would like to work on this, however we've been finding since moving to vitest that our tests are running about 3x slower (from 25min with jest, to 1h 30m with vitest). We have been investigating all the possible options and what not but we may end up moving away from vitest (which I'm sad about) but that means it's hard for me to justify spending the time on this...

There is a performance regression in 0.34.0 that might cause this. I am currently working on this.

@sheremet-va sheremet-va removed the enhancement New feature or request label Feb 16, 2024
@hi-ogawa
Copy link
Contributor

Just linking what Jest recently did to improve their summary output:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
p2-nice-to-have Not breaking anything but nice to have (priority) pr welcome
Projects
None yet
Development

No branches or pull requests

4 participants