Test Explorer #144

Closed
endragor opened this Issue Jun 5, 2015 · 28 comments

Projects

None yet

7 participants

@endragor
endragor commented Jun 5, 2015

Would like to see a view with time statistics per file/test. This will help to identify tests that are taking longer than I would expect.

@ArtemGovorov ArtemGovorov changed the title from Aggregated time statistics to Test Explorer Jun 9, 2015
@ArtemGovorov
Member

@endragor I'll steal the issue to extend the scope a bit if you don't mind.

It'd be nice to have a filterable and searchable tree of all tests, where apart from the execution time, other test data would be listed.

@usefulthink

👍

@brettjacobson

I was just curious if there is progress on the Test Explorer?

@ArtemGovorov
Member

@brettjacobson There's been some progress, but not very substantial yet. We do get a lot of requests for the feature though from IntelliJ users and Atom users as well.

I have been thinking of perhaps providing a couple of reports (Test Explorer and Coverage) that could be opened from an IDE but in a browser. That would be the easiest and the fastest way for us to implement and release it, as opposed to doing 3 IDE specific implementations.

Even though HTML report will not be as 'integrated' as the rest of wallaby features, these reports is not something that is used as frequently as other features, so perhaps a little context switching (from IDE to browser) is not a big deal here.

Your thoughts?

@endragor

That works for me.

@fvanwijk

I think it is a good start, since we are used to get a HTML coverage report with Karma/Istanbul as well (there is no good IDE integration yet). However, I really like to eventually have IDE integration for coverage etc. because lack of context switches makes Wallaby so powerful.

@brettjacobson

I think an HTML coverage report would be useful, but it would only be useful every once in awhile. A Test explorer window would be useful all the time I think, since it would serve many purposes.

@ArtemGovorov
Member

Thanks for your feedback guys.

@brettjacobson

A Test explorer window would be useful all the time I think, since it would serve many purposes.

How often would you use the window? I understand that sometimes one may need to use it for a few minutes or even more, but working with the window is already a context switching from writing code right? So my question is - would you need to keep the tests tree open all the time while you write code?

@brettjacobson

With NCrunch I keep the ncrunch Tests window open on a secondary monitor. Perhaps NCrunchs' window is valuable to us because our C# unit test suite is much larger (and slower) than our JS test suite, so its advantageous to get progress reports on test runs (even when optimized to run only affected tests).

I definitely use the ncrunch test window as a test code navigation aid. I think I would like to use a wallaby test window for the same purpose

Perhaps you can summarize what possible requirements for a Test Explorer that you've gathered/planned?

@ArtemGovorov
Member

With NCrunch I keep the ncrunch Tests window open on a secondary monitor.

So you wouldn't mind if in wallaby case it was a browser window if it had the features you need?

Here is a summary of features for the Test Explorer that I had in mind (note that the Coverage Report is a separate view with its own set of requirements/features, though some of its data may end up in the Test Explorer):

Minimal set of features for v1:
  • Displaying all tests in a tree (name + execution time).
  • Displaying details of a selected test (execution time, errors if any, console.log output for test).
  • Allowing to jump to the selected test code, its error stack lines and console.log positions.
  • Allowing to run the selected test/suite or all project tests.
Advanced features to add later:
  • Allowing to group/order/filter tests by file/test name, execution time, failed/passed state, etc.
  • Displaying detailed test execution progress and what tests are currently running.

Your thoughts?

@brettjacobson

I think that would be okay, if it was a browser window. Would a browser window in VS still be able to jump to the right place in the code windows??

Otherwise the features sound great!

@ArtemGovorov
Member

Would a browser window in VS still be able to jump to the right place in the code windows??

Yes, any code links inside the browser window will work as if they were in IDE, so clicking on a line link will navigate to the line location in the opened IDE.

@brettjacobson

That sounds like it would meet our needs.

@hellboy81

I'm curious about it. Is there screenshots?

@ArtemGovorov
Member

@hellboy81 will share some as soon as I can

@brettjacobson

any update on this item?

@hellboy81

+infinity

@ArtemGovorov
Member

We are working on it and have made some good progress. I'd say it's about 80% ready.

@hellboy81

Test Explorer teaser trailer needed!

@ArtemGovorov
Member

Disclaimer: it's an early (but already working) prototype and things may (and most likely will) change, but thought I'd still share it with you guys:

screen shot 2016-03-11 at 9 23 20 pm

@fvanwijk

Super cool!
Also add number of ignored tests at the upper right corner?

@brettjacobson

looks awesome! I agree, number of ignored tests would be valuable to see too

@soates
soates commented Apr 6, 2016

Looks very good - exactly what we need.

@soates
soates commented Jun 13, 2016

Any update on when this is due out?

@ArtemGovorov
Member
ArtemGovorov commented Jun 23, 2016 edited

No exact date yet, but we are very close. We decided to experiment with a couple of features that have increased the work scope, here's a sneak peek:
ui

@brettjacobson

Looking great!

@ArtemGovorov
Member
ArtemGovorov commented Jun 23, 2016 edited

Forgot to add to the recording that everything in the test explorer gets updated in realtime as one's making changes to the code. Updated the GIF to demonstrate it.

@ArtemGovorov
Member
ArtemGovorov commented Jul 19, 2016 edited

Introducing Wallaby.js App - realtime bird's eye view of your project's tests connected to your editor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment