New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Explorer #144

Closed
endragor opened this Issue Jun 5, 2015 · 28 comments

Comments

Projects
None yet
7 participants
@endragor

endragor commented Jun 5, 2015

Would like to see a view with time statistics per file/test. This will help to identify tests that are taking longer than I would expect.

@ArtemGovorov ArtemGovorov changed the title from Aggregated time statistics to Test Explorer Jun 9, 2015

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Jun 9, 2015

Member

@endragor I'll steal the issue to extend the scope a bit if you don't mind.

It'd be nice to have a filterable and searchable tree of all tests, where apart from the execution time, other test data would be listed.

Member

ArtemGovorov commented Jun 9, 2015

@endragor I'll steal the issue to extend the scope a bit if you don't mind.

It'd be nice to have a filterable and searchable tree of all tests, where apart from the execution time, other test data would be listed.

@usefulthink

This comment has been minimized.

Show comment
Hide comment
@usefulthink

usefulthink commented Aug 20, 2015

👍

@brettjacobson

This comment has been minimized.

Show comment
Hide comment
@brettjacobson

brettjacobson Oct 28, 2015

I was just curious if there is progress on the Test Explorer?

brettjacobson commented Oct 28, 2015

I was just curious if there is progress on the Test Explorer?

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Oct 28, 2015

Member

@brettjacobson There's been some progress, but not very substantial yet. We do get a lot of requests for the feature though from IntelliJ users and Atom users as well.

I have been thinking of perhaps providing a couple of reports (Test Explorer and Coverage) that could be opened from an IDE but in a browser. That would be the easiest and the fastest way for us to implement and release it, as opposed to doing 3 IDE specific implementations.

Even though HTML report will not be as 'integrated' as the rest of wallaby features, these reports is not something that is used as frequently as other features, so perhaps a little context switching (from IDE to browser) is not a big deal here.

Your thoughts?

Member

ArtemGovorov commented Oct 28, 2015

@brettjacobson There's been some progress, but not very substantial yet. We do get a lot of requests for the feature though from IntelliJ users and Atom users as well.

I have been thinking of perhaps providing a couple of reports (Test Explorer and Coverage) that could be opened from an IDE but in a browser. That would be the easiest and the fastest way for us to implement and release it, as opposed to doing 3 IDE specific implementations.

Even though HTML report will not be as 'integrated' as the rest of wallaby features, these reports is not something that is used as frequently as other features, so perhaps a little context switching (from IDE to browser) is not a big deal here.

Your thoughts?

@endragor

This comment has been minimized.

Show comment
Hide comment
@endragor

endragor Oct 29, 2015

That works for me.

endragor commented Oct 29, 2015

That works for me.

@fvanwijk

This comment has been minimized.

Show comment
Hide comment
@fvanwijk

fvanwijk Oct 29, 2015

I think it is a good start, since we are used to get a HTML coverage report with Karma/Istanbul as well (there is no good IDE integration yet). However, I really like to eventually have IDE integration for coverage etc. because lack of context switches makes Wallaby so powerful.

fvanwijk commented Oct 29, 2015

I think it is a good start, since we are used to get a HTML coverage report with Karma/Istanbul as well (there is no good IDE integration yet). However, I really like to eventually have IDE integration for coverage etc. because lack of context switches makes Wallaby so powerful.

@brettjacobson

This comment has been minimized.

Show comment
Hide comment
@brettjacobson

brettjacobson Oct 29, 2015

I think an HTML coverage report would be useful, but it would only be useful every once in awhile. A Test explorer window would be useful all the time I think, since it would serve many purposes.

brettjacobson commented Oct 29, 2015

I think an HTML coverage report would be useful, but it would only be useful every once in awhile. A Test explorer window would be useful all the time I think, since it would serve many purposes.

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Oct 29, 2015

Member

Thanks for your feedback guys.

@brettjacobson

A Test explorer window would be useful all the time I think, since it would serve many purposes.

How often would you use the window? I understand that sometimes one may need to use it for a few minutes or even more, but working with the window is already a context switching from writing code right? So my question is - would you need to keep the tests tree open all the time while you write code?

Member

ArtemGovorov commented Oct 29, 2015

Thanks for your feedback guys.

@brettjacobson

A Test explorer window would be useful all the time I think, since it would serve many purposes.

How often would you use the window? I understand that sometimes one may need to use it for a few minutes or even more, but working with the window is already a context switching from writing code right? So my question is - would you need to keep the tests tree open all the time while you write code?

@brettjacobson

This comment has been minimized.

Show comment
Hide comment
@brettjacobson

brettjacobson Oct 31, 2015

With NCrunch I keep the ncrunch Tests window open on a secondary monitor. Perhaps NCrunchs' window is valuable to us because our C# unit test suite is much larger (and slower) than our JS test suite, so its advantageous to get progress reports on test runs (even when optimized to run only affected tests).

I definitely use the ncrunch test window as a test code navigation aid. I think I would like to use a wallaby test window for the same purpose

Perhaps you can summarize what possible requirements for a Test Explorer that you've gathered/planned?

brettjacobson commented Oct 31, 2015

With NCrunch I keep the ncrunch Tests window open on a secondary monitor. Perhaps NCrunchs' window is valuable to us because our C# unit test suite is much larger (and slower) than our JS test suite, so its advantageous to get progress reports on test runs (even when optimized to run only affected tests).

I definitely use the ncrunch test window as a test code navigation aid. I think I would like to use a wallaby test window for the same purpose

Perhaps you can summarize what possible requirements for a Test Explorer that you've gathered/planned?

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Nov 1, 2015

Member

With NCrunch I keep the ncrunch Tests window open on a secondary monitor.

So you wouldn't mind if in wallaby case it was a browser window if it had the features you need?

Here is a summary of features for the Test Explorer that I had in mind (note that the Coverage Report is a separate view with its own set of requirements/features, though some of its data may end up in the Test Explorer):

Minimal set of features for v1:
  • Displaying all tests in a tree (name + execution time).
  • Displaying details of a selected test (execution time, errors if any, console.log output for test).
  • Allowing to jump to the selected test code, its error stack lines and console.log positions.
  • Allowing to run the selected test/suite or all project tests.
Advanced features to add later:
  • Allowing to group/order/filter tests by file/test name, execution time, failed/passed state, etc.
  • Displaying detailed test execution progress and what tests are currently running.

Your thoughts?

Member

ArtemGovorov commented Nov 1, 2015

With NCrunch I keep the ncrunch Tests window open on a secondary monitor.

So you wouldn't mind if in wallaby case it was a browser window if it had the features you need?

Here is a summary of features for the Test Explorer that I had in mind (note that the Coverage Report is a separate view with its own set of requirements/features, though some of its data may end up in the Test Explorer):

Minimal set of features for v1:
  • Displaying all tests in a tree (name + execution time).
  • Displaying details of a selected test (execution time, errors if any, console.log output for test).
  • Allowing to jump to the selected test code, its error stack lines and console.log positions.
  • Allowing to run the selected test/suite or all project tests.
Advanced features to add later:
  • Allowing to group/order/filter tests by file/test name, execution time, failed/passed state, etc.
  • Displaying detailed test execution progress and what tests are currently running.

Your thoughts?

@brettjacobson

This comment has been minimized.

Show comment
Hide comment
@brettjacobson

brettjacobson Nov 1, 2015

I think that would be okay, if it was a browser window. Would a browser window in VS still be able to jump to the right place in the code windows??

Otherwise the features sound great!

brettjacobson commented Nov 1, 2015

I think that would be okay, if it was a browser window. Would a browser window in VS still be able to jump to the right place in the code windows??

Otherwise the features sound great!

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Nov 1, 2015

Member

Would a browser window in VS still be able to jump to the right place in the code windows??

Yes, any code links inside the browser window will work as if they were in IDE, so clicking on a line link will navigate to the line location in the opened IDE.

Member

ArtemGovorov commented Nov 1, 2015

Would a browser window in VS still be able to jump to the right place in the code windows??

Yes, any code links inside the browser window will work as if they were in IDE, so clicking on a line link will navigate to the line location in the opened IDE.

@brettjacobson

This comment has been minimized.

Show comment
Hide comment
@brettjacobson

brettjacobson Nov 2, 2015

That sounds like it would meet our needs.

brettjacobson commented Nov 2, 2015

That sounds like it would meet our needs.

@hellboy81

This comment has been minimized.

Show comment
Hide comment
@hellboy81

hellboy81 Nov 13, 2015

I'm curious about it. Is there screenshots?

hellboy81 commented Nov 13, 2015

I'm curious about it. Is there screenshots?

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Nov 13, 2015

Member

@hellboy81 will share some as soon as I can

Member

ArtemGovorov commented Nov 13, 2015

@hellboy81 will share some as soon as I can

@brettjacobson

This comment has been minimized.

Show comment
Hide comment
@brettjacobson

brettjacobson Mar 3, 2016

any update on this item?

brettjacobson commented Mar 3, 2016

any update on this item?

@hellboy81

This comment has been minimized.

Show comment
Hide comment
@hellboy81

hellboy81 commented Mar 3, 2016

+infinity

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Mar 3, 2016

Member

We are working on it and have made some good progress. I'd say it's about 80% ready.

Member

ArtemGovorov commented Mar 3, 2016

We are working on it and have made some good progress. I'd say it's about 80% ready.

@hellboy81

This comment has been minimized.

Show comment
Hide comment
@hellboy81

hellboy81 Mar 4, 2016

Test Explorer teaser trailer needed!

hellboy81 commented Mar 4, 2016

Test Explorer teaser trailer needed!

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Mar 11, 2016

Member

Disclaimer: it's an early (but already working) prototype and things may (and most likely will) change, but thought I'd still share it with you guys:

screen shot 2016-03-11 at 9 23 20 pm

Member

ArtemGovorov commented Mar 11, 2016

Disclaimer: it's an early (but already working) prototype and things may (and most likely will) change, but thought I'd still share it with you guys:

screen shot 2016-03-11 at 9 23 20 pm

@fvanwijk

This comment has been minimized.

Show comment
Hide comment
@fvanwijk

fvanwijk Mar 11, 2016

Super cool!
Also add number of ignored tests at the upper right corner?

fvanwijk commented Mar 11, 2016

Super cool!
Also add number of ignored tests at the upper right corner?

@brettjacobson

This comment has been minimized.

Show comment
Hide comment
@brettjacobson

brettjacobson Mar 19, 2016

looks awesome! I agree, number of ignored tests would be valuable to see too

brettjacobson commented Mar 19, 2016

looks awesome! I agree, number of ignored tests would be valuable to see too

@soates

This comment has been minimized.

Show comment
Hide comment
@soates

soates Apr 6, 2016

Looks very good - exactly what we need.

soates commented Apr 6, 2016

Looks very good - exactly what we need.

@soates

This comment has been minimized.

Show comment
Hide comment
@soates

soates Jun 13, 2016

Any update on when this is due out?

soates commented Jun 13, 2016

Any update on when this is due out?

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Jun 23, 2016

Member

No exact date yet, but we are very close. We decided to experiment with a couple of features that have increased the work scope, here's a sneak peek:
ui

Member

ArtemGovorov commented Jun 23, 2016

No exact date yet, but we are very close. We decided to experiment with a couple of features that have increased the work scope, here's a sneak peek:
ui

@brettjacobson

This comment has been minimized.

Show comment
Hide comment
@brettjacobson

brettjacobson Jun 23, 2016

Looking great!

brettjacobson commented Jun 23, 2016

Looking great!

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Jun 23, 2016

Member

Forgot to add to the recording that everything in the test explorer gets updated in realtime as one's making changes to the code. Updated the GIF to demonstrate it.

Member

ArtemGovorov commented Jun 23, 2016

Forgot to add to the recording that everything in the test explorer gets updated in realtime as one's making changes to the code. Updated the GIF to demonstrate it.

@ArtemGovorov

This comment has been minimized.

Show comment
Hide comment
@ArtemGovorov

ArtemGovorov Jul 19, 2016

Member

Introducing Wallaby.js App - realtime bird's eye view of your project's tests connected to your editor.

Member

ArtemGovorov commented Jul 19, 2016

Introducing Wallaby.js App - realtime bird's eye view of your project's tests connected to your editor.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment