support the official vscode test API and Test Explorer (v1.59)

[connectdotz]

The official test api and explorer are finally ready and should be released in v1.59. This PR aimed to support the test explorer via the new test API so we can offer the long-requested feature:

The explorer support is driven by the existing engine (mainly from TestResultProvider), therefore, completely backward compatible and all config and functionality behave exactly as before. The code change is mainly in the new test-provider folder, plus some minor code changes in the existing code to support more interactive test configurations from the explorer and notify the test-provider when jest test results changed.

rc.1:
vscode-jest-4.1.0-rc.1.vsix.zip

updated rc.2:
vscode-jest-4.1.0-rc.2.vsix.zip

update rc3:
vscode-jest-4.1.0-rc.3.vsix.zip

[coveralls]

Pull Request Test Coverage Report for Build 1141539487

• 544 of 558 (97.49%) changed or added relevant lines in 14 files are covered.
• 1 unchanged line in 1 file lost coverage.
• Overall coverage increased (+0.2%) to 94.245%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/test-provider/test-provider.ts	86	88	97.73%
src/JestExt/process-listeners.ts	20	23	86.96%
src/JestExt/core.ts	57	61	93.44%
src/test-provider/test-item-data.ts	258	263	98.1%

Files with Coverage Reduction	New Missed Lines	%
src/TestResults/match-by-context.ts	1	94.44%

Totals
Change from base Build 980:	0.2%
Covered Lines:	2459
Relevant Lines:	2557

---

💛 - Coveralls

[connectdotz]

FYI, CI will fail until @types/vscode 1.59 releases as there is a breaking change for StatusBar.tooltip type. In order to work with the proposed API, vscode recommends removing @types/vscode and download the typing file directly (see typings folder). But I did not modify CI script to perform this since this is just a transient situation.

[connectdotz]

ok, 1.59 has been released today, the extension should work in the official vscode now. @orta @stephtr do you guys have some cycle to take a look and play with it? Please let me know if you have any comments, thanks.

[distante]

Works pretty nice on a newly updated vscode! I hope this gets released to the official channel soon. 👍

[distante]

@connectdotz I have notice something

When I save an spec file and it auto runs I see the ? below the red/green test status in the file gutter.

It also does not change the status in the test explorer until the test are done.

[connectdotz]

@distante your observation is accurate. The current sync point (between the jest runner and the explorer) is indeed when the test result is available, i.e. run is over. The editor and statusBar, on the other hand, could update status when the test "starts". Thus the discrepancy you saw...

I wasn't sure if the difference is noticeable, well, now I know 😅

Let me see if we can close the gap...

[stephtr]

ok, 1.59 has been released today, the extension should work in the official vscode now. @orta @stephtr do you guys have some cycle to take a look and play with it? Please let me know if you have any comments, thanks.

Unfortunately, I don't have any resources available, but I installed the RC and gave it a quick try. Looks nice so far!

I also noticed something similar to distante: When the tests are finished, the green circled checkmarks appear. If one then removes a line within the file (without saving) and clicks onto the green checkmark, differently looking checkmarks appear: One at the test's old location and another one at the test's new location. If one saves the changes, everything is fine again.

[connectdotz]

hmmm... looks like we ran out of the free credit in travis-ci.com... ok I guess it's time to move on to another CI system... @orta do you have any opinion? Looks like jest itself uses circle-ci, but that was before github actions... thoughts?

[updated]
I take it back, looks like jest is using github actions...

[connectdotz]

meanwhile, I made some changes to address the comments:

1. refactored the mechanism of how the existing system communicates with the new test-provider (supporting vscode TestExplorer). Now using an event mechanism for both run and session processes, so components like test-provider can register and process the same events JestExt received, without modifying JestExt. This would also enable us to further decouple components like statusBar, debug-provider, coverage in the future...
2. change the jest run command to always report colors so we can show nice native jest output in the test explorer's terminal. (we could remove the existing output channel... anybody has a strong opinion about this?)
3. replace our own inline-error messaging functionality with explorer's
4. added a new setting jest.testExplorer to fine-tune some experience:
   • enable/disable testExplorer support
   • hiding/showing the original status decoration when test explorer is enabled.
   • showing/hiding test explorer's inline error messaging
5. added an information-message popup on the start-up to remind people about test-explorer support since by default the test explorer will be enabled and they will not see the familiar "?" status decorations
6. adding the new vscode Testing category in package.json

vscode-jest-4.1.0-rc.2.vsix.zip

[stephtr]

During my brief test it worked quite nice!
I only noticed one issue (I don't think it happened with the 4.0 release):

• Have a (passing) test file, committed to Git.
• Add a test failure (like expect(2).toBe(1);)
• Save the file => test runs, test shows up as failure
• Undo the change & save => test doesn't rerun, appears as being broken, despite being correct

[connectdotz]

@stephtr thanks for testing, are you on watch mode?

[stephtr]

@stephtr thanks for testing, are you on watch mode?

Yes, exactly.

[connectdotz]

I suspect it could be a jest watch issue, I think I have seen similar behavior before, the reverted file has timestamp equal or less than the committed file thus not triggering the test run... if you turn on the test explorer's output, you would probably see message like 'No tests found related to files changed since last commit.'

If you get a chance, can you test executing explicit test from test explorer UI in a non-watch mode project in windows. These individual test run is new (in the past the we only supported file level test run), wanted to make sure it works for window users...

[stephtr]

You're completely right, it's really due to no changes since last commit.

I just tested manual test execution in the test explorer, for me it is working fine!
As usual, great work!

[connectdotz]

update for rc3:

address part of #749, particularly about status not update until the run was completed. Even though the status did get updated (on the file level) during the run, @agos had a point that it is easy to miss it, especially when there are lots of tests within. rc3 will do a deep status update for "scheduled" and "started" tests so it should be a lot obvious now...

FYI, also migrated our CI to github actions.

vscode-jest-4.1.0-rc.3.vsix.zip