Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Test] Switch to VSCode Extension Runner #4880

Open
2 tasks done
JustinGrote opened this issue Jan 4, 2024 · 5 comments
Open
2 tasks done

[Test] Switch to VSCode Extension Runner #4880

JustinGrote opened this issue Jan 4, 2024 · 5 comments
Labels
Area-Test Issue-Enhancement A feature request (enhancement). Up for Grabs Will shepherd PRs.

Comments

@JustinGrote
Copy link
Collaborator

JustinGrote commented Jan 4, 2024

Prerequisites

  • I have written a descriptive issue title.
  • I have searched all issues to ensure it has not already been reported.

Summary

VSCode now has a specific CLI and test runner extension for vscode E2E tests driven by Mocha
https://code.visualstudio.com/api/working-with-extensions/testing-extension
https://marketplace.visualstudio.com/items?itemName=ms-vscode.extension-test-runner

This will move our highly customized testing process to a more standard operation.

@JustinGrote JustinGrote added Issue-Enhancement A feature request (enhancement). Needs: Triage Maintainer attention needed! Up for Grabs Will shepherd PRs. Area-Test and removed Needs: Triage Maintainer attention needed! labels Jan 4, 2024
@andyleejordan
Copy link
Member

That'd be awesome!

@JustinGrote
Copy link
Collaborator Author

@andyleejordan I'm evaluating it on the Pester Tests extension first and things are pretty promising, we should be able to port most of the process without much adjustment.

@JustinGrote
Copy link
Collaborator Author

JustinGrote commented Jan 10, 2024

After initial evaluations, this is feasible for the vscode-powershell extension but there are two areas our custom runner handles that vscode-test runner does not currently:

  1. Handling sourcemaps correctly when live-transpiling the tests
  2. Installing Prerequisite Extensions

I'll work with the vscode cli team a bit more, but their work more or less duplicates what we've already done with the custom runner so as long as all of our behavior can be incorporated there, the Mocha test runner extension they have devised is much nicer than our standard Mocha test runner hack, and would also allow us to more easily separate test suites into E2E (that require vscode runtime) and non-E2E that can be run more quickly.

@andyleejordan
Copy link
Member

Amazing intel, thanks Justin!

@JustinGrote
Copy link
Collaborator Author

I've been able to move over my Pester Tests extension to use the cli and extension runner.

pester/vscode-adapter#235

It's also able to run "pure" tests that don't require vscode code separately from the vscode ones.

Still has the limitations of not being able to pre-install extensions (workaroundable by pre-populating a profile) and does not work for transpling vscode e2e tests directly (sourcemaps dont work right) so the tests must be built, however I have a new esbuild config that can watch-build the tests without using TSC and work correctly with the test runner extension and still support source maps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Test Issue-Enhancement A feature request (enhancement). Up for Grabs Will shepherd PRs.
Projects
None yet
Development

No branches or pull requests

2 participants