[BUG] Playwright test for vscode 1.0.0 doesn't find tests with Vscode 1.73.1

[qasamppa]

Context:

• Playwright Version: [what Playwright version do you use?] 1.27.1
• Operating System: [e.g. Windows, Linux or Mac] MacOS 12.6
• Node.js version: [e.g. 12.22, 14.6] 19.1.0
• Visual Studio Code version: [e.g. 1.65] 1.73.1
• Playwright for VSCode extension version: [e.g. 1.2.3] 1.0.0
• Browser: [e.g. All, Chromium, Firefox, WebKit] All
• Extra: [any specific details about your environment]

Code Snippet

Help us help you! Put down a short code snippet that illustrates your bug and
that we can run and debug locally. For example:

Describe the bug

Playwright test for vscode 1.0.0 doesn't show tests in repository.

[arpan-patelia]

I am facing a similar issue.
I created a new Playwright project using VS Code extension but the Test Explorer is asking me to install additional test extension. If I hit the refresh tests button, I get a notification asking me to run npm install --save-dev @playwright/test.
I already have it installed the correct way through VS Code extension but still I get this.

It happens for all my Playwright projects now, old and new ones.

Switched to latest Node.js LTS from the current 19.1.0 and created the project again but no luck.
Node version: 18.12.1 and 19.1.0
NPM version: 8.19.2
VS Code version: 1.73.1
Playwright Test for VS Code version: 1.0.0

[dgozman]

We are unable to reproduce this issue, with playwright v1.27.1/v1.28 and node v18/v19. Is there anything else that's special in your setup? Other VSCode extensions? Using yarn or some other package manager? Any environment variables you set?

[qasamppa]

I have many VSCode extensions and I am using npm package manager. The thing is, that I have exactly same setup like earlier, only VSCode app and Playwrite test for vscode extension versions have been changed when problem occurred. My colleague has same setup with VSCode version 1.69 and Playwrite test for vscode 1.0.0 setup combination and it works. I haven't tried to downgrade my VSCode yet.

[dgozman]

@qasamppa I am not sure we'll be able to help you without any more information from your side. I'd suggest trying to disable other extensions, or playing with VSCode version.

[qasamppa]

@dgozman I installed vscode version 1.70.2 and Playwright test for vscode 1.0.0 -extension started to work immediately. So definitely something to do with latest release(s) of vscode.

[pavelfeldman]

Could you tell us what other extensions you have installed? We can't reproduce this problem on clean VSCode + Playwright extension installed.

[qasamppa]

Bash Beautify v0.1.1, Cypress Helper 1.2.2, Cypress Snippets v0.0.5, Docker v1.22.2, ESLint v2.2.6, Git Blame 9.0.1, Git Graph v1.30.0, Git History v0.6.19, Jupyter Keymap v1.0.0 and Markdown Preview Mermaid Support v1.15.2, markdown lint v0.48.1, Playwright Test for VSCode 1.0.1, Playwright Test Runner v1.4.0, Playwright Test Snippets v1.2.1, Prettier - Code Formatter v9.9.0, Pylance v2022.11.30, Robot Framework Language Server v1.5.0, Shell Check v0.28.2, Stylelint v1.2.3, YAML v1.10.1, sort v2022.8.0, Jupyter v2022.9.1303220346, Jupyter Cell Tags v0.1.6, Jupyter Notebook Renderers v1.0.12, Jupyter Slide Show v0.1.5, Python v2022.18.2

Long list, yes :)

[arpan-patelia]

I tried disabling/uninstalling all extensions, removing .vscode directory and starting the IDE afresh with only Playwright extension, but none of it worked. Finally, when I installed system wide node and npm, then it got fixed.

So, in my case, it somehow stopped working with node/npm installed via nvm-node version manager. None of extension had any conflict.

[pavelfeldman]

@arpan-patelia thanks for these details! Could you help me via answering those questions?

• before installing node globally, were you able to run node from within VSCode's terminal?
• could it be that you installed VSCode from flatpack?
• do you have anything like "terminal.integrated.shell" set in your VSCode preferences?

It seems like Playwright VSCode Extension could not find node in your PATH and hence it could not run Playwright. So it fails very early and pretty bad. Knowing why it stopped locating node in your PATH would help.

[arpan-patelia]

@pavelfeldman just to be more specific, installing system wide npm fixed the issue. However, I still have issue with testInfo from Playwright Runner not being available in my tests - which is a different issue and I will report a separate bug for it.

Below are the answers for your questions.

1. before installing node globally, were you able to run node from within VSCode's terminal?

• Yes, I was able to run node/npx playwright test in VSCode's termial all the time, even when the Playwright extension was not working.

2. could it be that you installed VSCode from flatpack?

• No, VSCode is neither from flatpak and nor a Snap package.

3. do you have anything like "terminal.integrated.shell" set in your VSCode preferences?

• Yes, I have it enabled. Screenshot below for further related info.

[arpan-patelia]

#19181 can be linked here too.

As mentioned there, I too used nvm default alias system to use the system installed node version instead of the one installed by nvm.

Hope this helps anyone facing this issue.

[ejkreboot]

<tl;dr>
I have type=module in my config.json, so I need the ES convention using 'import'. However, if I select 'javascript' as my language rather than typescript when initializing playwright, the CommonJS idiom is used, with 'require'. As a result, playwright encounters an error when starting, and my test list is not populated in the explorer and npm playwright test rails as well. Re-initializing with Typescript as the language resolved this issue--although I do not otherwise use Typescript in my project.

Forgive my naiveté, but does Typescript presume ES modules, and does javascript presume CommonJS? I don't use Typescript, but I do use ES modules so I tripped over this option in playwright initialization.
</tl;dr>

Same issue, fixed in as follows. However the fix (removing "type: module" from package.json) breaks my app. See below for the actual fix.

These steps were not necesssary, but just wanted to confirm the problem and the fix:

1. Removed node and npm from my system.
2. Removed VS Code.
3. Downloaded and installed latest VS code.
4. Checked for updated to VS code (no updates)
5. Installed LTS version of node and npm (18.12.1) via MacOS installation package from nodejs.org
6. Cloned my project from github.
7. Used VS Code command palette to install Playwright, selected "Use javascript" option, and installing only Chromium browser
8. Restarted VS Code...test explorer states "No tests have been found"
9. Executed "npx playwright test". Got errors:

ReferenceError: require is not defined in ES module scope, you can use import instead
This file is being treated as an ES module because it has a '.js' file extension and '/Users/erikor/Documents/Business/XXXX/XXXX/package.json' contains "type": "module". To treat it as a CommonJS script, rename it to use the '.cjs' file extension.

Here was the fix:

10. Removed this line from package.json in my app:

  "type": "module",

11. Restarted VS code. Tests now appear on test explorer. But I can no longer start my app which relies this entry in package.json to work.

So, finally, I re-initialized playwright specifying Typescript as the language, and everything worked as advertised.

[smfelsher]

I was encountering the same issue.

<tldr>
The Playwright Test for VSCode extension is invoking a shell that does not have node in its path. I believe the extension would work if it invoked a shell in the same manner as the VSCode integrated terminal.
</tldr>

I installed the Playwright Test for VSCode extension. "@playwright/test": "^1.28.1" was added to my package.json file. Running npx playwright test on the command line successfully executed the tests in the example.spec.ts file installed by the extension. However, VSCode was not displaying any tests in the Test Explorer sidebar.

I have node installed for my user account using nvm. I did not have node installed on my system (Linux Manjaro).

The extension did discover the example test cases after I installed nodejs on my system using the software updater tool. It seems as though the extension is not invoking my shell (zsh), which would have put node in the path via the nvm utility.

It has been some time since I first installed VSCode, but I recall an issue with how the integrated terminal invoked the shell. nvm adds its magic to the .zshrc file. When I opened the integrated terminal in VSCode, node was not in the path. After a Google search, I added the nvm magic to a .zshenv file, and then node was in the path. I believe it's related to the shell being interactive, etc.

Here is the aforementioned nvm magic that I moved from .zshrc to .zshenv to have node in the path of the VSCode integrated terminal.

export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # This loads nvm bash_completion

[pavelfeldman]

Thanks for the thorough investigation @smfelsher. Extension does not really use any of the shells, but it does look for your Node installation using the which package, which is looking at the current process process.env.PATH. So if you VS Code ran from the environment that could discover Node (i.e. node was in PATH), things should work. Could you check if that is the case?

[smfelsher]

So if you VS Code ran from the environment that could discover Node (i.e. node was in PATH), things should work. Could you check if that is the case?

That's exactly what I tried to describe in my previous post.

• I had node installed for my user account using nvm.
• node was not installed on the system through the package manager.
• The extension does not display any tests in VS Code
• I installed node on the system using my package manager.
• The extension finds the tests.

Why?

I originally installed node using nvm. This only installs node for my user.

• The nvm installation adds node to the path by adding the magic to your shell profile. I use Z shell, so this would be ~/.zshrc in my case.

At this point, VSCode will not see node in the system path. This is the problem. Even if you open the integrated terminal in VS Code, node would not be in the path! When I moved the nvm magic into my ~/.zshenv file, then the VSCode integrated terminal would have node in the path. This has something to do with the difference between interactive and non-interactive shells and how these environments get loaded by VSCode, integrated terminals, and extensions.

So, people that use nvm to install node and also use VSCode could have an issue with this extension because of the way the extension tries to find node.

[pavelfeldman]

VS Code is detecting your dev environment via running shell command and is patching its own environment at runtime. It does it lazily though, after IDE was opened. We used to have retries for locating node that we've lost, i brought them back to see if that helps.

[chaitanya-cloudknox]

I had similar issue, the plugin is unable to resolve node (I am using nvm to manage multiple node) but the terminal (zsh) does.

I tried to reset/change terminal settings but I think this config helped fix the issue for me

"terminal.integrated.shellIntegration.enabled": true

[uncvrd]

I'd just like to report in that I ran in to this issue because I was trying to load in an environment variable in to baseURL in my config. I guess, because the value was not defined this was causing problems when searching for tests in my environment!

Checking VS Code Output I saw:

2023-02-26 20:43:31.105 [error] [Extension Host] SyntaxError: Unexpected end of JSON input
	at JSON.parse (<anonymous>)
	at Oe.listFiles (/Users/jordan/.vscode/extensions/ms-playwright.playwright-1.0.7/out/extension.js:14:4582)
	at process.processTicksAndRejections (node:internal/process/task_queues:96:5)
	at async je.listFiles (/Users/jordan/.vscode/extensions/ms-playwright.playwright-1.0.7/out/extension.js:84:712)
	at async Ue._rebuildModel (/Users/jordan/.vscode/extensions/ms-playwright.playwright-1.0.7/out/extension.js:84:14105)
	at async s.$refreshTests (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:98:141459)

Which led me down the rabbit hole discovering that the undefined value made my config invalid syntax-wise causing this parsing error.

Anyways...make sure your values are valid in the config! Hope this helps someone going forward :)

[JayDouglass]

Anyways...make sure your values are valid in the config! Hope this helps someone going forward :)

@uncvrd Thanks, this was exactly the problem.

I had both webServer.url and webServer.port in playwright.config.ts, which caused the tests to not show up in the test explorer. I removed the webServer.port entry and it displayed the tests again. @pavelfeldman It would be helpful if there was more debugging output.

[sarink]

If you're using nvm, and you launch VSCode from the Applications folder, it doesn't know the right version of node for your project. The simple fix is to navigate to your project directory in the terminal, nvm use, and then code ., now VSCode will be launched with the same version of node, and your playwright extension should be happy. I have spent many hours trying to debug this, it's a shame it isn't explained somewhere obvious

[erocheleau]

I still have that issue using WSL 2 on windows with zsh.

I have node installed through nvm.

I have tried moving the "nvm" magic into the .zshenv, it didn't do anything.
I have tried the nvm use and launching vscode from the command line code . (that's always what I use anyway).
All my terminals in VSCODE are able to use node.

npx playwright test when execute in command line.

Anyone has another idea?

VSCode version 1.81.1 - x64
Node version v18.16.0

[timdonovanuk]

Installing node/npm systmem wide using NodeSource, rather than nvm, fixed this for me.