Skip to content

Introduce vscode-extension-tester to support UI testing#51

Merged
rcjsuen merged 5 commits intomainfrom
add-smoke-testing
Apr 4, 2025
Merged

Introduce vscode-extension-tester to support UI testing#51
rcjsuen merged 5 commits intomainfrom
add-smoke-testing

Conversation

@rcjsuen
Copy link
Collaborator

@rcjsuen rcjsuen commented Apr 2, 2025

Problem Description

We do not have any tests in this repository. Due to our dependency of a Go binary that is sourced from another repository, it is important that we verify that the builds we generate are able to start the language server and provide the expected features to the client.

Proposed Solution

We will use the vscode-extension-tester library to drive some UI tests. This will help us perform some simplified validation on our builds.

Proof of Work

Installing extensions...
Extension 'docker-0.3.0.vsix' was successfully installed.


Detected user defined code settings
Writing code settings to /var/folders/ln/c6gntgwx7m5gwlhx2dkc73w40000gp/T/test-resources/settings/User/settings.json
Launching browser...
Browser ready in 6157 ms
Launching tests...
  Docker Bake
    Problems
      ✔ Bake file has an error about not being able to find an ARG
      ✔ Bake file has no warnings

  Dockerfile
    Problems
      ✔ Dockerfile has no errors
      ✔ Dockerfile has a JSONArgsRecommended warning

Shutting down the browser

  4 passing (19s)

rcjsuen added 2 commits April 2, 2025 13:57
Due to our dependency of a Go binary that is sourced from another
repository, it is important that we verify that the builds we generate
are able to start the language server and provide the expected features
to the client.

Signed-off-by: Remy Suen <remy.suen@docker.com>
Signed-off-by: Remy Suen <remy.suen@docker.com>
@rcjsuen rcjsuen self-assigned this Apr 2, 2025
Signed-off-by: Remy Suen <remy.suen@docker.com>
@rcjsuen rcjsuen requested a review from stanislavHamara April 2, 2025 18:12
@rcjsuen
Copy link
Collaborator Author

rcjsuen commented Apr 2, 2025

@stanislavHamara Could you checkout this branch and try running npm install and npm test? Want to make sure I didn't miss anything and that another developer is able to execute the tests locally.

@stanislavHamara
Copy link
Contributor

@stanislavHamara Could you checkout this branch and try running npm install and npm test? Want to make sure I didn't miss anything and that another developer is able to execute the tests locally.

I get this pop up when I start tests
image

Tests output:

0 passing (46s)
  2 failing

  1) Docker Bake
       Problems
         "before all" hook for "Bake file has an error about not being able to find an ARG":
     TimeoutError: Wait timed out after 15129ms
      at /Users/stanislavhamara/source/vscode-extension/node_modules/selenium-webdriver/lib/webdriver.js:929:22
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

  2) Dockerfile
       Problems
         "before all" hook for "Dockerfile has no errors":
     TimeoutError: Wait timed out after 15079ms
      at /Users/stanislavhamara/source/vscode-extension/node_modules/selenium-webdriver/lib/webdriver.js:929:22
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

@rcjsuen
Copy link
Collaborator Author

rcjsuen commented Apr 3, 2025

I get this pop up when I start tests image

@stanislavHamara Aha, I see. :)

  1. When you run npm test, right before the tests start running it generate a VSIX file. Could you provide the output after INFO Files included in the VSIX: so I can see what is in the VSIX file that it has built?
  2. Do you have a bin folder and if yes, what is inside the bin folder?

@stanislavHamara
Copy link
Contributor

I get this pop up when I start tests image

@stanislavHamara Aha, I see. :)

  1. When you run npm test, right before the tests start running it generate a VSIX file. Could you provide the output after INFO Files included in the VSIX: so I can see what is in the VSIX file that it has built?
  2. Do you have a bin folder and if yes, what is inside the bin folder?
 INFO  Files included in the VSIX:
docker-0.3.0.vsix
├─ [Content_Types].xml 
├─ extension.vsixmanifest 
└─ extension/
   ├─ LICENSE.txt [10.51 KB]
   ├─ NOTICE.html [17.36 KB]
   ├─ SECURITY.md [1.82 KB]
   ├─ TELEMETRY.md [1.34 KB]
   ├─ changelog.md [1.97 KB]
   ├─ package.json [4.69 KB]
   ├─ readme.md [4.92 KB]
   ├─ dist/
   │  ├─ 1.extension.js [21.23 KB]
   │  ├─ 770.extension.js [20.65 KB]
   │  └─ extension.js [350.64 KB]
   ├─ resources/
   │  └─ docker-logo-vertical-blue.png [244.23 KB]
   └─ syntaxes/
      └─ hcl.tmGrammar.json [18.77 KB]

No bin folder

Signed-off-by: Remy Suen <remy.suen@docker.com>
@rcjsuen
Copy link
Collaborator Author

rcjsuen commented Apr 3, 2025

@stanislavHamara Thanks for helping test this!

Could you fetch the latest add-smoke-testing branch again and then run npm install followed by npm test?

We do not need to build the language server binary anymore.

Signed-off-by: Remy Suen <remy.suen@docker.com>
Copy link
Contributor

@stanislavHamara stanislavHamara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🔥

@rcjsuen rcjsuen merged commit e8a3cce into main Apr 4, 2025
9 checks passed
@rcjsuen rcjsuen deleted the add-smoke-testing branch April 4, 2025 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants