Skip to content

Conversation

@matejchalk
Copy link
Collaborator

@matejchalk matejchalk commented Sep 25, 2025

Fixes #1122

The permissions error seems to be a cleanup bug in lighthouse/chrome-launcher (see GoogleChrome/chrome-launcher#355). Overriding the default system temporary directory path on Windows to a local path fixes the tests.

From Node.js os.tmpdir() docs:

On Windows, the result can be overridden by TEMP and TMP environment variables, and TEMP takes precedence over TMP. If neither is set, it defaults to %SystemRoot%\temp or %windir%\temp.

On non-Windows platforms, TMPDIR, TMP and TEMP environment variables will be checked to override the result of this method, in the described order. If none of them is set, it defaults to /tmp.

Some operating system distributions would either configure TMPDIR (non-Windows) or TEMP and TMP (Windows) by default without additional configurations by the system administrators. The result of os.tmpdir() typically reflects the system preference unless it's explicitly overridden by the users.

I suspect GitHub's recent windows-latest image migration might be the reason the jobs started failing for us recently.

@nx-cloud
Copy link

nx-cloud bot commented Sep 25, 2025

View your CI Pipeline Execution ↗ for commit ebabf97

Command Status Duration Result
nx code-pushup --nx-bail -- compare ✅ Succeeded 55s View ↗
nx code-pushup --nx-bail -- ✅ Succeeded 1m 4s View ↗
nx code-pushup --nx-bail -- print-config --outp... ✅ Succeeded 3m 48s View ↗

☁️ Nx Cloud last updated this comment at 2025-09-26 12:45:00 UTC

@github-actions
Copy link
Contributor

github-actions bot commented Sep 25, 2025

Code PushUp

🤨 Code PushUp report has both improvements and regressions – compared current commit ee07af5 with previous commit b7ce5c6.

🕵️ See full comparison in Code PushUp portal 🔍

🏷️ Categories

🏷️ Category ⭐ Previous score ⭐ Current score 🔄 Score change
Performance 🔴 38 🔴 38 ↑ +0.2
Documentation 🔴 24 🔴 24 ↑ +0.1
Code coverage 🟡 90 🟡 90 ↑ +0.1
Security 🟡 56 🟡 56
Updates 🟡 84 🟡 84
Accessibility 🟢 92 🟢 92
Best Practices 🟢 100 🟢 100
SEO 🟡 61 🟡 61
Type Safety 🟢 100 🟢 100
Bug prevention 🟢 100 🟢 100
Miscellaneous 🟢 100 🟢 100
Code style 🟢 100 🟢 100
👍 3 groups improved, 👍 4 audits improved, 👎 2 audits regressed, 15 audits changed without impacting score

🗃️ Groups

🔌 Plugin 🗃️ Group ⭐ Previous score ⭐ Current score 🔄 Score change
Lighthouse Performance 🔴 38 🔴 38 ↑ +0.2
JSDoc coverage Documentation coverage 🔴 24 🔴 24 ↑ +0.1
Code coverage Code coverage metrics 🟡 90 🟡 90 ↑ +0.1

18 other groups are unchanged.

🛡️ Audits

🔌 Plugin 🛡️ Audit 📏 Previous value 📏 Current value 🔄 Value change
Lighthouse First Contentful Paint 🟨 2.9 s 🟥 3.1 s ↑ +6.2 %
Lighthouse Total Blocking Time 🟥 1,590 ms 🟥 1,450 ms ↓ −8.7 %
Lighthouse Speed Index 🟥 6.2 s 🟥 6.2 s ↑ +0.3 %
JSDoc coverage Functions coverage 🟥 521 undocumented functions 🟥 521 undocumented functions  +0 %
Code coverage Line coverage 🟨 86.3 % 🟨 86.3 % ↑ +0.1 %
Code coverage Branch coverage 🟨 85.5 % 🟨 85.5 % ↑ +0.1 %
Lighthouse Avoids enormous network payloads 🟩 Total size was 2,032 KiB 🟩 Total size was 2,020 KiB ↓ −0.6 %
Lighthouse Server Backend Latencies 🟩 1,500 ms 🟩 300 ms ↓ −80.1 %
Lighthouse Largest Contentful Paint 🟥 11.2 s 🟥 10.6 s ↓ −5.4 %
Lighthouse Uses efficient cache policy on static assets 🟨 30 resources found 🟨 30 resources found ↑ +0.1 %
Lighthouse Reduce unused CSS 🟥 Potential savings of 102 KiB 🟥 Potential savings of 102 KiB ↓ −25.4 %
Lighthouse Reduce unused JavaScript 🟥 Potential savings of 156 KiB 🟥 Potential savings of 179 KiB ↑ +20.5 %
Lighthouse Remove duplicate modules in JavaScript bundles 🟥 Potential savings of 101 KiB 🟥 Potential savings of 101 KiB ↓ −34 %
Lighthouse Time to Interactive 🟥 12.9 s 🟥 12.8 s ↓ −1.1 %
Lighthouse Metrics 🟩 100% 🟩 100% ↓ −1.1 %
Lighthouse Minimizes main-thread work 🟥 11.2 s 🟥 11.1 s ↓ −0.8 %
Lighthouse JavaScript execution time 🟥 4.6 s 🟥 4.6 s ↓ −1.4 %
Lighthouse Max Potential First Input Delay 🟥 1,120 ms 🟥 1,150 ms ↑ +2.7 %
Lighthouse Initial server response time was short 🟩 Root document took 440 ms 🟩 Root document took 410 ms ↓ −6.4 %
Lighthouse Network Round Trip Times 🟩 10 ms 🟩 30 ms ↑ +143.1 %
Code coverage Function coverage 🟩 92.5 % 🟩 92.5 % ↑ +0.1 %

589 other audits are unchanged.

@pkg-pr-new
Copy link

pkg-pr-new bot commented Sep 25, 2025

Open in StackBlitz

@code-pushup/ci

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/ci@1124

@code-pushup/cli

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/cli@1124

@code-pushup/core

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/core@1124

@code-pushup/create-cli

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/create-cli@1124

@code-pushup/models

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/models@1124

@code-pushup/coverage-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/coverage-plugin@1124

@code-pushup/nx-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/nx-plugin@1124

@code-pushup/eslint-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/eslint-plugin@1124

@code-pushup/js-packages-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/js-packages-plugin@1124

@code-pushup/jsdocs-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/jsdocs-plugin@1124

@code-pushup/lighthouse-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/lighthouse-plugin@1124

@code-pushup/typescript-plugin

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/typescript-plugin@1124

@code-pushup/utils

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/utils@1124

@code-pushup/models-transformers

npm i https://pkg.pr.new/code-pushup/cli/@code-pushup/models-transformers@1124

commit: ebabf97

@matejchalk matejchalk force-pushed the fix-ligthouse-cleanup-e2e-tests branch from b113192 to 5216eb5 Compare September 26, 2025 12:02
@matejchalk matejchalk force-pushed the fix-ligthouse-cleanup-e2e-tests branch from 5216eb5 to ebabf97 Compare September 26, 2025 12:37
@github-actions github-actions bot added the 🔬 testing writing tests label Sep 26, 2025
@matejchalk matejchalk changed the title build(plugin-lighthouse): update lighthouse dependencies fix(plugin-lighthouse): prevent cleanup permissions error on windows Sep 26, 2025
@matejchalk matejchalk marked this pull request as ready for review September 26, 2025 13:08
@matejchalk matejchalk added the 🐛 bug something isn't working label Sep 26, 2025
@matejchalk matejchalk merged commit ac43d41 into main Sep 26, 2025
27 checks passed
@matejchalk matejchalk deleted the fix-ligthouse-cleanup-e2e-tests branch September 26, 2025 13:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🐛 bug something isn't working 🧩 lighthouse-plugin 🔬 testing writing tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Lighthouse E2E tests fail on Windows (permissions)

3 participants