chore: use ooni/probe-cli v3.16.6

[bassosimone]

No description provided.

[bassosimone]

I don't fully understand why the CI is failing here 🤷

Here's the full log:

jest -c test/lib/integration.config.js --testRegex 'e2e.js' --runInBand
PASS test/onboarding.e2e.js (29.971 s)
  Onboarding Story 1
    ✓ App is running (132 ms)
    ✓ App launches with correct title (92 ms)
    ✓ First screen renders correctly (141 ms)
    ✓ App goes to the second screen (292 ms)
    ✓ App loads up Pop Quiz (275 ms)
    ✓ App accepts first Pop Quiz Answer (1644 ms)
    ✓ App accepts second Pop Quiz Answer and goes to Crash Reporting page (1673 ms)
    ✓ App lets user opt-in for Crash Reporting (214 ms)
    ✓ Finishing Onboarding process brings up the Dashboard (307 ms)
    ✓ Check if Crash Reporting is enabled in Settings (295 ms)
  Onboarding Story 2
    ✓ App plays cross animation and shows warning if first pop quiz answer is wrong (4853 ms)
    ✓ App plays cross animation and shows warning if second pop quiz answer is wrong (1811 ms)
    ✓ App lets user opt out of Crash Reporting (250 ms)
    ✓ Finishing Onboarding process brings up the Dashboard (292 ms)
    ✓ Check if Crash Reporting is disabled in Settings (304 ms)
PASS test/sidebar.e2e.js

Sidebar Works

✓ has window title set to "OONI Probe" (3495 ms)
PASS test/dashboard.e2e.js (5.361 s)

Dashboard tests

✓ Run button is displayed on Dashboard (101 ms)

✓ Clicking on "Test Results" tab loads the Test Results Page (284 ms)

✓ Clicking on "Settings" tab loads the Settings page (309 ms)

All 5 test cards are visible

✓ websites test card is visible (144 ms)

✓ im test card is visible (138 ms)

✓ circumvention test card is visible (129 ms)

✓ performance test card is visible (135 ms)

✓ middlebox test card is visible (130 ms)
PASS test/localization.e2e.js (7.518 s)

Localized version is displayed for supported languages

✓ Check if the app shows content in French (104 ms)

Unsupported language (Malayalam) falls back to English

✓ Run button label in English (97 ms)
PASS test/settings.e2e.js (7.139 s)

Tests for Settings page

✓ Clicking on "Settings" tab loads it up correctly (312 ms)

✓ Language changes to Spanish (374 ms)

✓ Language changes in other screens (497 ms)

✓ Changing language back to En (373 ms)

✓ Clicking on button to edit Website categories brings up modal (234 ms)

✓ Modal Deselect All buttons work as expected (235 ms)

✓ Modal Select All buttons work as expected (329 ms)

✓ Individual categories can be enabled or disabled (788 ms)
PASS test/runtest-individual.e2e.js (242.898 s)

IM test

✓ IM test successfully starts (500 ms)

✓ IM test runs with all 4 network tests (5762 ms)

✓ Loads Test Results page on completion (815 ms)

Websites test

✓ Website measurement test loads correctly (392 ms)

✓ Website network runs successfully (4055 ms)

✓ Loads Test Results page on completion (89967 ms)

Custom websites test

✓ Run button is disabled by default (282 ms)

✓ Allows entering custom URLs (411 ms)

✓ Runs the custom websites test (220 ms)

✓ Tests custom URLs (3054 ms)

✓ Loads Test Results page on completion (844 ms)

Circumvention test

✓ Circumvention test successfully starts (434 ms)

✓ Circumvention test runs with all 3 network tests (10191 ms)

✓ Loads Test Results page on completion (31968 ms)

Performance test

✓ Performance test successfully starts (452 ms)

✓ Performance test performs both network tests (3181 ms)

✓ Loads Test Results page on completion (20892 ms)

Middleboxes test

✓ Middleboxes test successfully starts (430 ms)

✓ Middleboxes test performs both network tests (2143 ms)

✓ Loads Test Results page on completion (5509 ms)

Experimental test

✓ Experimental test successfully starts (488 ms)

✓ Experimental test performs some tests (164 ms)

✓ Loads Test Results page on completion (36173 ms)
FAIL test/runtest-all.e2e.js (237.019 s)

All network tests run successfully

✓ Tests start successfully on click of RUN button (369 ms)

✓ Website test runs from start to finish (95847 ms)

✓ Circumvention test runs from start to finish (39883 ms)

✓ IM test runs from start to finish (5955 ms)

✕ Middleboxes test runs from start to finish (30687 ms)

✕ Performance test runs from start to finish (30090 ms)

✕ Experimental test runs from start to finish (30100 ms)
● All network tests run successfully › Middleboxes test runs from start to finish
waitUntilTextExists waitUntil condition timed out after 30000ms

  at node_modules/webdriverio/build/commands/browser/waitUntil.js:24:19
  at Browser.wrapCommandFn (node_modules/@wdio/utils/build/shim.js:63:29)
  at Browser.waitUntilTextExists (node_modules/@wdio/utils/build/shim.js:63:29)

● All network tests run successfully › Performance test runs from start to finish
waitUntilTextExists waitUntil condition timed out after 30000ms

  at node_modules/webdriverio/build/commands/browser/waitUntil.js:24:19
  at Browser.wrapCommandFn (node_modules/@wdio/utils/build/shim.js:63:29)
  at Browser.waitUntilTextExists (node_modules/@wdio/utils/build/shim.js:63:29)

● All network tests run successfully › Experimental test runs from start to finish
waitUntilTextExists waitUntil condition timed out after 30000ms

  at node_modules/webdriverio/build/commands/browser/waitUntil.js:24:19
  at Browser.wrapCommandFn (node_modules/@wdio/utils/build/shim.js:63:29)
  at Browser.waitUntilTextExists (node_modules/@wdio/utils/build/shim.js:63:29)

PASS test/test-results.e2e.js (19.355 s)

Tests for Test-Results screen

✓ Page shows two IM measurement result rows (90 ms)

✓ Page shows three Websites measurement result rows (96 ms)

✓ Page shows two Circumvention test result rows (85 ms)

✓ Page shows two Middleboxes test result rows (84 ms)

✓ Page shows two Performance test result rows (146 ms)

IM test measurements

✓ There are total 4 IM measurements (208 ms)

✓ Result rows displayed for all 4 tests (131 ms)

✓ Detailed Measurements load up correctly (241 ms)

✓ Explorer button is displayed with correct link (1471 ms)

✓ Methodology link is correctly displayed (89 ms)

✓ Data button loads up raw data (278 ms)

Website test measurements

✓ There is at least one URL row (307 ms)

✓ Detailed Measurements load up correctly (268 ms)

✓ Explorer button is displayed with correct link (1258 ms)

✓ Methodology link is correctly displayed (91 ms)

✓ Data button loads up raw data (209 ms)

Circumvention test measurements

✓ There are total 2 Circumvention measurements (130 ms)

✓ Result rows displayed for all 3 tests (97 ms)

✓ Detailed Measurements load up correctly (233 ms)

✓ Explorer button is displayed with correct link (1331 ms)

✓ Methodology link is correctly displayed (134 ms)

✓ Data button loads up raw data (329 ms)

Performance test measurements

✓ There are total 2 Performance measurements (191 ms)

✓ Result rows displayed for all 2 tests (100 ms)

✓ Detailed Measurements load up correctly (385 ms)

✓ Explorer button is displayed with correct link (1125 ms)

✓ Methodology link is correctly displayed (111 ms)

✓ Data button loads up raw data (248 ms)

Middleboxes test measurements

✓ There are total 2 Middlebox measurements (188 ms)

✓ Result rows displayed for all 2 tests (100 ms)

✓ Detailed Measurements load up correctly (286 ms)

✓ Explorer button is displayed with correct link (855 ms)

✓ Methodology link is correctly displayed (85 ms)

✓ Data button loads up raw data (244 ms)

Experimental test measurements

✓ There is at least one Experimental measurements (251 ms)

✓ Detailed Measurements load up correctly (2225 ms)
Test Suites: 1 failed, 7 passed, 8 total

Tests:       3 failed, 97 passed, 100 total

Snapshots:   0 total

Time:        553.361 s

Ran all test suites.

error Command failed with exit code 1.

info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

/bin/bash /home/runner/work/_actions/GabrielBB/xvfb-action/v1.0/cleanup.sh

No xvfb processes to kill

Error: The process '/usr/bin/xvfb-run' failed with exit code 1

Ah, maybe the issue is this:

FAIL test/runtest-all.e2e.js (237.019 s)

All network tests run successfully

✓ Tests start successfully on click of RUN button (369 ms)

✓ Website test runs from start to finish (95847 ms)

✓ Circumvention test runs from start to finish (39883 ms)

✓ IM test runs from start to finish (5955 ms)

✕ Middleboxes test runs from start to finish (30687 ms)

✕ Performance test runs from start to finish (30090 ms)

✕ Experimental test runs from start to finish (30100 ms)

● All network tests run successfully › Middleboxes test runs from start to finish

waitUntilTextExists waitUntil condition timed out after 30000ms

  at node_modules/webdriverio/build/commands/browser/waitUntil.js:24:19
  at Browser.wrapCommandFn (node_modules/@wdio/utils/build/shim.js:63:29)
  at Browser.waitUntilTextExists (node_modules/@wdio/utils/build/shim.js:63:29)

● All network tests run successfully › Performance test runs from start to finish

waitUntilTextExists waitUntil condition timed out after 30000ms

  at node_modules/webdriverio/build/commands/browser/waitUntil.js:24:19
  at Browser.wrapCommandFn (node_modules/@wdio/utils/build/shim.js:63:29)
  at Browser.waitUntilTextExists (node_modules/@wdio/utils/build/shim.js:63:29)

● All network tests run successfully › Experimental test runs from start to finish

waitUntilTextExists waitUntil condition timed out after 30000ms

  at node_modules/webdriverio/build/commands/browser/waitUntil.js:24:19
  at Browser.wrapCommandFn (node_modules/@wdio/utils/build/shim.js:63:29)
  at Browser.waitUntilTextExists (node_modules/@wdio/utils/build/shim.js:63:29)

Apparently some end-to-end tests fail. (Maybe I can run them locally? In the meanwhile I'll restart the build.)

[bassosimone]

Okay, I am going to merge because the CI completed successfully. So, it seems it was just flaky before and some tests did not complete in the total amount of time they were given. FTR, this is the result of the run:

PASS test/runtest-all.e2e.js (207.818 s)
  All network tests run successfully
    ✓ Tests start successfully on click of RUN button (257 ms)
    ✓ Website test runs from start to finish (98805 ms)
    ✓ Circumvention test runs from start to finish (40356 ms)
    ✓ IM test runs from start to finish (4938 ms)
    ✓ Middleboxes test runs from start to finish (6353 ms)
    ✓ Performance test runs from start to finish (23857 ms)
    ✓ Experimental test runs from start to finish (29209 ms)

So, clearly 30 seconds of timeout is a bit border line for Performance and Experimental. I am surprised about the total runtime of Middleboxes in the previous run, though. 🤔