Skip to content

ci(e2e): enable the shared Playwright job for app-template - #115

Merged
rubenvdlinde merged 5 commits into
developmentfrom
ci/enable-e2e-app-template
Aug 4, 2026
Merged

ci(e2e): enable the shared Playwright job for app-template#115
rubenvdlinde merged 5 commits into
developmentfrom
ci/enable-e2e-app-template

Conversation

@rubenvdlinde

@rubenvdlinde rubenvdlinde commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Why

enable-playwright was unset, so the shared quality workflow reported the E2E job as skipped on every run. tests/e2e/app-shell.spec.ts sat in the repo without ever executing in CI — and a skipped check is indistinguishable from a passing one on the PR page.

Turning it on was one line. Making it green took two fixes, both of which turned out to be real defects the gate existed to catch.

Is app-template a genuine e2e candidate?

Yes. The suite is small — 7 app-shell tests — but every assertion targets a Vue 3 migration failure mode that produces a page that still renders, with an empty console: a failed createApp().mount() leaves the server-rendered container in place; root props changed shape in Vue 3; built-in widgets get tree-shaken unless registerBuiltinDashboardWidgets() runs; a missing icon renders nothing rather than a fallback; gridstack v12 lays items out 0 px wide without its stylesheet; Vue Router 4 dropped the bare path: '*' wildcard. None of those are reachable from a unit test or visible to a build.

And this repo is the scaffold every Conduction app is cloned from, so a silent break here propagates fleet-wide on the next app-create.

Nothing was weakened

No test skipped, no assertion weakened, no timeout raised, no error allow-list widened, and no spec excludedplaywright-test-path stays at its default. The full suite runs.

The failures were one cause, then a second — not five problems

The first run measured 5 passed / 5 failed. All five failures reported element(s) not found for main, #app-navigation-vue a svg, and .cn-widget-wrapper.

The Playwright report artifact (playwright-report/data/*.md carries a full accessibility-tree snapshot of the page at failure) settled it immediately. Every failing page was showing:

Required apps are missing — This app needs the following Nextcloud apps to be installed and enabled: openregister [Install and enable]

That is nc-vue's CnAppRoot dependency gate, honouring "dependencies": ["openregister"] in src/manifest.json. The app mounted perfectly (:32 passed throughout, manifest prop and all) — it was correctly refusing to render an app whose backing store was absent. The gate renders no <main>, no nav and no widgets, so every selector legitimately found nothing.

Fix 1 — provision the dependency (5 → 9 passed)

additional-apps is an existing input on the shared workflow, and openregister was already the literal example in the commented-out line in this file. Uncommented it. openregister 0.2.13-unstable.79 enabled, and the app started making real API calls against it.

Fix 2 — navigate via the app's own URL base (9 → 10 passed)

One failure survived, and it was the more interesting one. /apps/app-template/examples rendered "Recent examplesActionsNo examples yet" — the dashboard, not the Examples page.

src/main.js builds the router with createWebHistory(generateUrl('/apps/app-template')), so the history base is whatever this instance serves. With mod_rewrite it is /apps/app-template; without it, /index.php/apps/app-template. The shared workflow serves Nextcloud from php -S with a router script and no mod_rewrite, so the second form is canonical there. A spec that hard-codes the pretty form lands on a page whose router cannot strip its own base: vue-router matches nothing, falls through to the catch-all, and rewrites to the dashboard — under a plain HTTP 200 with no redirect.

That was producing two false passes, which is why this matters more than the one red test:

  • :68 no built-in widget resolves to the "Widget unavailable" placeholder iterated four routes and was asserting against the dashboard on all four.
  • :124 the router catch-all redirects unknown paths to the dashboard passed while proving nothing — when every path bounces to the dashboard, "the catch-all redirected me" and "routing is completely broken" are the same page.

Fixed at the cause in tests/e2e/_app-url.ts: ask the instance via OC.generateUrl — the same function @nextcloud/router wraps — instead of guessing. It throws rather than falling back to a literal, because a guessed base is precisely the failure mode being removed. No assertion was changed.

Result

Passed Failed
Before 5 5
After fix 1 9 1
After fix 2 10 0

Runtime 35.9s, well inside the 20-minute budget (timeout-minutes: 45).

Proof the green is load-bearing — truncation control

A green suite that would stay green against a broken app proves nothing. Measured by truncating every emitted JS bundle to 0 bytes after webpack succeeds (truncated, not deleted — the workflow rebuilds a deleted bundle, and a missing bundle returns HTTP 200 text/html, never a 404), on a scratch branch otherwise identical to this one:

Suite state Passed Failed
Normal 10 0
Bundle truncated to 0 bytes 3 7

The truncation is verified in the build log rather than assumed — the files are still present, just empty:

=== TRUNCATION CONTROL: emptying built JS ===
-rw-r--r-- 1 runner runner  1934808 app-template-main.js      <- before
-rw-r--r-- 1 runner runner        0 app-template-main.js      <- after

All 7 app-shell tests collapse. The survivors are auth.setup.ts (legitimately bundle-independent — Nextcloud's login page is server-rendered) and the 2 docs-screenshots capture specs, which assert nothing that requires the app to mount. That insensitivity is filed as #118 — it is not an exclusion, those specs are still in the gate and still green.

Filed, not hidden

Nothing is excluded from the gate, so there is no exclusion backlog. Two defects the gate surfaced are filed instead:

Neither is fixed here — this PR is the gate, and both deserve their own change with the gate watching.

app-shell.spec.ts has been in the repo unexecuted: enable-playwright was
unset, so the shared workflow reported E2E as 'skipped' on every run.

The suite is only 7 tests, but each one targets a Vue 3 failure mode that
is invisible without a browser — a failed mount that leaves the
server-rendered container in place, an unregistered icon that paints
nothing rather than a fallback, a tree-shaken widget registration that
renders .cn-unknown-widget with an empty console, and Vue Router 4's
removed 'path: *' wildcard leaving <main> blank with no 404.

This repo is the scaffold every Conduction app is cloned from, so a
silent break here propagates fleet-wide on the next app-create.
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/nextcloud-app-template @ ac1fa14

Check PHP Vue Security License Tests
lint
phpcs
phpmd
psalm
phpstan
phpmetrics
eslint
stylelint
build
composer ✅ 100/100
npm ✅ 738/738
PHPUnit
Newman
Playwright

Quality workflow — 2026-08-03 22:25 UTC

Download the full PDF report from the workflow artifacts.

With enable-playwright switched on, the suite could not start at all:
resolveBaseURL() threw 'Refusing to run against http://localhost:8080:
:8080 is the SHARED dev container' while loading the config, so zero
tests ran.

The guard is right about a developer machine and wrong about a runner.
On CI, :8080 is the job's OWN 'php -S 0.0.0.0:8080', created and torn
down inside that job — the most disposable instance there is, and the
only one the shared workflow offers. Refusing it protects nothing and
just makes the job permanently unable to run.

Gate the refusal on not-CI. Off CI the behaviour is byte-for-byte
unchanged, including the no-fallback rule. Same CI-gated exemption
opencatalogi's ci-seed.sh and petstore's _base-url.ts already use.

Also accept BASE_URL, which is the variable the shared quality.yml
Playwright job actually exports; checked last so an explicit local
override still wins.
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/nextcloud-app-template @ 967be1e

Check PHP Vue Security License Tests
lint
phpcs
phpmd
psalm
phpstan
phpmetrics
eslint
stylelint
build
composer ✅ 100/100
npm ✅ 738/738
PHPUnit
Newman
Playwright

Quality workflow — 2026-08-03 22:39 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde

Copy link
Copy Markdown
Contributor Author

First real verdict: 5 passed / 5 failed (2.5 min)

Two commits were needed before the suite could report anything at all, and both were genuine blockers rather than tuning:

  1. The suite could not start. resolveBaseURL() threw Refusing to run against http://localhost:8080: :8080 is the SHARED dev container while loading the config, so zero tests ran. On a runner, :8080 is the job's own php -S, created and destroyed inside the job — the guard was protecting nothing and blocking everything. It is now gated on not-CI; off CI the behaviour is byte-for-byte unchanged.

Result

✓  1 [setup]    authenticate as admin
✓  2 [chromium] the Vue 3 app actually mounts CnAppRoot
✘  3 [chromium] every manifest page renders its own content
✘  5 [chromium] no built-in widget resolves to the "Widget unavailable" placeholder
✘  7 [chromium] navigation icons render as real glyphs
✘  9 [chromium] dashboard widgets have non-zero width
✘ 11 [chromium] the router catch-all redirects unknown paths to the dashboard
✓ 13 [chromium] the admin settings panel mounts from its own entry point

Reading it

The two passing app-shell tests are the load-bearing ones for diagnosis: CnAppRoot genuinely mounts, and (asserted inside that same test) the manifest prop reaches App with its 5 pages. The admin settings panel — a separate webpack entry with its own createApp().mount() — also mounts.

So the bootstrap is fine. What is broken is everything downstream of it:

  • <main> renders no page content on /, /examples or /settings (element(s) not found).
  • No built-in dashboard widget resolves — the .cn-unknown-widget placeholder assertion fails alongside empty <main>.
  • Nav icons paint zero glyphs.
  • The router catch-all never matches, so an unknown path leaves <main> empty.

Every one of those is a failure mode the spec file's own comments predicted for an incomplete Vue 3 migration, and each produces a page that still renders with an empty console — which is exactly why none of it was noticed while the job was reporting skipped.

Status

Not ready to merge. These are real app regressions, not infrastructure, and merging would make development red. The fleet's own precedent (openregister's caller: "a gate that is red on arrival is a gate nobody turns on") is to land a green floor and grow it.

This matters more here than anywhere else: this repo is the scaffold every Conduction app is cloned from, so these regressions propagate on the next app-create.

Nothing was skipped, weakened, retimed, or allow-listed.

The manifest declares "dependencies": ["openregister"] and CnAppRoot
renders the "Required apps are missing" gate instead of the router outlet
when it is absent. On a bare CI Nextcloud that gate has no <main>, no nav
and no widgets, so all five app-shell failures were one cause: a missing
dependency, not five broken assertions.
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/nextcloud-app-template @ d4473ef

Check PHP Vue Security License Tests
lint
phpcs
phpmd
psalm
phpstan
phpmetrics
eslint
stylelint
build
composer ✅ 100/100
npm ✅ 738/738
PHPUnit
Newman
Playwright

Quality workflow — 2026-08-04 05:35 UTC

Download the full PDF report from the workflow artifacts.

src/main.js builds the router with createWebHistory(generateUrl(...)), so
the history base is /index.php/apps/app-template wherever mod_rewrite is
not configured — which is exactly how the shared CI workflow serves
Nextcloud. Specs that hard-coded the pretty form landed on a page whose
router could not strip its own base, matched nothing, and fell through to
the catch-all, rendering the dashboard under a 200 with no redirect.

That produced a wrong-content failure on the manifest-pages spec and two
FALSE PASSES: the built-in-widget spec and the catch-all spec were both
asserting against the dashboard they had been silently bounced to, on
every route they believed they were visiting.

Resolve the base from OC.generateUrl — the same function @nextcloud/router
wraps — instead of guessing it. No assertion changed.
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/nextcloud-app-template @ 120946f

Check PHP Vue Security License Tests
lint
phpcs
phpmd
psalm
phpstan
phpmetrics
eslint
stylelint
build
composer ✅ 100/100
npm ✅ 738/738
PHPUnit
Newman
Playwright

Quality workflow — 2026-08-04 05:47 UTC

Download the full PDF report from the workflow artifacts.

Unused type param name and missing JSDoc @param declarations.
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/nextcloud-app-template @ b76dcee

Check PHP Vue Security License Tests
lint
phpcs
phpmd
psalm
phpstan
phpmetrics
eslint
stylelint
build
composer ✅ 100/100
npm ✅ 738/738
PHPUnit
Newman
Playwright

Quality workflow — 2026-08-04 05:53 UTC

Download the full PDF report from the workflow artifacts.

@rubenvdlinde
rubenvdlinde merged commit 75851d7 into development Aug 4, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants