# E2E Test Inventory Tracking doc for issue [#227](https://github.com/RXTX4816/cockpit-compose/issues/227). Maps every manual feature scenario in the [Testing Guide](../testing.md) §6-7 to a planned (or existing) automated spec in `e2e/`, grouped into implementation waves. Status legend: ✅ done, confirmed passing on live VMs (repeatedly) · ⚠️ done, one known intermittent flake tied to host-level resource contention, not app/test logic (see [E2E Test Reference](E2E-Test-Reference.md#verification-status)) · 🚧 planned, not written yet Every new test should assert a **real effect**, not just DOM visibility — e.g. after an edit, re-read the compose file content; after Up/Down, confirm the stack's actual status rather than only a badge class. See `e2e/helpers/` for the shared helpers that make this practical (`baseData`, `stacks.ts`, `runtime.ts`). ## Wave 1 — Core CRUD, canonical VMs (`arch-podman`, `arch-docker`, `arch-both`) | Testing guide § | Scenario | Spec (planned) | Status | |---|---|---|---| | 6.1 | Stack listing & status badges (running/stopped/partial/paused) | `e2e/stacks.spec.ts` | ✅ (status transitions covered) | | 6.2 | Import / discover downed stacks, depth setting, runtime-switch clears path | `e2e/stacks.spec.ts`, `e2e/runtime-toggle.spec.ts` | ✅ runtime-switch clears path covered, and depth setting's real effect on what's found (not just its clamping bounds, already covered via `adversarial.spec.ts`) | | 6.3 | Start (Up) — simple | `e2e/stack-lifecycle.spec.ts` | ✅ | | 6.4 | Start (Up) — with profiles | `e2e/stack-lifecycle.spec.ts` | ✅ | | 6.5 | Stop / Restart | `e2e/stack-lifecycle.spec.ts` | ✅ | | 6.6 | Down (remove containers) | `e2e/stack-lifecycle.spec.ts` | ✅ | | 6.7 | Kill | `e2e/stack-lifecycle.spec.ts` | ✅ | | 6.8 | Logs (filter, search, pause/continue, clear, refresh) | `e2e/logs.spec.ts` | ✅ | | 6.9 | Events (stream, table rows, stop, clear) | `e2e/events.spec.ts` | ✅ (streaming start/stop + real restart events land in the table; note: EventsModal auto-starts on mount, no manual "Stream events" click needed) | | 6.10 | Exec / shell into a running container | `e2e/exec.spec.ts` | ✅ | | 6.11 | Run command (one-off, --rm) | `e2e/run-command.spec.ts` | ✅ (args mode + --entrypoint override mode) | | 6.12 | Pull images (progress, unpinned warning, cancel) | `e2e/pull-images.spec.ts` | ✅ (unpinned warning + real re-pull + Run in Background) | | 6.13 | Stack info (services/images/volumes/networks tabs, shared networks) | `e2e/stack-info.spec.ts` | ✅ (found & fixed a real bug — see Notes; shared-networks case covered by connecting a real container to another project's network via SSH) | | 6.14 | Edit YAML (multi-file tabs, add/delete file, import, snapshot history/diff/restore) | `e2e/yaml-editor.spec.ts` | ✅ (add/delete file found & documented a real accessibility bug — see Notes and issue [#277](https://github.com/RXTX4816/cockpit-compose/issues/277)) | | 6.15 | Edit env file (table/raw modes, duplicate-key warning, add file) | `e2e/env-editor.spec.ts` | ✅ (found a real doc/behavior mismatch — see Notes) | | 6.16.1 | Prune — basic flow (containers; volumes appears unreachable via UI, see reference doc) | `e2e/prune.spec.ts` | ✅ image, shared-image and container prune tests all pass, Podman included since [#274](https://github.com/RXTX4816/cockpit-compose/issues/274) was fixed (#308) — see Notes | | 6.17 | Scale services (increase replicas, port-conflict warning) | `e2e/scale.spec.ts` | ✅ | | 6.19 | Create stack (manual/template/git-URL methods, validation bypass) | `e2e/create-stack.spec.ts` | ✅ (all three methods + validation bypass covered; git-URL clones a real public repo — see Notes for the fixture choice and required VM package, and issue [#277](https://github.com/RXTX4816/cockpit-compose/issues/277) for a related intermittent flake) | | downed-stacks bulk actions (select-all + bulk Up, commit 85fe38d) | Bulk select/Up on downed stacks table | `e2e/downed-stacks-bulk.spec.ts` | ✅ | | 6.22 | Ports — clickable link, localhost-only vs all-interfaces tooltip | `e2e/ports.spec.ts` | ✅ (external, localhost, and specific-bind cases all covered; found a doc/behavior mismatch — see Notes; the localhost/specific-bind fixture also surfaced and fixed real arch-podman VM infra bugs — see Notes) | | adversarial | Malformed YAML, nonexistent scan directory, scan-depth bounds | `e2e/adversarial.spec.ts` | ✅ all 3 fixed and stable (3/3 × 2 full-file runs) — see Notes, these were real test bugs, not flakes | ## Wave 2 — Runtime/engine-specific, full 9-VM matrix Run these against the full matrix (batched per §"Managing resource usage" in [VM Testing](VM-Testing)) since behavior depends on engine + rootless/rootful + distro. | Testing guide § | Scenario | Spec (planned) | Status | |---|---|---|---| | 6.20, 6.21 | Footer + runtime toggle (Docker↔Podman switch, rootless badge, socket path, "not installed" revert) | `e2e/runtime-toggle.spec.ts` | ✅ (`arch-both`; "not installed" case breaks the real `podman` binary via SSH, always restored) | | 7.1 | Docker rootless mode | `e2e/runtime-rootless.spec.ts` (`fedora-full` — the one VM with a genuine per-user Docker socket) | ✅ Rootless badge shown, full Up/Down lifecycle works under rootless Docker | | 7.2 | Podman via `podman compose` external provider | (every spec that runs against `arch-podman`) | ✅ — on `arch-podman`, `podman compose`'s "external compose provider" genuinely *is* `/usr/bin/podman-compose` (verified via SSH: `podman compose version` prints the "Executing external compose provider" banner naming that exact binary); every Wave 1/3/4 spec already run against this VM exercises this path, no dedicated spec needed | | 7.3 | Podman via `podman-compose` (Python, no docker-compose) | (every spec that runs against `arch-podman`) | ✅ — same finding as 7.2: `arch-podman` has no `docker-compose` at all and its `podman-compose` is confirmed genuine Python (`/usr/bin/podman-compose` is `#!/usr/bin/python`, reports `podman-compose version 1.6.0`), so 7.2 and 7.3 are literally the same code path on this VM, both already covered by the existing suite | | 7.4 | Podman root (system socket), no rootless socket at all — issue [#242](https://github.com/RXTX4816/cockpit-compose/issues/242) regression: discovery + Stack Info container list under real Cockpit Administrative access, and that the two agree on state *at the same moment* after a real transition (Troubleshooting.md's status-agreement regression) | `e2e/runtime-rootless.spec.ts` (`fedora-podman-rootful` only) | ✅ | | 7.5 | Neither runtime present — error state | `e2e/runtime-unavailable.spec.ts` | ✅ (`arch-docker`; hides both `docker` and the standalone `docker-compose` legacy binary via SSH — hiding `docker` alone isn't enough, `detectComposeCommand()` falls back to it) | | Rootless/Rootful socket-mode toggle (added alongside #242's fix) — switching modes when both sockets are detected, footer badge, stale-list refresh on switch | `e2e/socket-mode-toggle.spec.ts` | ✅ (`fedora-full` only, via `loginWithAdminAccess` — Rootful stays disabled without real admin access) | | superuser prompt | Compose file owned by another uid (`composeFileSuperuser()`, `src/api/cockpit.ts:185-215`) | `e2e/superuser-prompt.spec.ts` | ✅ (`fedora-full` only — the one VM with a genuine rootless Docker socket; see spec header comment for why a stricter-permissions fixture was tried and reverted) | | Recheck button vs. a socket that exists but doesn't respond (`checkSocketHealth()`, `docs/wiki/Troubleshooting.md`) | none | 🚧 not attempted — the only way to produce this state is mutating a shared VM's live `systemd --user` socket/service, which isn't reproducible and would need a full VM reset afterward; needs a dedicated VM image with a genuinely broken/stub socket (e.g. a Unix socket bound via `socat`/`nc` that accepts connections but never replies) baked into cloud-init instead of a live hack. Deliberately deferred rather than attempted this pass — `checkSocketHealth()`'s probe command (`podman ps` / `docker version`) has no visible client-side timeout in `src/api/cockpit.ts`, so a genuinely hung socket could make the health check (and this test) hang indefinitely; that needs verifying against a real hung socket in isolation before wiring up the full VM/spec, which is a larger, more novel piece of work than anything else in this pass. | | distro quirks | Debian cgroupfs pause/unpause workaround, Fedora SELinux `label=false`, pasta/nftables networking | covered implicitly by running Wave-1 lifecycle specs against the full matrix rather than a dedicated spec | ✅ partial — `e2e/stacks.spec.ts` + `stack-lifecycle.spec.ts` run clean on `debian-podman` (4/4 + 9/9), `debian-docker` (13/13), and `debian-both` (14/14, both fresh-start and rebuilt); `fedora-podman` 12/13 (1 known host-load flake, passed alone); `fedora-docker` 4/13 — the other 9 failures are **not** distro-quirk test gaps, they're issue [#272](https://github.com/RXTX4816/cockpit-compose/issues/272) (a real app-level race, also reproduces on `arch-docker`, not Fedora-specific) / 🚧 `fedora-both` — every automated Up attempt stalls silently (confirm dialog and progress-modal clicks all report success, but zero dockerd activity ever happens and the row never leaves the undiscovered "Down" list), reproduces consistently across multiple clean VM rebuilds and both parallel and serial runs, but a manual Up through a real browser session on the same VM works fine — looks like an automation/timing-specific issue rather than a real app bug, not yet root-caused; see Notes | ## Wave 3 — Edge cases / error states | Testing guide § | Scenario | Spec (planned) | Status | |---|---|---|---| | 5.3 / 6.4 | Multi-service stack with profiles | covered by 6.4 above | ✅ | | 5.4 / 6.14 | Multi-file compose (extra compose file) | covered by 6.14 above | ✅ | | 6.16.2-6.16.6 | Prune edge cases (shared image across stacks still in use, exited one-shot container removal by name) | `e2e/prune.spec.ts` (additional cases) | ✅ | | 6.16.7 | Dangling named volume | `e2e/prune.spec.ts` | ❓ the earlier "unreachable via the current UI" finding turned out to rest on a false premise specific to Podman — see issue [#274](https://github.com/RXTX4816/cockpit-compose/issues/274) and `e2e/prune.spec.ts`'s header comment; revisit once #274 is fixed before deciding whether this is still a product question | | 6.18 | Backup & restore (archive create, restore into new dir, restored stack starts) | `e2e/backup-restore.spec.ts` | ✅ (found & documented a UX asymmetry vs BackupModal — see Notes) | | 6.14 (malformed) | Save rejects invalid YAML with error details | `e2e/yaml-editor.spec.ts` (additional case) | ✅ (see Notes for a known flake under cumulative session load) | | 6.23 | Clickable external links show a warning modal before navigating | `e2e/stacks.spec.ts` (additional case) | ✅ — not in StackInfoModal (its port badges call `window.open()` directly, no confirmation, see #260/`ports.spec.ts`); the real warning-modal flow is `ContainerTable`'s per-service image link inside an expanded row, covered here instead | | healthcheck / restart-policy / named-network / crash-loop / long-logs fixture stacks (pre-staged, §5) | Status badge & info correctness for each fixture | `e2e/fixture-stacks.spec.ts` (parametrized cases) | ✅ | ## Wave 4 — Features absent from this inventory entirely Discovered while cross-checking `docs/wiki/*.md` against this table for a coverage push — these are real, documented features with zero automated coverage that were never even tracked here (the table above only maps `docs/testing.md` §6-7's manual scenarios). | Feature | Doc | Spec (planned) | Status | |---|---|---|---| | Background Tasks (queue, states, Stop/Remove, real container-state completion, runtime-switch cancellation, log view) | [Background-Tasks.md](Background-Tasks) | `e2e/background-tasks.spec.ts` | ✅ (Pending→Running→Complete against real container state, Stop terminates the underlying process, Remove actually drops the task, runtime-switch cancellation race verified on `arch-both`, log-view-through-panel found a real z-index bug — see issue [#283](https://github.com/RXTX4816/cockpit-compose/issues/283) and Notes) | | Bulk Actions on running stacks (per-layout selection, select-all indeterminate, per-action confirm, runtime-switch clears selection) | [Bulk-Actions.md](Bulk-Actions) | `e2e/bulk-actions.spec.ts` | ✅ (Power User checkbox, Minimal card-click, Pretty card-click, Unix bracket-toggle, runtime-switch-clears-selection all covered) | | Process Viewer / Top (`docker compose top` equivalent) | [Process-Viewer.md](Process-Viewer) | `e2e/process-viewer.spec.ts` | ✅ | | Keyboard shortcuts (U/D/L/E/I) | [Stacks-Dashboard.md](Stacks-Dashboard#keyboard-shortcuts) | fold into `e2e/stacks.spec.ts` | ✅ | | Layout selector (4 layouts) | [Stacks-Dashboard.md](Stacks-Dashboard#layout-options) | fold into `e2e/stacks.spec.ts` | ✅ | | Status filter chips + auto-refresh degrade/Retry | [Stacks-Dashboard.md](Stacks-Dashboard) | fold into `e2e/stacks.spec.ts` | ✅ (auto-refresh degrade/Retry verified against a real broken `podman` binary on the VM, not a mock — see Notes) | ## Wave 5 — Passing on fedora-full and arch-both Found via a final systematic audit (cross-referencing every `src/components/*.tsx` against every `e2e/*.spec.ts`, and every documented sub-flow in `docs/wiki/*.md` against actual test assertions, not just "a spec file with this feature's name exists"). **All ten are written as specs and pass on real VMs.** Full-suite runs on 2026-10-03, one VM at a time with `VM_MEM=2048` and `--workers=1`: - `fedora-full`: 88 passed, 0 failed, 0 retries, 9 skipped (VM-specific skips plus the `test.fixme` waiting on #319) - `arch-both`: 83 passed, 0 failed, 13 skipped; the one flaky test (scale port conflict, closing the dialog before Apply had settled) was fixed and then passed 3/3 with retries off The first real runs needed a fix-up pass, and found four app bugs, all fixed: the backup archive written under a different name than previewed, Restore ignoring its overwrite confirmation (#320), the Scale confirm heading showing a raw `{{name}}` placeholder, and Run in Background reporting Failed for a stack that was running (#319, fixed in #333 together with #272). The #319 spec is active again, joined by a regression test for #272 (closing the Up dialog with ✕ mid-run backgrounds the Up instead of discarding it). Both passed 3/3 with retries off on `fedora-full`. No spec in the suite is `test.fixme` anymore. Each asserts a real backend effect (engine state or files over SSH), in keeping with the rest of the suite. SSH-side engine checks go through the new `engineCli(projectName)` helper in `e2e/helpers/vm.ts`, which picks the CLI the app itself ends up using on that VM (podman on podman-only VMs, `sudo podman` on `fedora-podman-rootful`, docker elsewhere). | Feature | Doc | Spec | Status | |---|---|---|---| | Global image prune (`GlobalPruneModal` — the host-wide "Prune images" button) | [Prune-Resources.md](Prune-Resources#global-image-prune-all-stacks) | `e2e/prune.spec.ts` — 2 tests: removes gotify/server once gotify is fully down (verified via `images` over SSH); the acknowledgement checkbox gates the Prune button | ✅ passing on `fedora-full` + `arch-both` | | Pause / Unpause | [Managing-Stacks.md](Managing-Stacks#pause--unpause) | `e2e/stack-lifecycle.spec.ts` — every `multi` container reports `paused`, the menu flips to Unpause, then all report `running` | ✅ passing on `fedora-full` + `arch-both` | | Scale's port-conflict warning | [Scaling-Services.md](Scaling-Services) | `e2e/scale.spec.ts` — `scale-test`'s `web` (8085:80): inline warning, confirm-step alert + icon naming only `web`, Apply not blocked, never >1 running `web`, port-less `worker` still scales to 2 | ✅ passing on `fedora-full` + `arch-both` — **see correction 1** | | Command history (`HistoryDatalist`) | [Running-Commands.md](Running-Commands), [Shell-Access.md](Shell-Access) | `e2e/exec.spec.ts` + `e2e/run-command.spec.ts` — history cleared, command used, modal reopened, `` `