Skip to content

test(e2e): stop 18 backend-only scenarios laundering gate-19 coverage through an always-skipped test #765

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rubenvdlinde merged 3 commits into from
Aug 8, 2026
Merged

test(e2e): stop 18 backend-only scenarios laundering gate-19 coverage through an always-skipped test #765

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d514e20
test(e2e): stop 18 backend-only scenarios laundering gate-19 coverage…
rubenvdlinde Aug 8, 2026
e6a43bf
test(e2e): TEMPORARY sentinel — positive control that the E2E job can…
rubenvdlinde Aug 8, 2026
a6c7b70
test(e2e): remove the temporary sentinel
rubenvdlinde Aug 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions openspec/specs/document-zaakdossier/spec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ This join pattern allows a single document to be linked to multiple cases withou
`registratiedatum` → current timestamp
- **AND** the file MUST receive system tags `object:{uuid}` and `doctype:{type}` via `TaggingHandler`

@e2e exclude Service-layer contract (`ZaakdossierService::uploadDocument` + `CreateFileHandler`/`TaggingHandler` wiring) asserted in tests/Unit/Service/ZaakdossierServiceTest and at the API layer in tests/newman/document-zaakdossier.postman_collection.json; there is no UI assertion that can observe the join-object and system-tag side effects.

#### Scenario: REQ-ZAK-001b Same informatieobject linked to two cases without duplication

- **GIVEN** informatieobject `advies-brandweer.pdf` is already linked to `vergunning-1`
Expand All @@ -41,6 +43,8 @@ This join pattern allows a single document to be linked to multiple cases withou
- **AND** the informatieobject record itself MUST NOT be duplicated
- **AND** both zaak dossier views MUST show the document

@e2e exclude Non-duplication of the informatieobject across two zaakinformatieobject joins is a persistence invariant asserted in tests/Unit/Service/ZaakdossierServiceTest; the UI cannot distinguish "one record, two joins" from "two records".

#### Scenario: REQ-ZAK-001c Unlink preserves informatieobject

- **GIVEN** `bijlage.pdf` is linked to `vergunning-1` via `zaakinformatieobject` `zio-0001`
Expand All @@ -49,6 +53,8 @@ This join pattern allows a single document to be linked to multiple cases withou
- **AND** the informatieobject itself MUST remain in the register
- **AND** the Nextcloud file MUST remain in Nextcloud Files

@e2e exclude "Only the join record is deleted, the informatieobject and the Nextcloud file survive" is a storage-layer assertion (tests/Unit/Service/ZaakdossierServiceTest) — the dossier UI shows the same absence for a true delete and a mere unlink.

---

### Requirement: REQ-ZAK-002 Informatieobjecten MUST follow the ZGW status lifecycle concept → definitief → gearchiveerd
Expand All @@ -69,6 +75,8 @@ MUST validate each transition and:
- **AND** a subsequent upload of a new version MUST return HTTP 409 with message
"Definitieve documenten kunnen niet worden gewijzigd"

@e2e exclude Status-lifecycle enforcement and the HTTP 409 on re-upload are server contracts asserted in tests/Unit/Service/ZaakdossierServiceTest and in the Newman collection; the UI has no control that can attempt a blocked version upload.

#### Scenario: REQ-ZAK-002b Reverse transition definitief → concept is rejected

- **GIVEN** informatieobject `besluit.pdf` with `status` = `definitief`
Expand All @@ -77,20 +85,26 @@ MUST validate each transition and:
- **AND** the response body MUST indicate the invalid transition
- **AND** the `status` and `vergrendeldOp` MUST remain unchanged

@e2e exclude Reverse-transition rejection is an API contract (HTTP 400 from `PATCH /api/informatieobjecten/{id}/status`) asserted in the Newman collection; the UI never offers the backwards transition, so no UI flow can exercise it.

#### Scenario: REQ-ZAK-002c Deletion of definitief document is rejected

- **GIVEN** informatieobject `besluit.pdf` with `status` = `definitief`
- **WHEN** a user attempts to delete the document via `DeleteFileHandler` or the dossier UI
- **THEN** the deletion MUST be rejected with HTTP 409 Conflict
- **AND** the informatieobject record MUST remain intact

@e2e exclude HTTP 409 from `DeleteFileHandler` for definitief documents is a backend guard asserted in tests/Unit/Service/ZaakdossierServiceTest + Newman; the guard must hold regardless of whether the UI renders a delete affordance.

#### Scenario: REQ-ZAK-002d Transition definitief → gearchiveerd is permitted

- **GIVEN** informatieobject with `status` = `definitief`
- **WHEN** `transitionStatus(id, 'gearchiveerd')` is called (e.g., by the archival process)
- **THEN** the status MUST update to `gearchiveerd`
- **AND** the transition MUST be recorded in the OpenRegister audit trail

@e2e exclude Archival transition is driven by a background/archival process, not a UI control, and the audit-trail write is an OpenRegister side effect; asserted in tests/Unit/Service/ZaakdossierServiceTest.

---

### Requirement: REQ-ZAK-003 Access to informatieobjecten MUST be gated by vertrouwelijkheidaanduiding
Expand All @@ -109,6 +123,8 @@ and download operation. Guards MUST be checked at the service layer, not only in
- **AND** the API MUST respond with HTTP 403 Forbidden
- **AND** the document MUST NOT appear in the dossier listing for that user

@e2e exclude Clearance guard (`InformatieobjectAccessGuard::canRead`) is asserted in tests/Unit/Service/InformatieobjectAccessGuardTest; exercising it through the UI would need two seeded users at different clearance levels, which the Playwright environment does not provision.

#### Scenario: REQ-ZAK-003b Filtered dossier listing respects clearance

- **GIVEN** a dossier with 10 documents at various vertrouwelijkheidaanduiding levels
Expand All @@ -118,20 +134,26 @@ and download operation. Guards MUST be checked at the service layer, not only in
with vertrouwelijkheidaanduiding above `intern` from the response
- **AND** documents with `openbaar`, `beperkt_openbaar`, or `intern` MUST be returned

@e2e exclude Listing-filter matrix (`filterDossierForUser`) is asserted per clearance level in tests/Unit/Service/InformatieobjectAccessGuardTest; a UI check could only observe one row count, not the filter contract.

#### Scenario: REQ-ZAK-003c Public share rejected for confidential documents

- **GIVEN** informatieobject has `vertrouwelijkheidaanduiding` = `vertrouwelijk`
- **WHEN** a user attempts to create a public share link for this document
- **THEN** `InformatieobjectAccessGuard.canPublish(informatieobject)` MUST return `false`
- **AND** the share creation MUST be blocked with an appropriate error message

@e2e exclude Public-share refusal (`canPublish`) is a service-layer guard asserted in tests/Unit/Service/InformatieobjectAccessGuardTest; share creation runs through Nextcloud's own sharing UI, outside the procest e2e surface.

#### Scenario: REQ-ZAK-003d Default vertrouwelijkheidaanduiding from informatieobjecttype

- **GIVEN** informatieobjecttype `intern-advies` has default `vertrouwelijkheidaanduiding` = `intern`
- **WHEN** a user uploads a document of this type without specifying a classification
- **THEN** the informatieobject MUST receive `vertrouwelijkheidaanduiding` = `intern`
- **AND** the user MAY override to a more restrictive level but NOT to a less restrictive one

@e2e exclude Default-classification inheritance from informatieobjecttype and the one-way override rule are resolved server-side on create; asserted in tests/Unit/Service/ZaakdossierServiceTest + Newman.

---

### Requirement: REQ-ZAK-004 The zaakdossier view MUST render documents grouped by informatieobjecttype
Expand Down Expand Up @@ -244,13 +266,17 @@ metadata search MUST be supported via `FileSearchController`.
- **AND** the document MUST become full-text searchable once extraction completes
- **AND** the upload response MUST not wait for extraction to finish

@e2e exclude Asynchronous job scheduling (`FileTextExtractionJob`) completes outside the request the browser observes; asserted in tests/Unit/Service/ZaakdossierServiceTest and by the background-job unit tests.

#### Scenario: REQ-ZAK-007b Dossier search returns only matching documents

- **GIVEN** a dossier with 25 documents, 3 containing the phrase "brandveiligheidsplan"
- **WHEN** the user searches "brandveiligheidsplan" in the dossier search bar
- **THEN** exactly the 3 matching documents MUST be returned with highlighted snippets
- **AND** documents not in this dossier MUST NOT appear in results

@e2e exclude Dossier-scoped search relevance depends on a 25-document seeded corpus with extracted full text; asserted at the API layer in tests/newman/document-zaakdossier.postman_collection.json.

---

### Requirement: REQ-ZAK-008 Bulk operations MUST support ZIP export with manifest, bulk status transition, and bulk metadata update
Expand Down Expand Up @@ -278,6 +304,8 @@ informatieobjecttype sub-folders plus a `manifest.csv` with columns:
- **THEN** the ZIP MUST contain only 6 documents
- **AND** `manifest.csv` MUST contain only those 6 rows

@e2e exclude ZIP contents and manifest rows are inspected in tests/Unit/Service/ZipManifestBuilderTest; the browser receives an opaque streamed download that Playwright cannot open to count clearance-excluded entries.

#### Scenario: REQ-ZAK-008c Bulk status transition returns per-document result

- **GIVEN** the user selects 5 concept documents and clicks "Markeer als definitief"
Expand Down Expand Up @@ -305,6 +333,8 @@ streaming begins.
- **AND** the response MUST include `Content-Range: bytes 0-1048575/54525952`
- **AND** the first 1 MB of file content MUST be returned

@e2e exclude HTTP 206 + `Content-Range` on a Range request is a transport contract asserted in tests/Unit/Http/RangeStreamResponseTest and in the Newman collection; a browser page navigation cannot set a Range header or assert partial-content framing.

#### Scenario: REQ-ZAK-009b Download blocked when user lacks clearance

- **GIVEN** informatieobject has `vertrouwelijkheidaanduiding` = `geheim`
Expand All @@ -313,6 +343,8 @@ streaming begins.
- **THEN** `InformatieobjectAccessGuard.canRead()` MUST deny access
- **AND** the server MUST return HTTP 403 Forbidden before streaming any content

@e2e exclude "403 before any bytes stream" is a guard-ordering assertion on the ZGW DRC endpoint, covered in tests/Unit/Service/InformatieobjectAccessGuardTest + Newman; the UI never links a document the caller may not read.

---

### Requirement: REQ-ZAK-010 Existing linked files MUST be back-filled with ZGW informatieobject metadata
Expand All @@ -334,6 +366,8 @@ informatieobject MUST be skipped.
and `integriteit.waarde` = SHA-256 hash of the file content
- **AND** a `zaakinformatieobject` join MUST link the informatieobject to `vergunning-1`

@e2e exclude `BackfillInformatieobjectMetadata` is an `info.xml` repair step that runs at install/upgrade, never from a UI action; asserted in the repair-step unit tests.

#### Scenario: REQ-ZAK-010b Back-fill is idempotent on re-run

- **GIVEN** `BackfillInformatieobjectMetadata` has already run and created informatieobject
Expand All @@ -342,3 +376,5 @@ informatieobject MUST be skipped.
- **THEN** NO new `informatieobject` or `zaakinformatieobject` records MUST be created
- **AND** existing records MUST remain unchanged

@e2e exclude Repair-step idempotence is proven by running the step twice and comparing record counts — a PHPUnit-level assertion (repair-step unit tests); there is no UI that re-runs a repair step.

72 changes: 29 additions & 43 deletions tests/e2e/spec-coverage/document-zaakdossier.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,11 @@
* per the Playwright-UI-only / Newman-for-API split. Each scenario below maps to
* those asserting layers; the @e2e tags give gate-19 the traceability link.
*
* Each test carries a defensive skip so a missing dev container / not-yet-seeded
* case never produces a false failure.
* Each test carries a defensive, *conditional* skip (`if (!response) …`) so a
* missing dev container never produces a false failure. Tests that can never
* run in this environment are declared `test.fixme` with the blocking issue
* number rather than skipped — a skip that is unconditional reads as an
* environment condition while being permanent.
*
* Note: Use /apps/procest/<route> (not /index.php/apps/procest/<route>) so the
* Vue history-mode router can resolve the route correctly.
Expand All @@ -41,72 +44,55 @@ test.describe('document-zaakdossier spec coverage', () => {
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-005a-drag-drop-triggers-metadata-dialog-before-upload
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-005b-per-file-upload-progress-with-shared-metadata
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-005c-file-validation-blocks-executable-uploads
test('dossier upload + sort flow is reachable from the dossier tab', async ({ page }) => {
// Quarantined by declaration: the upload dialog + sort controls need a case
// seeded with a linked informatieobject, which nothing in this repo provides
// for the Playwright environment. Previously an unconditional
// `test.skip(true, …)` at the end of the body, which reads as an environment
// condition but can never become true. See #764.
test.fixme('dossier upload + sort flow is reachable from the dossier tab — blocked by #764 (no seeded case fixture)', async ({ page }) => {
const response = await page.goto('/index.php/apps/procest/cases').catch(() => null)
if (!response) {
test.skip(true, 'Procest dev container not reachable')
return
}
const bodyText = await page.locator('body').innerText().catch(() => '')
expect(bodyText).not.toContain('Internal Server Error')
// The upload dialog + sort controls require a seeded case fixture.
test.skip(true, 'Requires a seeded case fixture in the dev container')
})

// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-006a-concept-document-version-history-shows-restore
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-006b-restore-is-disabled-for-definitief-documents
test('version history panel is reachable from a dossier row', async ({ page }) => {
// Quarantined by declaration — needs a concept document with more than one
// file version on a seeded case. See #764.
test.fixme('version history panel is reachable from a dossier row — blocked by #764 (no seeded versioned fixture)', async ({ page }) => {
const response = await page.goto('/index.php/apps/procest/cases').catch(() => null)
if (!response) {
test.skip(true, 'Procest dev container not reachable')
return
}
test.skip(true, 'Requires a seeded case fixture with versioned files')
await expect(page.locator('body')).not.toContainText('Internal Server Error', { timeout: 10000 })
})

// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-008a-zip-export-includes-manifestcsv-and-type-sub-folders
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-008c-bulk-status-transition-returns-per-document-result
test('bulk actions bar appears when dossier documents are selected', async ({ page }) => {
const response = await page.goto('/index.php/apps/procest/cases').catch(() => null)
if (!response) {
test.skip(true, 'Procest dev container not reachable')
return
}
test.skip(true, 'Requires a seeded case fixture with multiple documents')
})

// The following scenarios are backend-enforced (no dedicated UI surface) and
// are verified by the PHPUnit + Newman layers named in the file header. The
// @e2e tags below give gate-19 the traceability link to this UI suite, which
// drives the same dossier surface the backend serves.
//
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-001a-upload-creates-informatieobject-and-zaakinformatieobject
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-001b-same-informatieobject-linked-to-two-cases-without-duplication
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-001c-unlink-preserves-informatieobject
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-002a-transition-concept-definitief-locks-the-file
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-002b-reverse-transition-definitief-concept-is-rejected
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-002c-deletion-of-definitief-document-is-rejected
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-002d-transition-definitief-gearchiveerd-is-permitted
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-003a-user-below-clearance-level-cannot-read-a-document
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-003b-filtered-dossier-listing-respects-clearance
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-003c-public-share-rejected-for-confidential-documents
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-003d-default-vertrouwelijkheidaanduiding-from-informatieobjecttype
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-007a-upload-triggers-async-text-extraction
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-007b-dossier-search-returns-only-matching-documents
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-008b-zip-excludes-documents-above-caller-clearance
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-009a-zgw-drc-endpoint-streams-large-file-with-range-support
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-009b-download-blocked-when-user-lacks-clearance
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-010a-back-fill-creates-informatieobject-for-pre-existing-file
// @e2e openspec/specs/document-zaakdossier/spec.md#req-zak-010b-back-fill-is-idempotent-on-re-run
test('dossier backend-enforced scenarios are served by the dossier surface', async ({ page }) => {
// Quarantined by declaration — needs a case seeded with two or more dossier
// documents so a multi-select can raise the bulk bar. See #764.
test.fixme('bulk actions bar appears when dossier documents are selected — blocked by #764 (no multi-document fixture)', async ({ page }) => {
const response = await page.goto('/index.php/apps/procest/cases').catch(() => null)
if (!response) {
test.skip(true, 'Procest dev container not reachable')
return
}
await expect(page.locator('body')).not.toContainText('Internal Server Error', { timeout: 10000 })
// Behavioural assertions for these scenarios live in PHPUnit + Newman
// (see file header); the UI shell health is the Playwright-layer check.
test.skip(true, 'Backend-enforced — asserted in PHPUnit/Newman, not UI')
})

// The 18 backend-enforced scenarios (REQ-ZAK-001a…c, 002a…d, 003a…d, 007a…b,
// 008b, 009a…b, 010a…b) used to be listed here as `@e2e` tags on a test whose
// body ended in an unconditional `test.skip(true, 'Backend-enforced …')`.
// That test reported *skipped* on every run while still supplying gate-19
// with a traceability link for all 18 — coverage recorded by a test that
// never executed. The scenarios are genuinely backend-only, so their
// traceability now lives where the gate provides for it: a reason-bearing
// `@e2e exclude` on each scenario in
// openspec/specs/document-zaakdossier/spec.md. Their behaviour is asserted in
// PHPUnit + Newman (see the file header for the exact suites).
})
Loading