ZERO2 Monitoring Dashboard — Sprint 2 Full-stack Foundation

Current baseline: 2.1.0-sprint2-ui-batch1 Runtime scope: React web application + NestJS API + background worker + PostgreSQL

ZERO2 is a clinical monitoring application that imports a Clinical One Subject Data export, reconstructs the current subject state, projects the approved Stage 1 Tracker fields, calculates monitoring status, and exposes controlled read/write workflows with audit history.

Sprint 2 batches 1–6 retain the accepted Stage 1 behaviour while adding a versioned API contract, database-verified JWT permissions, persisted uploads, a restart-safe worker, explicit typed stage projections, and scheduled status recalculation, versioned filter configuration, and safe query compilation. A report-instance Stage 2 draft projection now keeps repeated ZERO Reports separate and exposes read-only tracking middleware. A client may consume the API on port 3000. Batch 6 closes the backend integration loop with persisted worker-runtime heartbeats, database/storage/worker health checks, non-root containers, upgrade-safe upload-volume permissions, and one-command Docker verification. UI batch 1 adds an English React workspace for desktop and tablet, same-origin API proxying, role-aware navigation and actions, configuration editing, import/job operations, subject and report monitoring, and protected on-demand views for sensitive import rows and audit snapshots.

Documentation

• Sprint 1 implementation, corrected problems, and improvement roadmap
• Backend API reference
• Sprint 2 batch 1 handoff
• Sprint 2 batch 2 handoff
• Sprint 2 batch 2 test-pack correction R1
• Sprint 2 batch 3 handoff
• Sprint 2 batch 4 handoff
• Sprint 2 batch 5 handoff
• Sprint 2 batch 6 handoff and Docker acceptance
• Sprint 2 UI batch 1 handoff

Release status

The inherited Sprint 1 backend completed clean-machine Windows/Docker UAT with the supplied Clinical One long-form data:

• Docker deployment and three Prisma migrations verified;
• authentication and ADMIN / NCC_MONITOR / READ_ONLY authorization verified;
• 25 extended API, security, data, and reliability tests passed;
• 32,784 real event rows imported into 138 subjects with zero row failures;
• all 25 exact Study Design mapping rules found;
• full repeat import produced no duplicate subjects, raw events, or monitoring values;
• PostgreSQL backup restored into an isolated database with all compared table counts matching.

Sprint 2 UI batch 1 has passed 41 backend Jest suites (133 tests), 4 frontend Vitest suites (9 tests), both production builds, and Prisma schema validation. The inherited batch 6 baseline also passed lint, Python syntax checks, and its dependency audit. Coverage includes report projection, idempotency, read-only API, permissions, DTOs, worker heartbeat ownership, dependency health, import integration, sensitive-detail redaction/reveal contracts, web API utilities, role mapping, and OpenAPI. The Docker/real-data UAT command is supplied, but must be run on a Docker-capable host before final acceptance.

Included capabilities

• NestJS 11 API on Node.js 22.
• PostgreSQL 16 persistence through Prisma 5.22.
• Committed migrations and non-destructive repeat seed behaviour.
• JWT authentication with bounded expiry, database revocation, and exact CORS allow-list.
• Five demonstration roles mapped to explicit permissions.
• English React 19 web application with session-scoped JWT handling.
• Desktop and tablet layouts with permission-aware navigation and actions.
• Redacted, collapsed import-error and audit summaries with explicit, audited, on-demand sensitive-detail retrieval for authorised roles.
• Legacy /api routes plus /api/v1 and generated OpenAPI documentation.
• Disk-streamed Clinical One CSV parsing with report-preamble detection.
• CREATED / UPDATED / REMOVED / CLEARED event replay.
• Repeating-section association and Stage 1 field projection.
• Typed per-stage effective-field projections with recorded origin/rule version.
• Persisted per-stage status snapshots and durable manual/daily recalculation.
• Versioned JSON/UI-manageable filter profiles with validation, activation, rollback, deterministic export, and audit history.
• Whitelist-only configured subject queries over approved Subject/projection fields.
• Report-instance Stage 2 draft projection with typed values, raw-event lineage, revisions, pending/empty states, and read-only list/detail APIs.
• Stable raw-row hash deduplication and repeat-import idempotency.
• Persistent file metadata, jobs, attempts, progress, heartbeat, exponential retry, and stale-worker recovery.
• Persisted idle/processing worker-runtime heartbeat with stale-instance and replacement-process protection.
• Import-to-raw-event observation lineage across repeat imports.
• Subject search, filtering, sorting, pagination, and detail.
• Manual NSC, approved due-date override, and subject hide/restore.
• Transactional status recalculation and audit creation.
• Dashboard summary, analytics, full filtered CSV export, and audit APIs.
• Public liveness, database/upload readiness, and full-system worker health.
• Non-root API/worker images with an upgrade-safe upload-volume initializer.
• Non-root Nginx web image with same-origin /api reverse proxy and security headers.
• Jest/Vitest unit and contract tests plus an extended pytest regression pack.

Clinical One values are read-only. The only approved local subject writes are NSC Assignment, Central Review Due Date override, and hide/restore.

Architecture

flowchart TD
  Browser[Desktop or tablet browser] -->|HTTP 8080| Web[React SPA on Nginx]
  Web -->|same-origin /api proxy| API[NestJS API]
  Client[Direct API client] -->|HTTP 3000| API
  CSV[Clinical One CSV] -->|stream to volume| API
  API -->|enqueue| DB[(PostgreSQL 16)]
  Worker[Background worker] -->|claim and heartbeat| DB
  Worker -->|read stored CSV| Files[(Upload volume)]
  API --> Files
  Worker -. future source adapter .-> C1[Clinical One]

Loading

The current /api/sync/* implementation is a mock/test seam. It is not a production Clinical One integration.

Repository layout

zero2-mvp/
  web/                       React/Vite UI, tests, and Nginx configuration
  src/
    common/                   security, guards, utilities
    configuration/            version lifecycle and safe filter compiler
    jobs/                     queue, job API, worker loop
    projections/              effective fields and per-stage status jobs
    reports/                  report-instance Stage 2 draft middleware
    runtime/                  worker registration and heartbeat state
    storage/                  persisted-file access
    modules/                  auth, subjects, import, status, reporting
    worker.ts                 worker process entry point
    worker-health.ts          worker container health command
  prisma/
    migrations/               nine committed database migrations
    schema.prisma             relational data model
    seed.ts                   non-destructive demo/catalogue/default config seed
  docs/                       design, repair, and API documentation
  config/                     trusted default filter-profile JSON
  samples/                    non-clinical sample CSV
  scripts/                    PowerShell smoke/stress helpers
  zero2_extended_test_pack/   pytest and optional k6 tests
  Dockerfile
  Dockerfile.web
  docker-compose.yml
  .env.example
  package.json

Never place Clinical One exports, .env, database dumps, or test exports in the repository. The supplied .gitignore excludes common sensitive filenames, but repository users remain responsible for checking every commit.

Prerequisites

Recommended Docker workflow:

• Windows 10/11, macOS, or Linux;
• Docker Desktop/Engine with Compose v2 or later;
• at least 3 GiB free memory for the web, API, worker, and database containers;
• PowerShell 7 for the Windows examples.

Optional local-development workflow:

• Node.js >=22.12.0;
• npm >=10.9.0;
• a reachable PostgreSQL database;
• Python 3 for the extended pytest pack;
• k6 only when running the optional load test.

Configuration

Copy the example before starting:

Copy-Item .env.example .env

Principal variables:

Variable	Required	Purpose
NODE_ENV	Yes	Runtime environment
PORT	Yes	API listen port; Docker uses 3000
WEB_PORT	Docker	Host port for the web application; defaults to 8080
JWT_SECRET	Yes	Random non-default value, minimum 32 characters
JWT_EXPIRES_IN	Yes	Seconds or duration such as 30m, 8h, 7d
CORS_ORIGIN	Yes	Comma-separated exact allowed origins; no *
POSTGRES_DB / POSTGRES_USER / POSTGRES_PASSWORD	Docker	PostgreSQL container credentials; keep synchronized with DATABASE_URL
DATABASE_URL	Yes	PostgreSQL Prisma connection string
UPLOAD_DIR	Yes in Docker	Shared persisted CSV directory
WORKER_POLL_MS	No	Idle job polling interval
WORKER_HEARTBEAT_MS	No	Idle/processing runtime heartbeat interval; default 5 seconds
WORKER_HEALTH_STALE_AFTER_SECONDS	No	Full-system/worker health freshness window; default 20 seconds
WORKER_INSTANCE_ID	No	Stable worker instance name; leave empty in Docker to use its hostname
JOB_STALE_AFTER_SECONDS	No	Lease timeout used for crash recovery
JOB_MAX_ATTEMPTS	No	Automatic attempt limit
JOB_RETRY_DELAY_SECONDS	No	Base delay for exponential retry
IMPORT_CONCURRENCY	No	Hard cross-subject row concurrency limit
STATUS_RECALC_ENABLED	No	Enables daily status recalculation; defaults true
STATUS_RECALC_TIME_ZONE	No	IANA time zone for the daily slot
STATUS_RECALC_LOCAL_HOUR	No	First eligible local hour, 0..23
STATUS_RECALC_BATCH_SIZE	No	Subject page size for recalculation
STATUS_RECALC_CONCURRENCY	No	Bounded subject recalculation concurrency
DEMO_USER_PASSWORD	Development	Initial password for demonstration users
ENABLE_API_DOCS	No	Enables /api/docs and /api/openapi.json
CLINICAL_ONE_*	Future	Reserved for the unimplemented production adapter
MOCK_C1_SYNC	Test only	Enables mock sync outside production; defaults off

Generate a strong JWT secret in PowerShell without printing it:

$envPath = Join-Path (Get-Location) ".env"
$content = [System.IO.File]::ReadAllText($envPath)
$placeholder = "replace-this-with-a-random-secret-of-at-least-32-characters"

$bytes = New-Object byte[] 48
$rng = [System.Security.Cryptography.RandomNumberGenerator]::Create()
$rng.GetBytes($bytes)
$rng.Dispose()

$secret = [Convert]::ToBase64String($bytes)
$content = $content.Replace($placeholder, $secret)
[System.IO.File]::WriteAllText($envPath, $content)

Validate the Compose configuration:

docker compose config --quiet
docker compose config --services

Expected services:

db
storage-init
migrate
api
worker
web

Quick start with Docker

Build and start:

docker compose up --build --detach
docker compose ps --all

The one-shot storage-init service repairs ownership on both new and existing upload volumes. The one-shot migration service waits for PostgreSQL, applies all committed migrations, and runs the idempotent seed. API and worker start as the unprivileged node user only after both initializers succeed. The web container starts after the API is healthy and proxies /api without exposing a token or backend address in the bundle. Expected local endpoints:

Purpose	URL
Web application	http://localhost:8080
Versioned API through web proxy	http://localhost:8080/api/v1
API base	http://localhost:3000/api
Versioned API	http://localhost:3000/api/v1
Liveness	http://localhost:3000/api/health/live
API readiness (database + upload storage)	http://localhost:3000/api/health/ready
Full-system health (database + storage + worker)	http://localhost:3000/api/health/system
OpenAPI UI	http://localhost:3000/api/docs

Check the deployment:

docker compose logs --tail 200 api
docker compose logs --tail 200 worker
docker compose logs --tail 200 web
docker compose exec api ./node_modules/.bin/prisma migrate status
curl.exe -sS -i http://localhost:3000/api/health/live
curl.exe -sS -i http://localhost:3000/api/health/ready
curl.exe -sS -i http://localhost:3000/api/health/system
curl.exe -sS -i http://localhost:8080/web-health

PostgreSQL is deliberately not published on a host port.

Local demonstration accounts

Role	Email	Initial password
ADMIN	admin@zero2.local	Password123!
DATA_MANAGER	data.manager@zero2.local	Password123!
NCC_MONITOR	monitor@zero2.local	Password123!
AUDITOR	auditor@zero2.local	Password123!
READ_ONLY	viewer@zero2.local	Password123!

These accounts are local test fixtures. Set DEMO_USER_PASSWORD to create them. The seed does not reset passwords or reactivate disabled accounts. When NODE_ENV=production and no demo password is supplied, it creates no demo users.

First authenticated request

$baseUrl = "http://localhost:3000/api"

$login = Invoke-RestMethod `
  -Method Post `
  -Uri "$baseUrl/auth/login" `
  -ContentType "application/json" `
  -Body (@{
    email = "admin@zero2.local"
    password = "Password123!"
  } | ConvertTo-Json)

$headers = @{
  Authorization = "Bearer $($login.accessToken)"
}

Invoke-RestMethod `
  -Uri "$baseUrl/auth/me" `
  -Headers $headers

Invoke-RestMethod `
  -Uri "$baseUrl/subjects?page=1&pageSize=25" `
  -Headers $headers

Do not print or persist $login.accessToken in logs.

Importing Clinical One Subject Data

The accepted input is the Clinical One long-form Subject Data CSV, not a one-row-per-subject spreadsheet. The parser finds the Subject Number header after any report preamble and retains source row numbers.

Study Design metadata such as tgt_visit_form_question.csv is used for mapping QA and must not be uploaded as Subject Data.

Submit a real file asynchronously:

$subjectCsv = "C:\secure-location\Subject Data.csv"

$uploadJson = curl.exe -sS `
  -X POST `
  "$baseUrl/import/baseline" `
  -H "Authorization: Bearer $($login.accessToken)" `
  -F "file=@$subjectCsv;type=text/csv"

$upload = $uploadJson | ConvertFrom-Json
$batchId = $upload.id

$upload |
  Select-Object id, fileName, accepted, mode, status

Poll the batch:

$terminal = @("SUCCESS", "PARTIAL_SUCCESS", "FAILED")

do {
  $batch = Invoke-RestMethod `
    -Uri "$baseUrl/import/batches/$batchId" `
    -Headers $headers

  $batch |
    Select-Object status, totalRows, insertedRows, updatedRows, failedRows

  if ($terminal -notcontains $batch.status) {
    Start-Sleep -Seconds 5
  }
} while ($terminal -notcontains $batch.status)

All imports are asynchronous. The obsolete ?mode=sync path is rejected so API restarts cannot lose in-process work. Poll the batch or /api/jobs/:jobId.

Import-history, job, and error endpoints require the import:manage permission (ADMIN and DATA_MANAGER in the demonstration map). Error lists are always redacted. A raw source row is retrieved only from its dedicated endpoint after an explicit reveal action, and every successful reveal is audited.

Stage 1 mapping behaviour

The mapper first replays C1 history into a current active snapshot. It then links rows within the same visit/event instance and repeating section.

Important exact rules include:

• Registration Date: Registration / Patient Demographics / Last name / CREATED event time.
• NCC Comments: Sample Submission / NCC Enrolment Sample Tracking / Central review comments.
• Central Review Status: Sample Submission / NCC Enrolment Sample Tracking / Sent for central review.
• NSC Assignment: dashboard-owned manual field; not inferred from C1.
• Central Review Due Date: registration date + 14 calendar days unless an approved active override exists.

Liquid-biopsy tracking remains outside the agreed Stage 1 projection. Source oncologist, email, state/country, and postcode values are stored only in raw events; subject APIs and exports return presence booleans.

The supplied Subject Data export has no question verification-state column. The backend therefore leaves those states missing rather than guessing Verified.

Status calculation

Registration Date, NSC, and every applicable required Stage 1 field contribute to status. Sample Comments and NCC Comments are optional.

Field result	Meaning
COMPLETE	Required value has an accepted completed/verified state
PENDING	Present but not yet complete
MISSING	Required evidence is absent before due date
OVERDUE	Required work remains incomplete after due date
NOT_APPLICABLE	Conditional field does not apply

Overall priority:

OVERDUE > MISSING > PENDING > COMPLETE > NOT_APPLICABLE

Traffic-light mapping:

OVERDUE=RED, MISSING/PENDING=AMBER, COMPLETE=GREEN,
NOT_APPLICABLE=GREY

The due date is evaluated using the Australia/Sydney calendar boundary, so a subject does not become overdue at the start of its due date.

The current result is stored in SubjectStageStatus; its typed effective fields are stored in SubjectFieldProjection. Both record rule version stage1-v1. Import, NSC changes, and due-date overrides refresh the projection transactionally. The worker also enqueues one idempotent recalculation per configured local day, and an admin may enqueue one explicitly.

Main API groups

• /api/auth — login and current identity.
• /api/subjects — list, detail, typed stage projection, NSC, override, visibility, and subject audit.
• /api/import — CSV upload, batch polling, and errors.
• /api/jobs — durable progress, attempts, failure details, retry, and status recalculation enqueueing.
• /api/configurations — JSON import/export, draft filter CRUD, lifecycle, active configuration discovery, and safe configured subject queries.
• /api/dashboard — status totals and last successful ingestion.
• /api/analytics — site/status/light aggregates.
• /api/export — complete filtered CSV export.
• /api/audit — recent global audit events.
• /api/health — process liveness, API readiness, and full-system health.
• /api/sync — mock test seam, not production C1 sync.

Every group is also exposed beneath /api/v1. OpenAPI is available when ENABLE_API_DOCS=true.

Request bodies, parameters, roles, response examples, and errors are documented in the API reference.

Testing

Unit and build verification

Create .env first because Prisma validation requires DATABASE_URL.

npm ci
npm --prefix web ci
npm run lint
npm run verify:all
npm audit --omit=dev
npm --prefix web audit --omit=dev

npm run verify:all runs the backend Jest suites, NestJS production build, Prisma validation, frontend Vitest suites, TypeScript build, and Vite production bundle.

Complete Docker integration verification

The recommended UI batch 1 command builds the stack, waits for container health, checks one-shot initialization and all migrations, then runs a non-clinical async-import smoke path. It preserves named volumes and leaves the stack running for inspection:

.\scripts\verify_docker_stack.ps1

Add -RunExtendedTests for the complete pytest regression pack, or -StopAfter to remove containers and the network after success while still preserving named volumes.

API smoke test only

With the stack healthy:

.\scripts\smoke_test.ps1

Extended API regression pack

.\zero2_extended_test_pack\run_extended_tests.ps1

The helper creates .venv-zero2-tests and a unique project-local pytest temporary directory. It tests API contracts, roles, invalid inputs, imports, data quality, idempotency, and concurrency.

The tests create non-clinical test records. Use a dedicated test database or volume, not a production database.

Optional k6 load test

After installing k6:

.\zero2_extended_test_pack\run_k6_load_test.ps1

Database migrations and backups

Fresh Docker deployments apply all committed migrations automatically. Check state with:

docker compose exec api ./node_modules/.bin/prisma migrate status

Before upgrading or deleting containers, create a PostgreSQL backup. A safe custom-format example inside the database container is:

docker compose exec -T db `
  pg_dump `
  -U zero2 `
  -d zero2_dashboard `
  --format=custom `
  --no-owner `
  --no-privileges `
  --file=/tmp/zero2-backup.dump

Copy it to a protected host location with docker cp, verify its SHA-256, and periodically test restoration into a separate database. A dump contains clinical data and must not be committed or uploaded to an unapproved service.

Service lifecycle and data preservation

Show status/logs:

docker compose ps
docker compose logs --tail 200 api
docker compose logs --tail 200 worker

Stop without deleting containers:

docker compose stop

Remove containers/network while preserving the named database and upload volumes:

docker compose down

Do not run docker compose down -v unless permanent deletion of the database volume is explicitly intended and a verified backup exists.

Security checklist

Before any shared environment:

1. replace or disable demo accounts;
2. replace the Docker database password and update DATABASE_URL;
3. store JWT/database/C1 secrets in an approved secret manager;
4. use TLS and an explicit CORS_ORIGIN allow-list;
5. restrict access to import errors, audit snapshots, raw events, and backups;
6. add rate limiting, structured access logs, metrics, and alerts;
7. define backup encryption, retention, and restore testing;
8. do not expose PostgreSQL directly to the public network.

The API rejects missing, short, known-placeholder JWT secrets and wildcard credentialed CORS.

Known limitations

• No frontend is included.
• Production Clinical One OAuth/API sync is not implemented.
• JWT refresh tokens and a user-management UI are not implemented.
• Stage 2 remains stage2-draft-v1 until the project confirms its 18 source fields, 8 logical rules, required fields, and due/overdue semantics.
• CI/CD, central monitoring, and scheduled backups are not included yet.
• insertedRows and updatedRows are source-event counters, not distinct subject counters.

These items and their priority are detailed in the implementation and improvement roadmap.

Troubleshooting

API rejects JWT_SECRET

Create .env and replace the example value with a random secret of at least 32 characters. The example placeholder is intentionally rejected.

Readiness returns 503

Readiness checks PostgreSQL and the upload directory. Check both initializers, PostgreSQL, and API logs:

docker compose ps --all
docker compose logs --tail 200 db storage-init migrate api

Full-system health returns 503

If /api/health/ready succeeds but /api/health/system fails, the worker has not registered a fresh heartbeat. Check docker compose ps worker and docker compose logs --tail 200 worker. A processing worker remains healthy because its runtime heartbeat is independent of job progress updates.

Port 3000 is already in use

Identify the listener before changing anything:

Get-NetTCPConnection `
  -State Listen `
  -LocalPort 3000 `
  -ErrorAction SilentlyContinue

Import is rejected

Confirm the request uses multipart field name file, exactly one .csv, no additional form fields, and a file size below 100 MiB. XLS/XLSX is not supported in this release.

Import appears to remain at zero rows

The API only enqueues work. Confirm that the worker service is running and poll /api/import/batches/:id or /api/jobs/:jobId. A stopped worker leaves the job safely pending; a crashed worker lease is recovered after the configured stale interval.

License and data

The package is marked UNLICENSED. It contains source code and non-clinical test fixtures only. Do not add real Clinical One exports or restored database files to a commit or release archive.