Repository navigation
Deployment and Operations
Docker Desktop with the Linux engine and Compose is required. The stack exposes
only http://127.0.0.1:8080, has no authentication, and is intended for local
use. It starts an empty PostgreSQL database; it does not import SQLite history.
From the project root in PowerShell, create five distinct random passwords in an
ignored runtime file, then start the services:
if (Test-Path .env.compose) { throw 'Keep the existing .env.compose; do not overwrite its passwords.' }
$names = 'TL_ADMIN_PASSWORD','TL_ANALYST_PASSWORD','TL_HUNT_PASSWORD','TL_EVALUATOR_PASSWORD','TL_TRIAGE_PASSWORD'
$bytes = [byte[]]::new(32)
$lines = foreach ($name in $names) { [System.Security.Cryptography.RandomNumberGenerator]::Fill($bytes); "$name=$([Convert]::ToHexString($bytes))" }
[System.IO.File]::WriteAllLines((Join-Path (Get-Location) '.env.compose'), $lines)
docker compose --env-file .env.compose up --build -d
docker compose --env-file .env.compose psOpen the dashboard at http://127.0.0.1:8080; /health checks API database
connectivity. PostgreSQL is not exposed on the host. The migration service creates
separate logins; the API receives only analyst and read-only hunting credentials.
Provider credentials can be supplied in .env.compose for explicitly initiated
enrichment; do not put secrets in VITE_* variables or images. Compose environment
variables are visible to users with Docker access. The memory limits total 2.625 GiB,
excluding Docker's own overhead. docker compose --env-file .env.compose down
stops the stack while retaining the database volume. Keep .env.compose: generating
new passwords does not change the PostgreSQL administrator password in an existing
volume.
This procedure covers the local Compose PostgreSQL database, not SQLite files. It
uses a data-only archive and the existing migrations to recreate the schema and
grants. Store the archive and revision file together in an access-restricted,
encrypted location outside the repository: they contain sensitive alerts, labels,
reviews and other investigation data. Keep .env.compose securely and separately;
the archive does not contain cluster roles, passwords or external provider keys.
Choose a new backup directory that does not already contain these filenames.
Stop all writers, including the optional worker and any external CLI or evaluation
processes, and do not run migrations during the backup. From the project root in
PowerShell, with the database running, set $backupDir to that directory:
docker compose --env-file .env.compose --profile jobs stop api worker
$db = docker compose --env-file .env.compose ps -q db
if (-not $db) { throw 'Start the database first.' }
docker exec $db psql -X -U triagelens_admin -d triagelens -At -v ON_ERROR_STOP=1 -c 'SELECT version_num FROM triagelens.alembic_version' | Set-Content -Encoding utf8 (Join-Path $backupDir 'revision.txt')
if ($LASTEXITCODE -ne 0) { throw 'Could not read migration revision.' }
docker exec $db pg_dump -U triagelens_admin -d triagelens -Fc --data-only --exclude-table-data=triagelens.alembic_version -f /tmp/triagelens-backup.dump
if ($LASTEXITCODE -ne 0) { throw 'Backup failed.' }
docker cp "${db}:/tmp/triagelens-backup.dump" (Join-Path $backupDir 'triagelens.dump')
if ($LASTEXITCODE -ne 0) { throw 'Copy failed.' }
docker exec $db rm /tmp/triagelens-backup.dumpResume the stack with docker compose --env-file .env.compose --profile jobs up -d
if the worker is used; omit --profile jobs otherwise. Check the archive and
revision file exist and regularly test recovery. A dump is a point-in-time snapshot,
not continuous recovery; changes after it are not included.
Restore first into a new, isolated Compose project and empty volume, never over
an existing database. Use the same trusted code and migrations as the backup;
build the application image from that checkout before starting the target. The
commands below use project name triagelens-recovery, which must not already
exist. Use the protected credentials file intended for the target and ensure its
optional queue password is set if the worker is needed. Stop and investigate any
failed command. Do not restore an archive from an untrusted source.
docker compose --env-file .env.compose -p triagelens-recovery up -d --build db migrate
if ($LASTEXITCODE -ne 0) { throw 'Target startup failed.' }
$migration = docker compose --env-file .env.compose -p triagelens-recovery ps -aq migrate
if ((docker wait $migration) -ne '0') { throw 'Target migration failed.' }
$target = docker compose --env-file .env.compose -p triagelens-recovery ps -q db
$revision = docker exec $target psql -X -U triagelens_admin -d triagelens -At -v ON_ERROR_STOP=1 -c 'SELECT version_num FROM triagelens.alembic_version'
if ($LASTEXITCODE -ne 0 -or $revision.Trim() -ne (Get-Content (Join-Path $backupDir 'revision.txt') -Raw).Trim()) { throw 'Revision mismatch; do not restore.' }
docker cp (Join-Path $backupDir 'triagelens.dump') "${target}:/tmp/triagelens-restore.dump"
if ($LASTEXITCODE -ne 0) { throw 'Copy failed.' }
docker exec $target pg_restore -U triagelens_admin -d triagelens --exit-on-error --single-transaction /tmp/triagelens-restore.dump
if ($LASTEXITCODE -ne 0) { throw 'Restore failed; investigate before using target.' }
docker exec $target rm /tmp/triagelens-restore.dumpIf restore fails, discard the isolated target and start again with an empty volume; sequence changes are not necessarily rolled back with the transaction. Before directing users or jobs to the recovered instance, verify representative records, independent labels, saved hunts, role permissions and application behavior. The migration recreates the version row and permissions; the archive deliberately excludes that row. The archive does not restore custom database settings, custom roles or changes outside the tracked migrations. In-flight jobs may be retried after recovery; check their state before enabling the worker. This procedure was tested with synthetic data on PostgreSQL 17, not as a production disaster drill.
SQLite remains the default. For a fresh UTF-8 PostgreSQL 17 database, install the
optional dependencies and run the baseline as a migration owner with permission
to create schemas and roles. Set TRIAGELENS_POSTGRES_MIGRATION_DSN in that
process before running Alembic.
python -m pip install --only-binary=:all: -r requirements-postgres.txt
python -m alembic upgrade headProvision separate non-owner logins for the four baseline roles and the optional worker role. Grant each login membership in just its corresponding group role. Supply distinct credentials through process-scoped environment variables.
| Group role | Connection variable | Caller |
|---|---|---|
triagelens_evaluator |
TRIAGELENS_POSTGRES_EVALUATOR_DSN |
CLI and evaluation coordinator |
triagelens_triage |
TRIAGELENS_POSTGRES_TRIAGE_DSN |
Restricted inference cache |
triagelens_analyst |
TRIAGELENS_POSTGRES_ANALYST_DSN |
API and job submission/status |
triagelens_hunt |
TRIAGELENS_POSTGRES_HUNT_DSN |
Read-only hunting queries |
triagelens_queue (optional) |
TRIAGELENS_POSTGRES_QUEUE_DSN |
Worker claims and job results |
Set DATABASE_BACKEND=postgresql for the API, CLI, and standard evaluation runner.
An explicit --db-path selects SQLite for that invocation. The dedicated live
protocol requires this explicit SQLite path. The PostgreSQL baseline starts empty
and leaves SQLite history in its original file.
The optional PostgreSQL worker runs an existing saved hunting filter and stores its returned row count, capped at the requested limit (1–1,000, default 500). Save the hunt in the dashboard first. Submission snapshots the filter. Use the same job ID, hunt name, and limit to retry a submission, including after the saved hunt is edited or removed. Use a fresh ID to run it again.
To enable the Compose worker, add a distinct TL_QUEUE_PASSWORD to the ignored
.env.compose file. Generate it once, then keep it with the other credentials.
if (-not (Select-String -LiteralPath .env.compose -Pattern '^TL_QUEUE_PASSWORD=' -Quiet)) {
$queueBytes = [byte[]]::new(32)
[System.Security.Cryptography.RandomNumberGenerator]::Fill($queueBytes)
Add-Content -LiteralPath .env.compose -Value "TL_QUEUE_PASSWORD=$([Convert]::ToHexString($queueBytes))"
}
docker compose --env-file .env.compose --profile jobs up --build -d
$jobId = [guid]::NewGuid().ToString()
docker compose --env-file .env.compose exec api python deploy/start.py queue enqueue 'My saved hunt' $jobId --limit 500
docker compose --env-file .env.compose exec api python deploy/start.py queue status $jobIdThe worker adds a 256 MiB limit to the Compose stack. To stop the stack with
the worker enabled, use docker compose --env-file .env.compose --profile jobs down.
For standalone PostgreSQL, apply python -m alembic upgrade head and grant
triagelens_queue to a dedicated login. Set the queue and hunt DSNs in the
worker process and run python -m src.worker run. In a separate process with
the analyst DSN, use python -m src.worker enqueue 'My saved hunt' <job-id>
and python -m src.worker status <job-id>. Use run --once for a single poll.
Claims use 30-second leases. Transient failures and expired leases can retry after 5 and 10 seconds, with at most three attempts. An expired attempt may repeat the read-only query against current data. The lease token prevents a stale attempt from publishing a result. Job rows and filter snapshots remain in PostgreSQL. Status shows a bounded count or a short failure code.
To run the offline regression checks, set TRIAGELENS_TEST_POSTGRES_DSN to a
disposable PostgreSQL instance on 127.0.0.1 using a login that can create
temporary test databases and roles. Then run:
python -m pytest -q -x --tb=short tests/test_database_contract.py tests/test_compose_startup.pyThe trusted evaluator preserves independent labels and raw records. The triage login reads verdicts only. The hunt login queries a read-only analyst view that includes alert labels, with a server-side statement timeout. The raw record has a JSONB GIN index for supported containment queries. Time partitioning is disabled. Within each CLI or evaluation run, the trusted coordinator passes a label-filtered timeline and a restricted cache login to the inference stage in the same process.
For verification, point TRIAGELENS_TEST_POSTGRES_DSN at an administrative login
on a disposable instance at 127.0.0.1. The tests create and remove a test
database and test logins. The baseline group roles remain on the test instance.
python -m pytest -q -x --tb=short tests/test_database_contract.py