Skip to content

Scripts Reference

Jump to bottom
zach115th edited this page Jul 4, 2026 · 6 revisions

Scripts Reference

All scripts live in scripts/ and are mounted read-only into the app and worker containers at /iriswebapp/scripts/. No docker cp needed.

Run scripts via:

docker exec iriswebapp_app python /iriswebapp/scripts/<script.py> [args]

iris_misp_sync_dev.py

MISP integration development and maintenance helper.

configure                  Configure the IrisMISPSync module via API (reads from .env)
smoke-test                 Create a case + IOC, verify MISP sync, clean up
cleanup-smoke              Remove objects created by a previous smoke-test run
seed-known-good            Seed a reference case with known-good MISP-synced IOCs
backfill-type-taxonomy     Backfill IocType.type_taxonomy from the bundled MISP catalog
test-ai-fallback           Test the AI type resolver for unmatched IOC types
resync-ioc --ioc-id <id>   Re-sync one IOC to MISP inline (bypasses celery)

resync-ioc is the recovery path for IOCs that the on_postload_ioc_create hook silently dropped (e.g. during MISP outages or pre-fork-safety-fix concurrency crashes).

validate_misp_sync.py

Validate the MISP sync state: checks that every case with a misp_event_link row has a corresponding MISP event, and every linked IOC has a corresponding MISP attribute.

sim_alert_template_pick.py

Regression harness for the AI case-template suggester.

--list                    List available test scenarios
--scenario <name>         Run a single scenario
--all                     Run all scenarios (exit 2 on any non-MATCH)

Useful as a pre-commit / CI gate when editing case_template_suggester.md. All 13 template-match scenarios should pass at ≥ 0.92 confidence.

seed_2025_test_case.py

Seed a realistic 2025 incident case for testing AI surfaces and correlation. Creates cases, IOCs, assets, timeline events, notes, and evidence.

download_misp_tag_bundles.py

Download the latest MISP taxonomies and galaxies from GitHub and bundle them into source/app/resources/. Refresh when new MISP taxonomy versions are released.

backfill_ioc_note_links.py

Retroactively create ioc_note_link rows by substring-matching IOC values against note bodies. Refangs [.]/[:]/[/]/(.)/hxxp:// before matching.

Useful for cases created before the IOC ↔ Note provenance feature shipped.

backfill_evidence_drive_links.py

Retroactively link evidence rows to drives via barcode matching. Run with --apply to commit changes (dry-run by default).

backfill_hayabusa_evtx_basename.py

Rewrite the EVTX filename line in pre-existing Hayabusa working-timeline events to show basename only (full path still in event_raw.evtx_file). Dry-run by default; pass --apply to commit.

reconcile_vanilla_schema.py

Fix schema gaps that can occur when importing a vanilla DFIR-IRIS database. Vanilla's upstream alembic/env.py left ADD COLUMN migrations partially uncommitted while advancing alembic_version — this script reconciles both phases.

import_vanilla_db.sh

Full database migration from vanilla DFIR-IRIS to iris-ng. Two commands: one on the old host to export, one on the new host to import.

Step 1 — on the source (vanilla DFIR-IRIS) host:

# Run from the root of the vanilla iris-web checkout.
# --out is optional; omit it for an auto-named directory.
bash scripts/import_vanilla_db.sh export --out ./iris-export

This creates a directory containing up to five files:

iris-export/
├── iris.dump              ← Postgres dump (required)
├── server_data.tar.gz     ← uploaded evidence files
├── user_templates.tar.gz  ← uploaded .docx report templates
├── iris-downloads.tar.gz  ← generated reports
└── secrets.env            ← IRIS_SECRET_KEY + IRIS_SECURITY_PASSWORD_SALT

Copy the entire directory to the new iris-ng host.

Step 2 — on the destination (iris-ng) host:

# Run from the root of the iris-ng checkout.
# --from is REQUIRED and must be the path to the exported directory.
bash scripts/import_vanilla_db.sh import --from /path/to/iris-export

Example with a timestamped bundle name:

bash scripts/import_vanilla_db.sh import --from /root/iris-ng/iris-export-20260629-184838

The script asks for confirmation, then: stops app/worker/ai_worker, drops+restores iris_db, sanity-checks the schema, restores named volumes, carries secrets into .env, reconciles any columns vanilla's broken alembic/env.py left uncommitted, brings the app back up (Alembic adds iris-ng tables on startup), and runs a post-Alembic table-presence check.

Flags:

Flag Mode Description
--out <dir> export write bundle to this path (default: auto-named)
--project <name> export compose project name on the source host (default: cwd basename)
--db-container <name> export override DB container name (default: iriswebapp_db)
--from <dir> import required — path to the exported bundle directory
--skip-volumes import do not restore evidence/template volumes
--skip-secrets import do not carry over IRIS_SECRET_KEY / IRIS_SECURITY_PASSWORD_SALT
-f / --force both skip confirmation prompts

Supports vanilla DFIR-IRIS v2.4.x and v2.5.0-beta.1.

IRIS_NG_NEW_TABLES in the script lists all iris-ng-specific tables; the post-import sanity check verifies they exist. When adding a new iris-ng table, add it to that array.

Common post-migration issue — login fails with valid credentials: IRIS_SECURITY_PASSWORD_SALT was not applied to the destination .env (or the wrong value was applied). Diagnose with diff on the two secrets files; apply manually with Python re.sub; restart with --force-recreate (not docker restart). See Getting Started → Troubleshooting for the full procedure.

migrate_postgres_17.sh

One-time upgrade of the iris-ng database from PostgreSQL 12 to PostgreSQL 17. Run in two phases — dump while pg12 is still running, then restore after the image swap.

Phase 1 — with the pg12 stack running:

bash scripts/migrate_postgres_17.sh dump

This writes a timestamped iris-pg12-backup-YYYYMMDD-HHMMSS.sql to the working directory and prints the exact next steps.

Between phases — follow the printed instructions:

  1. Edit docker/db/Dockerfile: FROM postgres:12-alpine → FROM postgres:17-alpine
  2. docker compose -f docker-compose.dev.yml down
  3. docker volume rm <project>_db_data (run docker volume ls | grep db_data if unsure of the name)
  4. docker compose -f docker-compose.dev.yml up -d --build --no-deps db
  5. Wait ~10s for pg17 to initialize.

Phase 2 — restore into pg17:

bash scripts/migrate_postgres_17.sh restore

Auto-finds the most recent iris-pg12-backup-*.sql in the working directory, or pass the filename explicitly: restore <file>.

After restore, bring up the rest of the stack:

docker compose -f docker-compose.dev.yml up -d --force-recreate

--force-recreate is required — without it, worker and ai_worker may stay on the old image with stale import caches.

scram-sha-256 password re-issue (handled automatically). pg17 defaults to scram-sha-256 auth for remote connections. A pg12 dump stores role passwords as MD5 hashes, which are incompatible with scram-sha-256 auth. The restore phase automatically re-issues the postgres and raptor passwords in plaintext (read from .env) so pg17 stores them as scram-sha-256. If you restore manually without this step, the app container cannot connect to the database.

Why pg_dumpall, not pg_upgrade? pg_dumpall produces version-independent SQL — a single dump jumps directly from pg12 to pg17 with no intermediate steps and no need for both pg binaries to be present simultaneously.

Private / internal helpers (prefixed _)

These are not meant to be run directly — they're imported by other scripts or used for one-off debugging:

Script Purpose
_check_chat_csrf.py Verify CSRF token flow for the chat endpoint
_test_hayabusa_parser.py Unit-test the Hayabusa parser against a sample file
_verify_ai_panel_template.py Verify the AI case-summary panel template renders

patreon.com/zach115th
buymeacoffee.com/zach115th

Clone this wiki locally