Demo Data Developer Guide

Demo Data — Developer Guide

How the demo-data importer works, and how to add demo data for a new module. The user-facing page is Demo Data.

The design goal: a module's sample data is a single JSON file of plain rows, and one generic importer turns it into a correct, cross-referenced, repeatable database seed. Adding demo data should mean writing JSON — not code.

---

1. 📁 The files involved

Colour key: 📋 data · ⚙️ importer · 🖥️ UI · 🔌 diagnostic · 📄 docs

	File	What it does
📋	database/demo-data/<module>.json	the sample rows for one module (e.g. core.json, tickets.json)
⚙️	api/system/import_demo_data.php	the engine. Loads the JSON, resolves references + tokens, inserts in a transaction
🖥️	system/demo-data/index.php	the System → Demo Data page — one card/button per module
🔌	api/system/debug-tools/D001_demo_core_import.php	the Demo Core Data Import diagnostic (traces a failing core import)
📄	this wiki	you are here

There is no per-module import code — the importer is generic. Everything specific to a module lives in its JSON.

---

2. The shape of a demo file

A demo file is an object of tiers, each tier an object of tables, each table a list of rows:

{
  "tier1": {
    "departments": [
      { "_ref": "it", "name": "IT Support", "is_active": 1, "display_order": 1 }
    ]
  },
  "tier2": {
    "teams": [
      { "_ref": "servicedesk", "name": "Service Desk", "is_active": 1 }
    ]
  },
  "tier3": {
    "analyst_teams": [
      { "analyst_id": "@analysts.jsmith", "team_id": "@teams.servicedesk" }
    ]
  }
}

• The table key is the real database table name. Row fields are its columns. The importer builds the INSERT from whatever keys are present — so a row is just "the columns I want to set".
• Tiers are ordering buckets, processed tier1 → tier5. Their only job is dependency order: a row that references another must live in a later tier (or a later position in the same tier) than the row it points at. Name them for meaning if you like, but the importer only cares about order.

---

3. Meta fields (underscore-prefixed)

These are stripped before insert — they control the importer, they are not columns.

Field	Purpose
_ref	Name this row so others can reference it. Becomes @<table>.<ref> (see §4).
_skip_insert + _match_by + _match_value	Don't insert — match an existing row and map its id. Used for the admin account: { "_ref": "admin", "_skip_insert": true, "_match_by": "username", "_match_value": "admin" }. The row is looked up, its id recorded under @<table>.<ref>, and it is preserved when the table is cleared.

---

4. Cross-references — @table.ref

Any string value of the form @<table>.<ref> is replaced with the real primary-key id of the row that declared that _ref, at insert time:

{ "analyst_id": "@analysts.jsmith", "team_id": "@teams.servicedesk" }

Because ids are assigned as rows insert, the referenced row must already have been inserted — hence tiers. An unresolved reference throws and rolls back the whole import, so a typo fails loudly rather than writing a bad row.

🔑 Referencing rows in the same table you're currently inserting works too, as long as the target appears earlier in the list.

---

5. Tokens — computed values

Any string value matching one of these is expanded per row:

Token	Expands to
__BCRYPT:secret__	a bcrypt hash of secret (demo passwords — the analysts use __BCRYPT:demo1234__)
__RELATIVE_DATE:-60d__	a UTC datetime N days from now; optional hours: __RELATIVE_DATE:-60d+3h__
__RELATIVE_DATEONLY:-7d__	a UTC date (no time) N days from now
__NOW__	the current UTC datetime
__GENERATE__	a fresh demo ticket number (tickets only)
__UNIQUE__	replaced with a uniqid() (embed it in a larger string)

Relative dates are why a freshly imported ticket looks recent: stamp demo dates relatively, never with a literal date.

---

6. String → id translation for normalised tables

Some tables store a status/priority/etc. as an id into a lookup table, but the demo JSON is written with the human-readable name for legibility. The importer swaps them using a small table in import_demo_data.php:

$lookupTranslations = [
    'tickets' => [
        ['status',   'status_id',   'ticket_statuses'],
        ['priority', 'priority_id', 'ticket_priorities'],
    ],
    // changes, tasks, status_incidents, ...
];

So "status": "Open" in tickets.json becomes status_id = <id of 'Open'>. An unknown name throws. If your new module normalises a field into a lookup table, add a row here — otherwise write the real *_id (or an @ref) in the JSON.

A couple of bespoke cases live near it: morningChecks_Results resolves Status via Label (not name), and tickets drops legacy requester_email / requester_name. Follow that pattern only if a table genuinely needs it.

---

7. What the importer guarantees

• One transaction. Any error → full rollback, nothing changes. The response is { success, imported: {table: count}, total }.
• Repeatable. Before inserting, every table that has insertable rows is cleared (DELETE, FK checks off, reverse tier order). _skip_insert rows are preserved via a WHERE NOT (...) guard — that's how the admin account survives.
• Primary keys. Ids are mapped via getPrimaryKeyColumn(), which defaults to id. Override it there for a table whose PK isn't id (e.g. morningChecks_Checks → CheckID), or map to null for a pure join table with no id worth referencing.

---

8. ✅ Adding demo data for a new module

1. Write database/demo-data/<module>.json — tiers, tables, rows, using _ref / @refs / tokens above. Order tables so referenced rows insert first.
2. Allowlist it — add <module> to $allowedModules in api/system/import_demo_data.php (the import refuses any module not on this list).
3. Add a card for it in system/demo-data/index.php so it appears on the Demo Data page, with a one-line description of what it seeds.
4. Lookup translations (only if needed) — if the module stores a name-as-id, add its columns to $lookupTranslations (§6).
5. Primary key (only if needed) — if a table's PK isn't id, add it to getPrimaryKeyColumn() (§7).
6. Test on a fresh install (a throwaway Docker instance is ideal): import Core Data first, then your module. Confirm the counts, and that re-importing gives a clean copy, not duplicates.

🔑 Core Data first, always. Nearly every module references analysts, teams, departments or users by @ref, and those refs are only in scope once core.json has been imported in the same database. Document any cross-module dependency in the module's Demo Data card.

If a core import misbehaves, the D001 — Demo Core Data Import debug tool (System → Debug tools) runs the whole thing in-process and reports table-by-table what happened, including a write-probe inside a rolled-back transaction.

---

9. Worked example — the RBAC seed (#48)

core.json seeds a realistic role-based-access example purely as data — no importer changes were needed, which is the whole point of the generic design. It:

• sets demo analysts non-admin with can_access_all_modules = 0;
• gives the Escalation Team can_access_all_modules = 1 and an Escalation Team rbac_role holding every capability (rbac_role_capabilities, one row per key from capAll());
• scopes the Service Desk team to a 12-module subset via team_modules, with a blank Service Desk role;
• links roles to teams via rbac_team_roles.

All of it is @ref wiring across rbac_roles, rbac_role_capabilities, rbac_team_roles, team_modules, teams and analysts — a good template for any permissions-related demo. See the Demo Data user page for what it looks like in the UI.