Skip to content

How it is built

pak edited this page Aug 30, 2026 · 3 revisions

How it is built

A map of the codebase for someone about to change it. It says where each piece lives, which piece is the product and which piece only exists to check the product, and what the architecture forbids you from doing. It does not teach STIX (the app does that at /guide) and it does not restate the identifier specification (that is docs/identifiers.md).

For running your own instance, see Self-hosting. For the branch and commit conventions, Contributing.

The five places code lives

Directory What it is Runs in production?
frontend/src/ the application: React canvas, STIX core, IndexedDB store yes, in the analyst's tab
frontend/src/stix/ the STIX core that ships: ids, bundle builder, importer, matrix, JCS, OASIS validation yes
backend/app/stix_core/ the reference STIX implementation in Python, on stix2 never
backend/scripts/ generators of committed files: the golden fixtures, and the three shipped datasets never, dev and CI only
enricher/ the optional passive-enrichment sidecar, a separate container only if you start it

backend/ is not a backend. It is a library and an oracle, and nothing in the Docker image serves it. See Enrichment sidecar for the one component that does talk to the network.

Six scripts are not oracle work at all, and four of them do the same job for one knowledge base each: build_attack_dataset.py distills MITRE's ATT&CK STIX into frontend/public/attack-dataset.json, merging Enterprise, Mobile and ICS into one corpus and failing loudly if two of them ever put two names on one number; build_f3_dataset.py, build_atlas_dataset.py and build_aadapt_dataset.py do the fraud, the AI and the digital asset matrices. Then build_actors_dataset.py distills the MISP threat-actor galaxy into actors-dataset.json, and build_countries_dataset.py reads ISO 3166-1 out of the iso-codes package into countries.json. All six produce committed assets that frontend/src/ fetches once, on demand.

Adding a seventh, for a fifth framework, is a written-down sequence rather than an exploration: docs/adding-a-framework.md. So production data files are generated from backend/, and following a new version of a corpus means re-running its script and committing the output. The Datasets workflow does that every Monday and opens a pull request when something moved.

These files are not decoration. They decide which spelling an analyst is offered, and our identifiers are computed from names, so they decide whether two people's objects merge on import. That is also why the actor corpus is arbitrated rather than merged: the galaxy and MITRE disagree on where an actor ends, MITRE folding UNC2452 into APT29 where the galaxy keeps it apart, so every galaxy actor whose name ATT&CK already resolves is dropped at build time. Shipping both spellings would manufacture the duplicate the whole project exists to avoid.

No backend, and what that forces

The server is nginx over frontend/dist/. It holds code, never data. That is one decision, and most of the awkward parts of the codebase are consequences of it rather than choices of their own.

The STIX logic has to exist in TypeScript. There is nowhere else to put it. frontend/src/stix/bundle.ts is the builder a server would have delegated to stix2, written out by hand. It carries checks the library would have imposed for free, such as valid_until having to be strictly after valid_from, which the Python twin does not need to spell out.

Validation runs before the download, in the tab. frontend/src/stix/validate.ts checks every object against the 41 vendored OASIS schemas in frontend/src/stix/schemas/. The validators cannot be compiled at runtime: the production CSP forbids unsafe-eval and ajv compiles through new Function. So frontend/scripts/build-validators.mjs precompiles them into src/stix/generated/validators.mjs, which is generated at predev/prebuild/pretest and is not committed. If you clone and open a source file expecting to find it, that is why it is missing.

Storage is one browser profile on one machine. store.ts is the whole persistence layer, and the export file is the only backup that leaves it. That constraint is user-visible, and the README and SECURITY.md own the user-facing half of it.

The attack surface is the content the analyst opens, not a request handler: bundles, backup files, pasted documents. importer.ts is deliberately tolerant (unknown types counted, never fatal) and deliberately distrustful where it has to be: a bundle is third-party JSON, and an as string cast on it is a lie at runtime. The fields that end up stored and then rendered, name, value, the label and alias lists, the refs arrays, go through asText, asArray and asRefs instead. Plain casts survive on the rest, so the guarantee is per field, not module-wide: if you start rendering a field that has one, convert it first.

Two tabs on the same origin cannot see each other. sync.ts opens a BroadcastChannel named dmas.changes and every read-write transaction announces on it, from a single point inside tx() so that no mutation can forget. Nothing merges: the other tab is told it is stale, which is enough, and the product is single-user by nature.

Two STIX builders, and the vectors between them

There are two implementations of the same thing. This is not duplication left by accident.

TypeScript Python
Files frontend/src/stix/{ids,bundle,importer,relationships,jcs}.ts backend/app/stix_core/{ids,bundle,importer,relationships}.py
Role the product the oracle
Depends on nothing but uuid and hand-written JCS stix2, and pycti in tests
Checked against golden fixtures from the Python side pycti itself, in backend/tests/test_stix_ids.py

The Python side exists so that a claim about OpenCTI can be checked against OpenCTI's own client rather than asserted. test_stix_ids.py recomputes every identifier recipe with pycti and fails if they diverge, so the day OpenCTI changes its algorithm the test breaks instead of the users' imports.

The TypeScript side is pinned to the Python side by two generated files, both committed, neither ever edited by hand:

File Generated by What it pins
frontend/src/stix/golden-vectors.json backend/scripts/generate_golden_vectors.py 41 object-id cases and 12 observable-id cases, plus the namespace
frontend/src/stix/golden-bundle.json backend/scripts/generate_golden_bundle.py one fixture investigation, exported under three option sets

The bundle fixture is the interesting one. It carries a single frozen investigation and the three exports the Python builder produced from it: report/amber with an author, grouping/none without notes, and report/amber with confidence: 75. bundle.test.ts requires the TypeScript builder to reproduce, for each of them, the same objects, the same fingerprint and the same warnings. The warnings are part of the contract because they are how the analyst learns what was dropped.

The CI stix-core job regenerates both files and fails if git diff is not empty, printing the diff. So the two implementations cannot drift quietly.

What that means when you change something

  • Changed an id recipe? Change it in ids.py and ids.ts, then uv run python scripts/generate_golden_vectors.py.
  • Changed the builder? Same in both, then generate_golden_bundle.py.
  • Never hand-edit golden-*.json. Both files carry a _comment saying so, and CI will overwrite you anyway.

Where the oracle does not help

An oracle only catches what a fixture exercises. Two divergences got through and are documented in the code, both in backend/app/stix_core/bundle.py:

  • a day-only date (2026-03-14, what <input type=date> returns) was normalised by the TypeScript builder and raised inside stix2 on the Python side. The fixture carried only full timestamps.
  • an analyst-entered valid_from on an indicator was honoured in TypeScript and ignored in Python. No vector covered the case, so the oracle could not flag it.

In both, and in the deterministic bundle id (uuid5 of the fingerprint, where stix2 draws a uuid4), the product was right and the oracle caught up. That is the expected direction: the browser builder is what ships. Adding a case to the fixture is part of fixing a divergence, not an optional follow-up.

The guide is derived, not written

frontend/src/guide.ts holds no knowledge of its own. It turns frontend/src/stix/relationships.ts around so it reads the other way: instead of "is this pair allowed?", it answers "what can I do with a threat actor?".

Everything the page shows comes from something the application already uses:

The page shows It comes from
which relationships are legal allowedRelationships() in stix/relationships.ts
what each verb means relationHelp()
what to do when nothing is legal findBridges() in bridges.ts
example detection patterns patternFromObservable() in pattern.ts

The reason is stated in the module header and it is worth repeating: prose copied out of the matrix falls out of step the first time a type is added, and help that lies costs more than no help at all.

Two details that are easy to break:

  • canLink() answers in the same order the canvas does: direct, then reversed, then bridge, then the generic related-to, then nothing. It mirrors beginRelation in components/Workspace.tsx. Reordering it would make the guide describe an application that does not exist.
  • observableSourcesTowardSdo() exists so a test, not a sentence, checks the rule that surprises everybody (an observable is never the source of a relationship toward an object). If the matrix ever contradicts it, the test falls over instead of the page quietly lying.

related-to is filtered out of the explorer everywhere and shown once, alone, as the last resort it is. Sample values come from the ranges reserved for documentation (RFC 2606, RFC 5737) so they can never be mistaken for a real IOC.

To change what the guide says, change the matrix or relationHelp. Adding prose to components/StixGuide.tsx is the one thing this design exists to prevent.

Two pre-rendered prose pages

/guide and /about are prose, so they are served as complete HTML. The canvas stays a single-page app: a graph does not get indexed, and there is no audience for a static rendering of it.

The mechanism is three Vite entry points (vite.config.ts: main, guide, about) and one post-build step. npm run build is tsc -b && vite build && node prerender.mjs. prerender.mjs loads src/prerender.tsx through Vite's SSR API, renders each page, substitutes the body into the empty <div id="root"></div> that Vite produced, writes dist/<route>/index.html and removes the original dist/<route>.html so the same page is not reachable at two addresses. nginx then serves it through try_files $uri $uri/index.html /index.html with no special rule.

Separate entry points are the point: the prose pages load neither the canvas, nor storage, nor the PDF readers.

Two guardrails in prerender.mjs, both of which have a failure behind them:

  • it throws if the built shell has no empty #root, so a template change cannot silently produce a page with nowhere to put content;
  • it throws if a render comes out below a byte floor (5000 for the guide, 4000 for /about), which catches a render that technically succeeded and produced almost nothing.

It writes no bundle or stylesheet name of its own, so a hash change cannot leave a page bare.

The two pages differ on purpose:

/guide /about
Rendered with renderToString renderToStaticMarkup
Script hydrated by guide-entry.tsx none at all
Why the dropdowns come alive; React needs its comment markers to graft onto the existing HTML a page explaining that nothing leaves the browser has no business running code to say so

The guide component is also mounted inside the SPA at #/guide. It is the same component, not a cut-down one: mode changes only the shape of internal links, the wording of the way out, and when the controls become live. Until the script loads, the static page's selects are disabled, because a dead control casts doubt on the rest of the page.

The store

frontend/src/store.ts replaces the FastAPI routers it names in its header (investigations.py, export.py, importer.py) and reproduces their rules. Records keep the row shape of the old SQLite schema, properties as serialised JSON, because that is the shape the bundle builder and the golden fixtures consume. If a field looks like it went through a database, it did, historically.

The database is IndexedDB, name stixit (kept after the rebranding: renaming it would orphan existing data), version 2, five object stores: investigations, entities, relationships, notes, captures. Key path id; every store but investigations carries an investigation_id index.

What the store guarantees

  • An error means nothing was written. tx() aborts the transaction before propagating, so a throw after a put cannot commit half a change.
  • Every write is announced exactly once. announceChange() is called from tx() when the mode is readwrite, not from each mutation.
  • A full profile is an actionable error, not a technical one. A QuotaExceededError becomes StoreError(507); isQuotaError() and the exported QUOTA_EXCEEDED constant let the UI say what to do about it.
  • A dead connection is never memoised. onerror, onclose and onversionchange all clear the cached promise; onblocked rejects with a 409 telling the analyst to close the other tabs, because otherwise the open never settles and the home screen stays empty forever.
  • Relationships are validated against the matrix on write, on creation and on updateRelationship. Self-links are refused (422).
  • Observable names are refanged on write, so hxxp://evil[.]com never reaches the store.
  • Deletion is a cascade, and the cascade is undoable. deleteEntity returns an EntitySnapshot (the entity, its relationships, its notes, the captures it was unlinked from). restoreEntity rewrites the rows with their original identifiers rather than going through createEntity, which would mint new ones and leave the restored relationships pointing into the void.
  • Some writes deliberately do not touch updated_at: savePositions, pinNote, saveScratchpad, the capture operations, and markExported. Node positions and working notes are not intel, and marking an export as done must not make the investigation look modified at the very moment it was exported. updated_at is what the status bar compares against, so widening this list is a product decision, not a tidy-up.

Backups

exportBackup / importBackup handle the full local dump (format dmas-backup, version 1), which carries what STIX cannot: the triage tray, positions, pasted captures, scratchpad notes. Three rules worth knowing before touching that code:

  • restoring replaces whole investigations by id, and purges their existing content first, so an entity deleted since the backup does not come back as an orphan;
  • rows whose investigation_id is not among the file's investigations are dropped and counted in skippedRows, never written into an investigation the confirmation dialog did not name;
  • dmas.enrich.endpoints is never restored from a file. It holds a sidecar URL and its token, and a backup is a file that may have come from someone else; restoring one used to repoint enrichment at their server without the confirmation ever mentioning settings. Refused settings are counted and reported.

Where to go from here

You want to change Start at
which links are offered, or what a verb means frontend/src/stix/relationships.ts, frontend/src/relationHelp.ts
what a bundle contains frontend/src/stix/bundle.ts, then the Python twin, then regenerate
how an identifier is computed ids.ts + ids.py, spec in docs/identifiers.md
what a third-party bundle turns into frontend/src/stix/importer.ts
what survives a reload frontend/src/store.ts
what the guide page says the matrix, never the component

Neighbouring pages: Exporting, Importing into OpenCTI, Canvas reference, Triage walkthrough, Troubleshooting.

Clone this wiki locally