-
Notifications
You must be signed in to change notification settings - Fork 0
How it is built
A map of the codebase for someone about to change it. It says where each piece lives, which piece is the product and which piece only exists to check the product, and what the architecture forbids you from doing. It does not teach STIX (the app does that at /guide) and it does not restate the identifier specification (that is docs/identifiers.md).
For running your own instance, see Self-hosting. For the branch and commit conventions, Contributing.
| Directory | What it is | Runs in production? |
|---|---|---|
frontend/src/ |
the application: React canvas, STIX core, IndexedDB store | yes, in the analyst's tab |
frontend/src/stix/ |
the STIX core that ships: ids, bundle builder, importer, matrix, JCS, OASIS validation | yes |
backend/app/stix_core/ |
the reference STIX implementation in Python, on stix2
|
never |
backend/scripts/ |
generators of committed files: the golden fixtures, and the three shipped datasets | never, dev and CI only |
enricher/ |
the optional passive-enrichment sidecar, a separate container | only if you start it |
backend/ is not a backend. It is a library and an oracle, and nothing in the
Docker image serves it. See Enrichment sidecar for the one
component that does talk to the network.
Six scripts are not oracle work at all, and four of them do the same job for
one knowledge base each: build_attack_dataset.py distills MITRE's ATT&CK STIX
into frontend/public/attack-dataset.json, merging Enterprise, Mobile and ICS
into one corpus and failing loudly if two of them ever put two names on one
number; build_f3_dataset.py, build_atlas_dataset.py and
build_aadapt_dataset.py do the fraud, the AI and the digital asset matrices.
Then build_actors_dataset.py distills the MISP threat-actor galaxy into
actors-dataset.json, and build_countries_dataset.py reads ISO 3166-1 out of
the iso-codes package into countries.json. All six produce committed assets
that frontend/src/ fetches once, on demand.
Adding a seventh, for a fifth framework, is a written-down sequence rather than
an exploration:
docs/adding-a-framework.md. So production data files are generated
from backend/, and following a new version of a corpus means re-running its
script and committing the output. The Datasets workflow does that every
Monday and opens a pull request when something moved.
These files are not decoration. They decide which spelling an analyst is offered, and our identifiers are computed from names, so they decide whether two people's objects merge on import. That is also why the actor corpus is arbitrated rather than merged: the galaxy and MITRE disagree on where an actor ends, MITRE folding UNC2452 into APT29 where the galaxy keeps it apart, so every galaxy actor whose name ATT&CK already resolves is dropped at build time. Shipping both spellings would manufacture the duplicate the whole project exists to avoid.
The server is nginx over frontend/dist/. It holds code, never data. That is
one decision, and most of the awkward parts of the codebase are consequences of
it rather than choices of their own.
The STIX logic has to exist in TypeScript. There is nowhere else to put it.
frontend/src/stix/bundle.ts is the builder a server would have delegated to
stix2, written out by hand. It carries checks the library would have imposed
for free, such as valid_until having to be strictly after valid_from, which
the Python twin does not need to spell out.
Validation runs before the download, in the tab. frontend/src/stix/validate.ts
checks every object against the 41 vendored OASIS schemas in
frontend/src/stix/schemas/. The validators cannot be compiled at runtime: the
production CSP forbids unsafe-eval and ajv compiles through new Function. So
frontend/scripts/build-validators.mjs precompiles them into
src/stix/generated/validators.mjs, which is generated at predev/prebuild/pretest
and is not committed. If you clone and open a source file expecting to find it,
that is why it is missing.
Storage is one browser profile on one machine. store.ts is the whole
persistence layer, and the export file is the only backup that leaves it. That
constraint is user-visible, and the README and
SECURITY.md
own the user-facing half of it.
The attack surface is the content the analyst opens, not a request handler:
bundles, backup files, pasted documents. importer.ts is deliberately tolerant
(unknown types counted, never fatal) and deliberately distrustful where it has
to be: a bundle is third-party JSON, and an as string cast on it is a lie at
runtime. The fields that end up stored and then rendered, name, value, the
label and alias lists, the refs arrays, go through asText, asArray and
asRefs instead. Plain casts survive on the rest, so the guarantee is per
field, not module-wide: if you start rendering a field that has one, convert it
first.
Two tabs on the same origin cannot see each other. sync.ts opens a
BroadcastChannel named dmas.changes and every read-write transaction
announces on it, from a single point inside tx() so that no mutation can
forget. Nothing merges: the other tab is told it is stale, which is enough, and
the product is single-user by nature.
There are two implementations of the same thing. This is not duplication left by accident.
| TypeScript | Python | |
|---|---|---|
| Files | frontend/src/stix/{ids,bundle,importer,relationships,jcs}.ts |
backend/app/stix_core/{ids,bundle,importer,relationships}.py |
| Role | the product | the oracle |
| Depends on | nothing but uuid and hand-written JCS |
stix2, and pycti in tests |
| Checked against | golden fixtures from the Python side |
pycti itself, in backend/tests/test_stix_ids.py
|
The Python side exists so that a claim about OpenCTI can be checked against
OpenCTI's own client rather than asserted. test_stix_ids.py recomputes every
identifier recipe with pycti and fails if they diverge, so the day OpenCTI
changes its algorithm the test breaks instead of the users' imports.
The TypeScript side is pinned to the Python side by two generated files, both committed, neither ever edited by hand:
| File | Generated by | What it pins |
|---|---|---|
frontend/src/stix/golden-vectors.json |
backend/scripts/generate_golden_vectors.py |
41 object-id cases and 12 observable-id cases, plus the namespace |
frontend/src/stix/golden-bundle.json |
backend/scripts/generate_golden_bundle.py |
one fixture investigation, exported under three option sets |
The bundle fixture is the interesting one. It carries a single frozen
investigation and the three exports the Python builder produced from it:
report/amber with an author, grouping/none without notes, and
report/amber with confidence: 75. bundle.test.ts requires the
TypeScript builder to reproduce, for each of them, the same objects, the same
fingerprint and the same warnings. The warnings are part of the contract
because they are how the analyst learns what was dropped.
The CI stix-core job regenerates both files and fails if git diff is not
empty, printing the diff. So the two implementations cannot drift quietly.
- Changed an id recipe? Change it in
ids.pyandids.ts, thenuv run python scripts/generate_golden_vectors.py. - Changed the builder? Same in both, then
generate_golden_bundle.py. - Never hand-edit
golden-*.json. Both files carry a_commentsaying so, and CI will overwrite you anyway.
An oracle only catches what a fixture exercises. Two divergences got through and
are documented in the code, both in backend/app/stix_core/bundle.py:
- a day-only date (
2026-03-14, what<input type=date>returns) was normalised by the TypeScript builder and raised insidestix2on the Python side. The fixture carried only full timestamps. - an analyst-entered
valid_fromon an indicator was honoured in TypeScript and ignored in Python. No vector covered the case, so the oracle could not flag it.
In both, and in the deterministic bundle id (uuid5 of the fingerprint, where
stix2 draws a uuid4), the product was right and the oracle caught up. That
is the expected direction: the browser builder is what ships. Adding a case to
the fixture is part of fixing a divergence, not an optional follow-up.
frontend/src/guide.ts holds no knowledge of its own. It turns
frontend/src/stix/relationships.ts around so it reads the other way: instead of
"is this pair allowed?", it answers "what can I do with a threat actor?".
Everything the page shows comes from something the application already uses:
| The page shows | It comes from |
|---|---|
| which relationships are legal |
allowedRelationships() in stix/relationships.ts
|
| what each verb means | relationHelp() |
| what to do when nothing is legal |
findBridges() in bridges.ts
|
| example detection patterns |
patternFromObservable() in pattern.ts
|
The reason is stated in the module header and it is worth repeating: prose copied out of the matrix falls out of step the first time a type is added, and help that lies costs more than no help at all.
Two details that are easy to break:
-
canLink()answers in the same order the canvas does: direct, then reversed, then bridge, then the genericrelated-to, then nothing. It mirrorsbeginRelationincomponents/Workspace.tsx. Reordering it would make the guide describe an application that does not exist. -
observableSourcesTowardSdo()exists so a test, not a sentence, checks the rule that surprises everybody (an observable is never the source of a relationship toward an object). If the matrix ever contradicts it, the test falls over instead of the page quietly lying.
related-to is filtered out of the explorer everywhere and shown once, alone,
as the last resort it is. Sample values come from the ranges reserved for
documentation (RFC 2606, RFC 5737) so they can never be mistaken for a real IOC.
To change what the guide says, change the matrix or relationHelp. Adding
prose to components/StixGuide.tsx is the one thing this design exists to
prevent.
/guide and /about are prose, so they are served as complete HTML. The canvas
stays a single-page app: a graph does not get indexed, and there is no audience
for a static rendering of it.
The mechanism is three Vite entry points (vite.config.ts: main, guide,
about) and one post-build step. npm run build is
tsc -b && vite build && node prerender.mjs. prerender.mjs loads
src/prerender.tsx through Vite's SSR API, renders each page, substitutes the
body into the empty <div id="root"></div> that Vite produced, writes
dist/<route>/index.html and removes the original dist/<route>.html so the
same page is not reachable at two addresses. nginx then serves it through
try_files $uri $uri/index.html /index.html with no special rule.
Separate entry points are the point: the prose pages load neither the canvas, nor storage, nor the PDF readers.
Two guardrails in prerender.mjs, both of which have a failure behind them:
- it throws if the built shell has no empty
#root, so a template change cannot silently produce a page with nowhere to put content; - it throws if a render comes out below a byte floor (5000 for the guide, 4000
for
/about), which catches a render that technically succeeded and produced almost nothing.
It writes no bundle or stylesheet name of its own, so a hash change cannot leave a page bare.
The two pages differ on purpose:
/guide |
/about |
|
|---|---|---|
| Rendered with | renderToString |
renderToStaticMarkup |
| Script | hydrated by guide-entry.tsx
|
none at all |
| Why | the dropdowns come alive; React needs its comment markers to graft onto the existing HTML | a page explaining that nothing leaves the browser has no business running code to say so |
The guide component is also mounted inside the SPA at #/guide. It is the same
component, not a cut-down one: mode changes only the shape of internal links,
the wording of the way out, and when the controls become live. Until the script
loads, the static page's selects are disabled, because a dead control casts
doubt on the rest of the page.
frontend/src/store.ts replaces the FastAPI routers it names in its header
(investigations.py, export.py, importer.py) and reproduces their rules.
Records keep the row shape of the old SQLite schema, properties as serialised
JSON, because that is the shape the bundle builder and the golden fixtures
consume. If a field looks like it went through a database, it did, historically.
The database is IndexedDB, name stixit (kept after the rebranding: renaming it
would orphan existing data), version 2, five object stores: investigations,
entities, relationships, notes, captures. Key path id; every store but
investigations carries an investigation_id index.
-
An error means nothing was written.
tx()aborts the transaction before propagating, so a throw after aputcannot commit half a change. -
Every write is announced exactly once.
announceChange()is called fromtx()when the mode isreadwrite, not from each mutation. -
A full profile is an actionable error, not a technical one. A
QuotaExceededErrorbecomesStoreError(507);isQuotaError()and the exportedQUOTA_EXCEEDEDconstant let the UI say what to do about it. -
A dead connection is never memoised.
onerror,oncloseandonversionchangeall clear the cached promise;onblockedrejects with a 409 telling the analyst to close the other tabs, because otherwise the open never settles and the home screen stays empty forever. -
Relationships are validated against the matrix on write, on creation and
on
updateRelationship. Self-links are refused (422). -
Observable names are refanged on write, so
hxxp://evil[.]comnever reaches the store. -
Deletion is a cascade, and the cascade is undoable.
deleteEntityreturns anEntitySnapshot(the entity, its relationships, its notes, the captures it was unlinked from).restoreEntityrewrites the rows with their original identifiers rather than going throughcreateEntity, which would mint new ones and leave the restored relationships pointing into the void. -
Some writes deliberately do not touch
updated_at:savePositions,pinNote,saveScratchpad, the capture operations, andmarkExported. Node positions and working notes are not intel, and marking an export as done must not make the investigation look modified at the very moment it was exported.updated_atis what the status bar compares against, so widening this list is a product decision, not a tidy-up.
exportBackup / importBackup handle the full local dump (format
dmas-backup, version 1), which carries what STIX cannot: the triage tray,
positions, pasted captures, scratchpad notes. Three rules worth knowing before
touching that code:
- restoring replaces whole investigations by id, and purges their existing content first, so an entity deleted since the backup does not come back as an orphan;
- rows whose
investigation_idis not among the file's investigations are dropped and counted inskippedRows, never written into an investigation the confirmation dialog did not name; -
dmas.enrich.endpointsis never restored from a file. It holds a sidecar URL and its token, and a backup is a file that may have come from someone else; restoring one used to repoint enrichment at their server without the confirmation ever mentioning settings. Refused settings are counted and reported.
| You want to change | Start at |
|---|---|
| which links are offered, or what a verb means |
frontend/src/stix/relationships.ts, frontend/src/relationHelp.ts
|
| what a bundle contains |
frontend/src/stix/bundle.ts, then the Python twin, then regenerate |
| how an identifier is computed |
ids.ts + ids.py, spec in docs/identifiers.md
|
| what a third-party bundle turns into | frontend/src/stix/importer.ts |
| what survives a reload | frontend/src/store.ts |
| what the guide page says | the matrix, never the component |
Neighbouring pages: Exporting, Importing into OpenCTI, Canvas reference, Triage walkthrough, Troubleshooting.
Using it
Running it
Changing it
When it breaks
In the application:
In the repository: