Enrichment sidecar

Enrichment sidecar

The sidecar is the only part of Draw Me A STIX that talks to the outside world. It is a small FastAPI service that runs dig, whois and subfinder, queries crt.sh and CIRCL, and hands back candidates for the triage tray. It is optional, and the application is designed to be useful without it.

The README covers the one-line quick start. This page covers what it does not: why the split exists, the exact API contract, what each enricher will and will not tell you, and how to write your own.

Why it is a separate service

The application is a static site. It is served by nginx, it holds no server-side state, and during an investigation it makes no outbound call at all. That property is what makes it safe to open a TLP:RED case in it.

Enrichment breaks that posture on purpose: it resolves names, it queries public sources, and it needs a credential to do so. Putting that inside the application would mean every deployment inherits an outbound network path and a token, whether or not anyone wanted enrichment.

So it lives in its own container, with its own profile in docker-compose.yml, and it does not start unless you ask for it. That also means you can run it somewhere else entirely: on a jump host with an egress path, on a machine whose IP is expected to appear in a registrar's logs, or on a different network segment than the analyst's browser. The two halves only ever meet over HTTP.

With no endpoint, no call is possible

This is worth being precise about, because it is the property people ask about first.

Endpoints live in the browser's localStorage, under dmas.enrich.endpoints. Every call the application makes to a sidecar is built from an endpoint record: base URL, path, bearer token. With the list empty there is no URL to build a request from, so frontend/src/enrich.ts has no code path that reaches the network. It is not a flag that disables a feature, it is an absent address.

The interface follows the same rule rather than offering something that cannot work: the Enrich button in the inspector only appears when at least one endpoint is configured, and it only appears on observables (SCOs) and on vulnerability. That gate is wider than what the shipped enrichers accept, which is domain-name, ipv4-addr, ipv6-addr and vulnerability: on an email-addr or a file the button is there, and the dialog then tells you no enricher applies to that type.

You can verify all of this from the browser's network tab, which is the point.

Running it

With Compose, the two lines in the README are the whole of it. The sidecar listens on port 8001, and the enrich profile means a plain docker compose up starts the application alone.

Without Docker, from enricher/:

uv sync
uv run uvicorn app.main:app --host 127.0.0.1 --port 8001

The image is built from python:3.13-slim and carries only what the enrichers shell out to: dnsutils (for dig), whois, and subfinder pinned to v2.6.6. It runs as a non-root user (uid 10001), because those binaries are handed input the analyst typed. A HEALTHCHECK polls /health every 30 seconds.

Then point the application at it: the magnifier icon in the top bar, or "Enrichment endpoints…" in the command palette. Add a name, the URL, the token, and use Test, which calls GET /enrichers and prints back the identifiers it found. A failure there is far easier to read than a failure in the middle of an enrichment.

In production, HTTPS

If the application is served over HTTPS, the sidecar must be too. A browser blocks mixed content, and the error it reports is not helpful. Put the sidecar behind a reverse proxy with a real hostname and a certificate, and list that application's origin in ENRICHER_ORIGINS.

Add a request body size limit at that proxy as well. FastAPI reads and decodes the JSON body before it runs the auth dependency, so a large body is buffered and parsed without a token ever being checked. The field length caps in the sidecar bound the object after parsing; only the proxy can bound the bytes received. See Self-hosting for the proxy side.

Configuration

Everything is read from the environment at startup. Compose sets two of them from .env, as STIXIT_ENRICHER_TOKEN=${ENRICHER_TOKEN:-} and STIXIT_ENRICHER_ORIGINS=${ENRICHER_ORIGINS:-http://localhost:8000}: the short names are Compose substitutions, and the sidecar itself only ever reads the STIXIT_ENRICHER_ ones. To use any of the others, add it to the environment: block of the enricher service in docker-compose.yml.

Variable	Default	What it controls
STIXIT_ENRICHER_TOKEN	generated at startup	The bearer token required on /enrichers and /enrich.
STIXIT_ENRICHER_ORIGINS	http://localhost:5173,http://localhost:4173,http://localhost:8000	Comma-separated CORS allowlist.
STIXIT_ENRICHER_CACHE_TTL	300	Default lifetime, in seconds, of a cached answer.
STIXIT_ENRICHER_CRTSH_CACHE_TTL	3600	Lifetime for crtsh answers.
STIXIT_ENRICHER_CVE_CACHE_TTL	86400	Lifetime for cve answers.
STIXIT_ENRICHER_CACHE_MAX	1000	Cap on cache entries, which bounds the RAM.
STIXIT_ENRICHER_TOOL_TIMEOUT	15	Hard timeout, in seconds, on one binary call.
STIXIT_ENRICHER_SUBFINDER_MAX	200	Cap on the subdomains subfinder returns.
STIXIT_ENRICHER_CRTSH_MAX	200	Cap on the subdomains crtsh returns.
STIXIT_ENRICHER_HTTP_TIMEOUT	45	Ceiling on ONE HTTP attempt, for an enricher that sets no value of its own.
STIXIT_ENRICHER_HTTP_BUDGET	90	Total wait for such a source, retries included.
STIXIT_ENRICHER_HTTP_ATTEMPTS	4	Maximum attempts within that budget.
STIXIT_ENRICHER_DOCS	unset	Set to 1 to expose /docs, /redoc and /openapi.json.

The defaults in that table are the sidecar's own, the ones you get when the variable is absent. Compose narrows one of them: it passes ${ENRICHER_ORIGINS:-http://localhost:8000}, so a Compose deployment with no .env allows that single origin rather than the three.

Each variable is commented in enricher/app/config.py with the reasoning behind its value. Three points that catch people out:

The token is generated if you leave it empty, and printed as a warning in the logs. The shipped Compose file passes ${ENRICHER_TOKEN:-}, so an empty .env produces a present-but-empty variable, which is treated as absent. The service then works, behind a random token nobody has seen. Look for the startup warning, or set the variable.

Two HTTP durations, not one, and only crtsh obeys them. The per-attempt timeout has to stay longer than a successful answer takes, or you cut off a request that was about to land. The budget bounds the analyst's total wait. crt.sh regularly needs the whole of the first and part of the second, and it is the one enricher that takes these three variables as they come. cve passes its own values in code (15 second timeout, 25 second budget, three attempts) because CIRCL answers fast when it answers at all, so raising the budget in the environment changes nothing for a CVE lookup.

The docs routes are off by default and that is deliberate. FastAPI mounts them outside the application router, so they cannot be placed behind the auth dependency: on a public deployment they answered 200 without a token and handed out the request schema of /enrich. /docs and /redoc also load Swagger and fonts from third-party CDNs, which is third-party script executing on the sidecar's origin. Turn them on locally when you need the spec.

The API contract

Three routes. Two of them require the token.

Route	Auth	Purpose
GET /health	none	{"status": "ok"}, for a container healthcheck or a probe.
GET /enrichers	bearer	The catalogue: what this sidecar can do.
POST /enrich	bearer	Run one enricher on one selector.

Authentication is Authorization: Bearer <token>, compared in constant time. Anything else is a 401 with missing or invalid token.

CORS is strict and closed by default: only the origins in STIXIT_ENRICHER_ORIGINS are allowed, only GET and POST, and only the Authorization and Content-Type headers. A browser will refuse the answer for any other origin, which is the failure people most often mistake for a broken token.

GET /enrichers

Returns a list. accepts holds the STIX types this enricher takes as input; the application uses it to decide which enrichers to offer for the selected node.

[
  {
    "id": "dig",
    "label": "DNS (dig)",
    "description": "DNS resolution: A/AAAA, MX, NS, and reverse PTR on an IP.",
    "accepts": ["domain-name", "ipv4-addr", "ipv6-addr"]
  }
]

POST /enrich

The request is three fields, one selector at a time. Nothing about the graph, the notes or the rest of the investigation is ever sent.

{ "enricher": "dig", "type": "domain-name", "value": "nest.corax.example" }

enricher and type are capped at 64 characters, value at 512 (an FQDN caps at 253, an IPv6 at 45, a CVE at about 20). The shape of the value is then validated against the type before any process is spawned: a domain must match an ASCII pattern, an IP must parse and must carry no RFC 4007 scope id, a CVE must match CVE- then four digits, a dash, and four digits or more. That last check is case-insensitive, and the selector is upper-cased before it goes any further.

The response:

{
  "enricher": "dig",
  "candidates": [
    { "ref": "c0", "stix_type": "ipv4-addr", "name": "198.51.100.7", "properties": {} }
  ],
  "relations": [
    { "source_ref": "source", "rel_type": "resolves-to", "target_ref": "c0", "description": "A" }
  ],
  "notes": [
    { "target_ref": "source", "content": "whois - Registrar: Example Registrar" }
  ]
}

Three things carry the whole design:

• ref is local to this response. c0, c1, and so on. The sidecar knows nothing about the canvas, has no idea what identifiers exist there, and never invents one.
• source is the reserved reference for the enriched node. It is how a relation says "this points back at what you asked about". A candidate that declares ref: "source" is skipped by the application, so a third-party enricher cannot overwrite the enriched node.
• A note is for what does not deserve an entity. A registrar, a BGP prefix, a CVSS score. It attaches to source or to a candidate's ref, and becomes a STIX note object on export if you leave the notes option on in the export dialog.

Error codes

Code	Meaning
401	Missing or invalid bearer token.
404	Unknown enricher id.
422	The enricher does not accept that type, or the body failed validation.
502	The tool failed. The body says only enrichment unavailable.
503	The remote source did not answer or is rate limiting. The body carries a message written for the analyst.

The distinction between 502 and 503 matters in practice. A 502 means something is wrong on the sidecar side, and its detail (the tool's stderr) stays in the server logs so no internal path or version leaks to the browser. A 503 means nothing is broken, the source is saturated, and the useful thing to tell the analyst is when to come back. The application displays that message verbatim.

The 422 handler is custom: it returns the type, the location and the message of each error, but never the value that was refused. FastAPI's default handler echoes it back, and that value is a selector, so investigation data, ending up copied into an HTTP response on a path that is reachable before authentication.

Caching

Answers are cached in memory, keyed on (enricher, type, lowercased value). Nothing is written to disk and a restart empties it, which is what keeps the "no persistent log of selectors" promise honest.

An answer with neither candidates nor notes falls back to the short default TTL rather than the enricher's override, so an empty result can be replayed soon. The check is on candidates or notes: the CVE enricher only ever returns notes, and it is the one that needs the long cache most.

What the shipped enrichers do

id	Label	Accepts	Gives back
dig	DNS (dig)	domain-name, ipv4-addr, ipv6-addr	A/AAAA, MX, NS as candidates with resolves-to relations; reverse PTR on an IP.
subfinder	Subdomains (subfinder)	domain-name	Subdomain candidates, passively enumerated. No relation.
crtsh	Certificates (crt.sh)	domain-name	Subdomain candidates from Certificate Transparency, plus a certificate summary note.
whois	Whois	domain-name, ipv4-addr, ipv6-addr	A note: registrar, organisation, creation, update and expiry dates.
asnmap	ASN (Team Cymru)	ipv4-addr, ipv6-addr	An autonomous-system candidate with a belongs-to relation, plus notes for the registry and BGP prefixes.
cve	CVE	vulnerability	A note: description, CVSS, publication date.

Some detail worth having before you rely on them.

dig queries A, AAAA, MX and NS for a domain, and PTR for an IP. STIX 2.1 has one relationship for all DNS records, resolves-to, so the record type is carried in the relation's description field. On an IP the direction reverses: the PTR domain resolves to the enriched IP, not the other way round. Output is validated as well as input, because the content of a PTR, MX or NS record is written by whoever controls the zone being queried, and it lands in an exported STIX bundle.

subfinder runs with -silent and the default source set. It drops the apex itself, deduplicates, and caps at 200 names. It returns no relations: STIX 2.1 has no honest SRO between a domain and its subdomain, and inventing a semantically false link would poison the bundle. The subdomain arrives in the tray on its own, and you enrich it in turn if you want an IP.

crtsh asks crt.sh for unexpired certificates carrying a name under the domain. Names under the domain become candidates (capped at 200); names that are not under it, which happens with a shared CDN or hosting certificate, go into the note instead, up to ten listed and then a count. The note also carries the certificate count, up to three issuers, and the span of not_before dates, which places the infrastructure in time. Same discipline as subfinder: no relation.

This source is slow, and that is its normal regime rather than an incident: it alternates between answering in around 40 seconds, instant 502s, and silence. It also returns a 404 under load for domains that do have certificates, so 404 is treated as retryable here. Announcing "no certificate" on a hiccup would be a silent false negative, which is worse than an unavailability message.

whois parses raw whois output coarsely, by key aliases, tolerating the two big families (ICANN-style Creation Date: and AFNIC/RIPE-style created:). Values that are redacted, protected or n/a are dropped rather than reported. It creates no entity: a registrar is an attribute of the domain, not an object of the investigation graph.

asnmap keeps the name but not the tool. ProjectDiscovery's asnmap binary now routes through their cloud API and needs a key, which the sidecar's "binaries, not APIs" rule rules out. It queries Team Cymru's IP-to-ASN whois service instead, which needs no key. An IP that is not announced comes back as NA and produces nothing.

cve queries CIRCL, no API key. It parses both the cvelistv5 format that Vulnerability-Lookup serves today and the older cve-search format, so an older self-hosted instance still works. CIRCL counts its quota per IP address (20 per minute anonymously), shared by everything leaving your sidecar's address. Their 429 is cached by their own Varnish for about half an hour on that exact URL, so retrying the same CVE immediately achieves nothing while a different CVE will most likely go through. The sidecar says exactly that when it happens, so nobody concludes the tool is broken and keeps hammering.

What it refuses to do

Discovery, never reputation. No enricher returns a verdict, a score, or a "malicious/benign" label. That is not a gap waiting to be filled, it is the boundary: Draw Me A STIX is the drafting step before a platform, and reputation belongs where the knowledge base lives. An enricher that answered "this domain is malicious" would be putting a third party's judgement into a bundle that is supposed to carry only what the analyst asserted.

Nothing active. Every source is passive: DNS lookups, whois, CT logs, public APIs. No port scan, no probe of the target.

No API keys. Every shipped source works without one. This is why asnmap uses Team Cymru and why the CVE source is CIRCL. It keeps the sidecar something you can start and use, rather than an accounts checklist.

No shell, ever. Binaries are invoked with an argument array, and the selector is validated before the process is spawned. Related: an IPv6 scope id is rejected outright, even a legitimate one, because Python accepts almost anything after the % and GNU whois writes its query followed by a CRLF, which turns a newline in a scope into an extra line of protocol.

No selector in the logs. The sidecar logs technical reasons, counts and enricher names. It never writes the value it was asked about, including in error paths, which is why the tools' stderr is not logged either: dig and whois repeat the queried name in it.

How results reach the canvas

They do not. They reach the triage tray, like every other automated source in the tool.

For each candidate, the application:

1. Skips it if it claims the reserved source ref.
2. Deduplicates it against everything that already exists in the investigation, canvas and tray together. If a match is found, no twin is created: the relation is redirected to the node already there. Two domains resolving to the same IP converging on one node is exactly the useful information.
3. Otherwise creates the entity with status candidate and a source of enrich:<enricher id>, so you can see where it came from in the tray.

Relations are created immediately, but a relation to a candidate has no visible edge until you accept that candidate. A relation between two already-confirmed entities gets its edge right away. A relation the relationship matrix refuses (a 422) is expected and silently dropped, since the sidecar does not know the matrix.

Notes attach to whichever entity their target_ref names, most often the enriched node, and appear in the inspector.

The dialog reports all three counts, including "already known object(s), linked to the existing ones". A fully deduplicated enrichment did produce something, and calling that "no results" would be a lie.

Triage-walkthrough covers what to do with the tray once results are in it.

The browser gives up after 150 seconds. That is deliberately wider than the sidecar's own budget on slow sources: it is there so the interface is not blocked forever when the connection itself hangs, not to bound the enrichment.

Where the token is kept, and what that is worth

An endpoint is a URL and a credential, and the application treats those two differently. The URL, label and identifier go to localStorage under dmas.enrich.endpoints: a setting, local, forgettable, and it should survive closing the tab. The token goes to sessionStorage under dmas.enrich.tokens, unless the per-endpoint "Keep the token on this machine" checkbox moves it into localStorage too. The security policy sets out what that separation is worth, and what it is not.

The consequence you will actually meet is in the settings dialog: a row whose session token has expired shows its token field again, rather than letting you discover the problem as an unexplained 401 on your first enrichment.

Endpoints, being local settings, are per browser profile. Moving to another machine means adding them again.

Writing your own enricher

Two ways in, depending on whether you want to extend this sidecar or replace it.

A module in this sidecar

An enricher is a module in enricher/app/enrichers/ that exports two things: an INFO of type EnricherInfo, and an enrich(stix_type, value) -> EnrichResponse. Then register it in app/registry.py, in the _REGISTRY dict. That is the whole extension mechanism.

from app.schemas import SOURCE_REF, Candidate, EnricherInfo, EnrichResponse, Relation
from app.tools import run_tool, validate_selector

INFO = EnricherInfo(
    id="example",
    label="Example",
    description="One sentence, shown as-is in the enrichment dialog.",
    accepts=["domain-name"],
)


def enrich(stix_type: str, value: str) -> EnrichResponse:
    selector = validate_selector(stix_type, value)
    output = run_tool(["some-binary", selector])
    candidates = [Candidate(ref="c0", stix_type="ipv4-addr", name=parse(output))]
    relations = [
        Relation(source_ref=SOURCE_REF, rel_type="resolves-to", target_ref="c0")
    ]
    return EnrichResponse(enricher=INFO.id, candidates=candidates, relations=relations)

The rules that are not optional:

• Call validate_selector first, and use its return value, not the raw input. It checks the shape against the type and gives back the canonical form, which is also what keeps the cache honest.
• Use run_tool for binaries: it passes an argument array with no shell, applies the timeout, and turns a missing binary or a non-zero exit into a ToolError. Add the binary to enricher/Dockerfile, pinned.
• Use http_get_json for remote sources: it carries the retry, budget and backoff logic, and raises SourceUnavailable with a message written for an analyst.
• Pick your exception deliberately. SourceUnavailable becomes a 503 and its message travels to the browser. ToolError becomes a generic 502 and its detail stays in the logs. Never put a host path, a version, or the selector into a SourceUnavailable message.
• Validate what comes back, not only what goes in. is_domain and is_ip exist for this: a PTR record, a SAN in a certificate or a passive DNS entry is written by a third party and ends up in an exported bundle.
• Cap your results. A chatty domain has thousands of subdomains, and an unbounded tray is unusable.
• Do not invent relationships. If STIX 2.1 has no honest SRO for the link you found, return the candidate alone and let the analyst decide. See Canvas-reference for what the relationship matrix allows.
• Never log the selector. Counts and reasons only.

Tests live in enricher/tests/, one file per enricher, and uv run pytest from enricher/ runs them.

A service of your own

The application only speaks the contract above, so anything that answers GET /enrichers and POST /enrich with a bearer token and the right CORS headers is a valid endpoint. You do not have to use this codebase, or Python. Point the application at your own service, in your own network, with your own sources, including commercial ones with keys the shipped sidecar deliberately avoids.

Three things to get right: the CORS allowlist must contain the application's origin, the deployment must be HTTPS if the application is, and ref values must be unique within a response with source left reserved.

When it does not work

The usual failures, in the order they occur:

Symptom	Most likely cause
No Enrich button on a node	No endpoint configured, or the node is neither an observable nor a vulnerability.
token rejected (401)	Wrong token, or a session token that died with the previous tab.
endpoint unreachable	Mixed content (HTTPS app calling an HTTP endpoint), a CORS origin that is not on the list, or a wrong URL. The browser gives no detail for any of the three.
The dialog lists no enricher for a valid type	The catalogue call failed silently. Use Test in the endpoint settings, which shows the real error.
did not answer after N attempt(s)	The remote source is saturated. crt.sh does this routinely. Try again in a few minutes.
A rate-limit message on a CVE	CIRCL's quota, counted per IP and cached for about 30 minutes on that exact request. A different CVE will work.

Troubleshooting goes further on the ones that are not specific to enrichment.