A shared runtime for user code: transforms today, analyses next

[angela-helios]

Two parts of the platform run code that users write: the integration engine runs transforms (Python today), and the analytics application runs analyses and compute steps (Python today, with SQL and possibly JavaScript mentioned as later). Both need the same thing: a place to run a program that is not ours, on data that is sensitive, without letting it take down the host or reach anything it should not. This discussion is for deciding whether that place is one shared service, and what it is built on.

What the engine needs from it

The engine's contract with a transform is small and already defined: one request per record ({"id", "record", "context"}), one reply ({"id", "resources"} or {"id", "error": {"kind", "message"}}), JSON per line. Today the engine starts one long-lived Python process per feed and talks to it over stdin and stdout. The contract does not say where the process runs, so the runtime can change underneath it. What the engine needs from any runtime:

• Isolation. A transform that loops, leaks or crashes affects only itself. A transform cannot read another feed's data, the engine's store, or the network, except what it is explicitly given (a terminology lookup, for example).
• Latency per record in the low milliseconds once warm, because a feed can carry a million rows. A cold start per record is not acceptable; a cold start per feed or per batch is.
• Versioned code. A load records the transform version it ran with; the runtime must run a named version, not "whatever is on disk".
• A dry run. The same runtime executes "Try it" on five rows without any side effect, so the UI's test is the real thing.
• Dependencies. Transforms use the standard library today; some will want a FHIR model library or a terminology client. The runtime decides what is installed and how.

The options

Option	Isolation	Warm latency	Cold start	Operations	Notes
Subprocess per feed (today)	Process only: no filesystem or network limits	~1 ms	~100 ms	None	Fine for trusted transforms on a single host; not a sandbox
Container per transform (Docker / Podman, long-lived)	Namespaces, cgroups, no network by default	~1 ms over a socket	1 to 3 s	Container runtime on the host	The usual answer for self-hosted; images carry dependencies
MicroVM per execution (Firecracker; written in Rust, runs Lambda and Fly.io)	Hardware virtualization	~1 ms once booted	~125 ms boot plus Python start	KVM required; bare metal or nested virtualization	Strongest isolation that still starts fast; a snapshot per transform version makes cold starts near zero
AWS Lambda (or Azure Functions, Cloud Run)	Managed microVMs	~1 ms warm	100 ms to seconds	None, but AWS only	Natural for the managed variant; per-record invocation is too chatty, so batch per call
In-process interpreter (an embedded Python, or a WebAssembly runtime such as Wasmtime with a Python build)	Wasm: a real sandbox with explicit capabilities; embedded CPython: none	~0.1 ms	~10 ms	None	Wasm is the lightest sandbox; the Python-on-Wasm toolchain is maturing and dependency support is limited

Two things are the same whichever is chosen. First, the contract (JSON per record, batched) and the registry (named, versioned transforms) belong to the shared service, not to each caller. Second, the service should offer the two deployment variants the rest of the platform offers: run it yourself (containers, or Firecracker where KVM is available) and run it managed (Lambda), behind one API.

A proposal to react to

1. A runtime service with one HTTP API: POST /run with a transform version and a batch of inputs, returning one output per input; POST /try for dry runs; the registry of versions behind it. The engine's Transformer trait gets an implementation that calls it, next to the subprocess one.
2. Containers first for self-hosted (one long-lived container per transform version, no network, read-only filesystem, CPU and memory limits), because every target host can run them today.
3. Firecracker as the hardening step for the same service where KVM exists, with a snapshot per transform version, measured before it is adopted.
4. Lambda as the managed variant, batch per invocation.
5. Languages beyond Python (SQL, JavaScript) are new executors behind the same API, not new services.

Questions

• Does the analytics side's analysis step fit the same contract (a program, a versioned name, a batch in, a batch out), or does it need something the engine does not (long-running jobs, access to the FHIR server during execution)?
• Which hosts will customers run on? Firecracker needs KVM, which rules out most managed Kubernetes nodes and all nested-virtualization clouds.
• Who owns the service? It sits between the two products and belongs to neither.

Related: the engine's transform contract and Transformer trait are in #1 (decision 3 of docs/decisions.md).

[angela-helios]

Summary of a conversation with Andres on where this lands, so it is on record here.

What the three modes need

Mode	Shape	What the runtime must support
Transforms (HIE)	records in, resources or a data error out, milliseconds per record	A function call. Fits as is; the product's "Try it" is HIE's POST /api/transforms/{name}/try.
Hook (one patient, quick answer)	request/response	A function call. Fits as is.
Batch (analyses)	jobs of minutes that read HFS and terminology while they run and write results to an object store	Long-running jobs, controlled access to HFS/HTS from inside the user's code.
Workbook	cells that share state	A session that outlives one call.

So the service is more than "run this function": it needs jobs, sessions, and a way to hand user code a scoped credential for HFS.

Where it runs

• Kubernetes as the base, including managed clusters: customers install the product in their own cloud, and all three shapes map directly (hook → Deployment, batch → Job, workbook → a pod with its own lifetime). Adding another language later (Rust, JavaScript, SQL) is one more image; the service does not change.
• Lambda is out: single cloud, no state between calls, a 15-minute ceiling that batch would brush against.
• Isolation is chosen per pod with RuntimeClass, which is what keeps it out of the code: gVisor by default (no hardware requirement, available on any cluster, GKE ships it as Sandbox), Firecracker (through Kata) only where we control the hosts, since it needs KVM and the ordinary nodes of a managed cluster do not expose it.

Ownership

A third service, owned by neither HIE nor the product, that both consume.

Two things HIE needs from it

1. The contract, not the runtime. HIE talks to its worker over JSON lines today (records in, resources or a DataError out, per record). That protocol should be the same whether the worker is a local subprocess or a remote pod, so HIE only changes transport.
2. HIE must not depend on Kubernetes for the simple case. A lab with a folder and a CSV will not stand up a cluster. The local subprocess stays the default (and what the public sandbox runs); the remote service is the production and multi-tenant option.

The part that still needs design is the credential: how the service gives user code a token scoped to one tenant to read HFS and HTS during a batch job.

[angela-helios]

The runtime now has a home: https://github.com/HeliosSoftware/python-runtime. This is the contract HIE honours today with its local Python worker, written down so the service can offer the same thing and HIE only changes transport.

What a transform is

A Python module that defines transform(record, context):

• record is one source row as JSON: a CSV row as {column: value}, an NDJSON line, a FHIR resource, a parsed HL7 v2 message, a database row with the database's types.
• context is {"feed": "<source id>", "file": "<load name>", "line": <row number>}; more keys may be added, never removed.
• It returns a FHIR resource (dict), a list of resources, or None (nothing to deliver).
• It raises hie.DataError("why") when the data is wrong: HIE rejects that one record with the message and moves on. Any other exception is a bug in the transform: HIE rejects the record with the exception and traceback so the person who owns the code sees it.

The helpers a transform may import from hie must exist in the runtime exactly as in HIE's hie_runtime/hie.py: DataError, and for HL7 v2 hl7(record, "PID-5.1"), hl7_segments, hl7_field, hl7_datetime. That file is the reference; copying it is fine, diverging is not.

The wire today: JSON lines over stdin/stdout

hie_runtime/worker.py is started with the transform's path, prints {"ready": true}, then answers one request per line:

{"id": 1, "record": {...}, "context": {"feed": "lab-results", "file": "labs.csv", "line": 2}}

{"id": 1, "resources": [{"resourceType": "Observation", ...}]}
{"id": 1, "error": {"kind": "data", "message": "value 'alto' is not a number"}}
{"id": 1, "error": {"kind": "transform", "message": "KeyError: 'unit'", "trace": "..."}}

id is echoed so replies can be matched. Nothing else travels: no credentials, no network from the transform.

What the service would expose

The same thing over HTTP, batched, is enough for HIE:

• PUT /transforms/{name} with {"code": "..."} — HIE pushes the code when it is saved (PUT /api/transforms/{name} on HIE's side already restarts the local worker; it would call this instead).
• POST /transforms/{name}/run with {"requests": [{"id", "record", "context"}, ...]} answering {"replies": [...]} in the shapes above, one reply per request, in order. HIE sends batches of up to batch_size records (100 by default).
• POST /transforms/try with {"code": "...", "requests": [...]} for the client application's "Try it" and HIE's /api/transforms/{name}/try, without saving.

Errors of the service (down, timeout, a 5xx) are transient for HIE: the records wait and are retried, and the outage alert names the runtime. Only a reply with error rejects a record.

On HIE's side this becomes one setting: runtime: local (the subprocess, the default and what the public sandbox runs) or runtime: <url>. The subprocess stays, so a single machine with a folder and a CSV never needs the service.

Not HIE's concern

How the code is isolated (gVisor, Firecracker, Kubernetes), how long a batch job may run, sessions for a workbook, and the credential user code gets to read HFS/HTS: the service decides, as discussed above. HIE's transforms are short, stateless, and never talk to HFS themselves.