Form Engine

Form engine

This page describes how openrunic customises data capture without dynamic schema changes: versioned immutable definitions, one JSON document per submission, and a typed projection of the fields somebody actually wants to query. It is for anyone working on forms, and for anyone tempted to add a generic attribute table.

The problem it solves

Every practice wants different intake questions, different encounter templates, and different referral forms. The obvious answers are both bad. Adding a column per customer means schema changes per tenant. Storing every answer as a row in a generic attribute-value table means every query is a self-join, every value is a string, and nothing is typed.

The failure mode being avoided is a specific and well-documented one in this category of software: a customization layer that is genuinely capable but hostile, configured through cryptic option strings with no preview, storing answers in a generic attribute table underneath. It is powerful enough that people build critical workflows on it and unpleasant enough that they resent it daily.

openrunic splits the problem in three.

flowchart LR
    A["FormDefinition<br/>versioned, immutable once published<br/>definition + compiled + promotionManifest"]
    B["FormSubmission<br/>one row, one JSONB document<br/>validated against the pinned version"]
    C["FormPromotedValue<br/>typed columns, one row per<br/>promoted field per repetition"]
    A -->|pinned by FK| B
    B -->|pure projection, same transaction| C

Loading

FormDefinition

Keyed by (tenantId, key, version). key is lowercase kebab-case; version is an integer. Status moves DRAFT to PUBLISHED to RETIRED.

Publishing freezes the row. A change is a new version, never an edit. That is not just convention: the Zod write contract in packages/database/src/schemas/form.ts has no update input at all. Only formDefinitionCreateInput and formDefinitionPublishInput exist. The builder screen mirrors this by computing the next version number when the current status is published.

The consequence is the one that matters clinically: an in-flight submission always keeps the definition it was authored against. A form that changes mid-visit does not retroactively invalidate what was already answered.

bindTo says which of the four consumers a form serves: PATIENT, ENCOUNTER, PORTAL, or REFERRAL. One engine, four bindings, rather than four half-built systems.

Three JSON columns carry the form itself:

Column	Contents
definition	The authored document. What the builder produced.
compiled	Publish-time artefacts: the validator descriptor, the render tree, the print layout, and the FHIR Questionnaire mapping.
promotionManifest	The list of fields to materialise into typed columns.

The database validates definition and compiled only for shape, that they are JSON objects of a sane size. Their contents are validated by the schema the form compiler generates at publish time from the definition itself. That is the whole point of the engine: field-level validation is data, not code.

FormSubmission

One row per submission. values is one JSONB document validated against the pinned definition's generated schema.

The schema comment states the rule directly: this is deliberately not entity-attribute-value. There is exactly one row per submission, never one row per answered field.

The foreign key to FormDefinition is onDelete: Restrict, which pins the version and prevents a definition being removed out from under recorded answers.

effectiveAt is the clinically effective instant, which is what promoted values are graphed against. It may differ from when the form was keyed in, and that difference is exactly what makes a retrospectively entered vital chart at the right point in time.

Two refinements are enforced by the write contract: a submission completed by a user must name that user, and a completed submission must record when it was completed.

A submission can be the origin of an Observation and of a ConsentGrant, both of which carry a formSubmissionId back to it. Portal consent forms are form submissions; there is no parallel consent capture path.

FormPromotedValue: the promoted columns

Promotion is what makes a form field searchable, graphable, and reportable without giving up typing.

The table has one typed column per value kind: valueText, valueNumber, valueDate, valueBoolean, valueCode with valueCodeSystem, and valueQuantity with valueUnit. It also denormalises definitionKey and definitionVersion from the definition, explicitly so a query needs no join to filter, and copies effectiveAt from the submission.

The rule set is stated identically in the schema and in packages/database/src/forms.ts, and all six parts matter:

1. A field is promoted only if the published definition's manifest lists it. Authors opt in per field. Nothing is implicit.
2. Promotion runs on write, in the same transaction as the submission upsert. Rows are replaced wholesale on every save, so a re-save can never leave a stale value behind. The projection is pure, so it can also be rebuilt from scratch.
3. Exactly one row per (submission, fieldKey, repeatIndex). Repeating groups index from zero.
4. Exactly one typed value column is populated, chosen by the declared type. A quantity fills value and unit. A coded answer fills code and system.
5. Null, undefined, and empty string produce no row at all. That keeps "unanswered" and "answered blank" distinguishable, and keeps the table sparse.
6. effectiveAt is copied from the submission, so graphing a promoted field over time is one index scan rather than a join tree.

promoteSubmission(manifest, submission, options) implements this. It is pure: no clock, no database access. Rows come back in manifest order and then repetition order, so createMany inserts and snapshot tests are both stable.

A present-but-wrong-shape value throws FormPromotionError rather than being skipped. Quietly dropping a value would put a hole in a clinical flowsheet, and a hole nobody was told about is worse than a failed save.

The manifest itself is strictly typed, unlike the definition. Field keys must match /^[a-zA-Z][a-zA-Z0-9_]*$/, must not repeat, and a promoted quantity field must declare a default unit. There is a cap of 200 promoted fields per manifest.

Why a fixed table rather than dynamic DDL

Per-form dynamic DDL would fight the expand-and-contract migration discipline that everything else in the schema follows. A form published on Tuesday would create columns on Tuesday, outside the reviewed migration history, in an order that varies by deployment. FormPromotedValue gives typed values, real indexes, and a stable schema at the cost of one extra table.

The condition model, as it stands

Conditional visibility is stored inside the definition document, which the database treats as opaque. There is no typed condition schema in packages/database or packages/types today.

The only concrete shape in the repository is the builder's client-side field type in apps/web/src/app/admin/forms/FormsScreen.tsx, where condition is a single nullable string per field, edited as text and rendered as a caption. That is a placeholder, not the final representation, and it is worth knowing before designing against it. A structured predicate belongs in @openrunic/forms-engine, which is early work on the feat/emr-packages branch.

The builder screen

/admin/forms in the staff app is a three-pane builder at wide widths: field catalogue, canvas, and properties. Preview is always one toggle away rather than a separate mode. Published versions are immutable and the screen says so, and publishing states its consequence in a confirmation dialog rather than happening silently.

The builder screen is on the feat/emr-app branch and has not merged to dev.

Related pages

• Data model
• Web app
• FHIR boundary