Form Sections and Conditional Logic Developer Guide

Form sections & conditional logic — Developer Guide

How section headings and conditional visibility work, why field identity had to be fixed before either could be built, and which decisions are load-bearing.

Shipped as #940 (sections), #941 (conditions), #942 (field identity) and #943 (the submissions access gate).

The user-facing page is Forms.

---

1. 📁 The files involved

Colour key: 🗄️ schema · ⚙️ engine · 🔌 API · 🖥️ UI · 🎨 CSS · 🌍 i18n · 🧪 tests · 📄 docs

🎨	File	What it does
🗄️	database/freeitsm.sql	form_fields.config (LONGTEXT NULL) + form_fields.is_deleted (TINYINT DEFAULT 0)
🗄️	includes/db_verify_schema.php	the same two columns, so an upgrade picks them up
⚙️	includes/form_logic.php	the source of truth. formLogicVisibility(), formLogicTestRule(), formLogicNormaliseValue()
⚙️	includes/services/forms.php	FIELD_TYPES (+section), ANSWERABLE_TYPES, CONDITION_OPS; validateFields(), validateFieldConfig(), syncFields() (now id-based), submitForm() visibility gate, createVersion() id remap
⚙️	assets/js/form-logic.js	the browser mirror — FormLogic.visibility/testRule/parseOptions/hasOptions
🔌	api/forms/get_form.php	returns config; excludes is_deleted = 1
🔌	api/forms/get_submissions.php	includes retired fields (with is_deleted), excludes sections; gained requireModuleAccessJson('forms') (#943)
🔌	api/forms/ai_generate.php	section added to the generator's allowed types
🔌	api/self-service/get_catalogue_form.php	returns config; excludes retired fields
🔌	api/v1/resources/forms.php	serialises config; excludes retired fields; docblock records the sync change
🖥️	forms/edit/index.php	the builder — _key identity, the Only show this when… editor, section rows, buildRulesForSave(), pruneInvalidConditions(), AI-apply id carry-over
🖥️	forms/fill.php	section rendering, readField() / collectValues() / applyVisibility(), visible-only validation
🖥️	forms/submissions.php	the removed marker on retired columns
🖥️	self-service/catalogue.php	the portal's renderer — sections, applyVisibility(), hidden answers excluded from submit
🖥️	self-service/includes/footer.php	$needsFormLogic loads assets/js/form-logic.js for the portal
🎨	assets/css/forms.css	.field-item-section, .field-conditions, .cond-row, .preview-section, .preview-cond, .col-retired
🌍	lang/en/forms.php + lang/pt-BR/forms.php	the cond block, fieldtypes.section, typename.section, field.section_ph, preview.conditional*, subs.retired* — same commit, 446 keys each
🧪	tests/forms-logic/run.php	56 assertions; runs one case table through both evaluators
📄	CHANGELOG.local.md, this wiki	#940–#943

There is no new table. A section is a form_fields row and a condition is JSON on that row, mirroring how is_portal_visible was added to forms.

---

2. 🔑 Field identity — why this had to come first

Before #942, syncFields() matched a form's fields by position:

// the OLD code
foreach ($fields as $i => $f) {
    if ($i < count($existingIds)) {
        $upd->execute([...$f..., $existingIds[$i]]);   // "the i-th existing row"
    }
}

form_submission_data.field_id points at a form_fields.id. Updating those rows in payload order means dragging a question rewrites the labels while the stored answers stay put. A form answered Alice Brown / 14 Elm Road / 52000 against Employee name / Home address / Starting salary reported, after moving Starting salary to the top, that the person's salary was "Alice Brown".

It was silent, and forms/submissions.php builds its columns from the current field list — so the table stayed perfectly well-formed and merely showed the wrong answers under each heading. Removing a middle field was worse: it shifted everything up and hard-deleted the trailing row's form_submission_data.

This also blocked conditions outright. A rule saying "show X when Y is Yes" has to name Y by something that survives a drag; "the third question" does not.

The fix: every payload row carries the id it is editing, a row with no id is genuinely new, and a field that disappears from the payload is soft-deleted rather than dropped.

$id = $f['id'] ?? null;
if ($id !== null && isset($ownIds[$id])) {
    $upd->execute([...$f..., $i, $id, $formId]);   // update THAT field
} else {
    $ins->execute([$formId, ...$f..., $i]);
    $id = (int)$conn->lastInsertId();
}

⚠️ $ownIds is the set of ids that actually belong to this form. A payload id outside it is treated as new rather than trusted — otherwise a crafted save could re-point another form's field, and with it another form's answers.

The three identity spaces

Field identity is expressed three different ways, and mixing them up is the easiest mistake to make here:

Where	Looks like	Why
Database + all read paths	form_fields.id	the only stable, persistent identity
Save payload	an integer id, or "idx:N"	a rule may point at a field being created by this same save, which has no id yet
Builder (forms/edit/index.php)	_key, a session counter	a new field has no id, and an index changes the moment anything is dragged

validateFields() accepts both payload forms; syncFields() resolves "idx:N" to a real id in a second pass, once pass 1 has minted every id. On the builder side rehydrateRuleRefs() converts ids → _key after a load, and buildRulesForSave() converts _key → id-or-idx:N on save.

---

3. ⚙️ Sections are rows, not a table

A section is a form_fields row with field_type = 'section'.

That looks impure — it is presentational, and it lives in a table of questions — but it buys two things that a separate form_sections table would have cost:

• One flat sort_order still describes the whole form. No "section 2, position 3" composite ordering, no ambiguity about where a sectionless field sits.
• The builder's existing drag-and-drop keeps working untouched. onFieldDrop() splices one array; cross-container HTML5 drag between section boxes would have been a rewrite of the most fiddly code in the module.

Membership is positional: a section owns every field after it until the next section. FormsService::ANSWERABLE_TYPES is FIELD_TYPES minus section, and that constant is what keeps a heading out of validation, submission and the workflow payload.

---

4. ⚙️ The condition format

form_fields.config is JSON, currently carrying one key:

{"visible_if": {
  "match": "all",
  "rules": [{"field": 94, "op": "equals", "value": "Yes"}]
}}

NULL config — which is every field that existed before this change — means always visible. That is why an upgraded form renders byte-for-byte as it did.

Operators: equals · not_equals · contains · is_empty · is_not_empty · greater_than · less_than. match is all or any.

The backwards-only rule

validateFieldConfig() rejects a rule whose target sits at the same index or later:

if ($refIndex >= $i) {
    throw new ServiceError('validation', 'invalid_field',
        "fields[{$i}]: a condition can only depend on an earlier question.");
}

🔑 This is the single most load-bearing decision in the feature. Because a rule can only look backwards, a cycle is impossible to construct — so neither evaluator ever needs cycle detection, and evaluating in sort_order always has the answers it depends on already computed. It is enforced on the server, so the REST API and a hand-crafted request are bound by it too, not just the builder.

The builder's pruneInvalidConditions() runs after every drag: a reorder is the only way to invert a rule, and dropping it (with a toast) is more honest than letting the save 422 about a question the user wasn't thinking about.

---

5. ⚙️ Two evaluators, one case table

The rule engine exists twice, deliberately:

	Decides	Why it exists
includes/form_logic.php	yes — on submit	a browser can simply not run our JS
assets/js/form-logic.js	no	so the form reacts as someone types

Two copies that disagree is the entire risk of this design, so tests/forms-logic/run.php runs one case table through both and compares. The JS half is executed in headless Chrome with the library inlined into the harness (a file:// page cannot reliably <script src> a sibling, and a silent load failure would look exactly like a pass). No Chrome → that section SKIPs loudly rather than quietly passing.

Both normalise an answer into {scalar, list} before testing, so one operator handles a text answer and a checkboxes answer (stored as a JSON array string) without branching per type.

Server-side re-derivation on submit

$visible = formLogicVisibility($fields, $normalised);

foreach (array_keys($normalised) as $fieldId) {
    if (empty($visible[$fieldId])) unset($normalised[$fieldId]);   // never asked → not stored
}
// ...then: skip required + format checks for anything not visible

🔑 Required means required if shown. Without this, hiding a required field client-side would still 422 on submit — and a crafted post could skip a genuinely required question by claiming it was hidden. Re-deriving from the submitted answers closes both.

---

6. 🔑 Soft delete, and the one view that looks backwards

is_deleted = 1 hides a field from the builder, forms/fill.php, the portal and the REST API — everywhere the form is asked.

api/forms/get_submissions.php is the deliberate exception: it returns retired fields (ordered is_deleted, sort_order, id) so forms/submissions.php can keep their column, marked removed. Sections are excluded there instead, since they never held an answer.

The rule: a question stops being asked; the answers it already collected do not stop being real.

---

7. ⚙️ Version forking has to remap

createVersion() copies fields row by row, not with INSERT..SELECT:

$idMap[(int)$f['id']] = (int)$conn->lastInsertId();
// ...second pass rewrites each rule's `field` through $idMap

A bulk copy would leave the new version's rules pointing at the frozen original's field ids — so editing the copy would change what the frozen snapshot showed. Retired fields are not carried forward: they exist to keep old answers readable on the version they belong to.

---

8. ⚠️ Why the three renderers were NOT merged

A form is drawn in three places, and they were left as three:

Renderer	CSS vocabulary
forms/edit/index.php (preview)	.preview-field
forms/fill.php	.form-field
self-service/catalogue.php	.cat-field

Collapsing them into one renderer was the original plan and was dropped on purpose: the three genuinely look different, so unifying the markup would have changed how existing forms render — a visible regression in exchange for an internal tidy-up.

Instead they share one brain (assets/js/form-logic.js: which types exist, which carry options, what is visible) and keep their own skins. Adding a field type still means touching three small markup templates, but only one type list and one visibility rule.

If you do unify them later, write the characterisation test first: snapshot the rendered HTML of a fixture form covering all nine types through all three renderers, then assert it is unchanged.

---

9. 🖥️ The AI-apply trap

applyGeneratedForm() rebuilds the whole fields array from the generator's response. With id-based sync that would read as "retire every question and create new ones" — no data loss, but the form silently detaches from every answer already given to it.

It therefore carries ids across by label, the same way the proposal diff decides what counts as unchanged, and re-points any carried-over condition through the old field's label to the new field's _key. Any rule whose trigger the AI removed is dropped.

---

10. 🧪 Verifying a change here

php tests/forms-logic/run.php

56 assertions. Every negative is paired with a positive control — "the hidden required field did not block submission" is equally true of a build that validates nothing, so the control proves the same question is enforced once shown.

Confirm the suite can fail before trusting a green run: break formLogicTestRule() (e.g. return true at the top) and it should report ~14 failures, including the JS-vs-PHP comparisons.

⚠️ The suite does not use the single-rolled-back-transaction pattern the other suites use — FormsService opens its own transactions and MySQL has no nested ones. It creates ZZ_TEST_-prefixed forms and removes them in a finally, driven by the title rather than by a tracked id list (a fatal mid-createVersion() can commit a row before its id is ever recorded, and the version-chain FK then blocks the whole cleanup).

For the builder itself, remember that grepping rendered HTML proves nothing about whether the JS parses. Extract each inline <script> block and feed it to new Function() in headless Chrome, with a deliberately-invalid block included as a control.

---

11. ⚠️ Upgrading

form_fields.config and form_fields.is_deleted are both in includes/db_verify_schema.php, so System → Database Verification adds them. Until it is run, saving a form fails on the missing columns — this feature has no soft-degrade path, unlike ticket snooze.

REST API clients need one behaviour change flagged: PATCH /forms/{id} now syncs by id. Send each field's id back as GET returns it and it is updated in place; a field with no id is new, and an omitted field is retired. A client that rebuilds the list from scratch replaces the questions rather than renaming them — which is deliberate, and the opposite of the old behaviour that quietly re-pointed historical answers.