feat(rest): treatAsHistorical import option — skip the state machine for historical-data migration (#3479)

.changeset/import-historical-fsm.md

@@ -0,0 +1,32 @@
+---
+"@objectstack/spec": patch
+"@objectstack/objectql": patch
+"@objectstack/rest": patch
+"@objectstack/platform-objects": patch
+---
+
+feat(rest): `treatAsHistorical` import option — skip the state machine for historical-data migration (#3479)
+
+Sibling of #3433 (seed exemption), one entry point over. #3165's `initialStates` enforced
+the FSM entry point on every INSERT, so importing established historical facts —
+a batch of already-`closed` tickets, `closed_won` deals, `completed` projects —
+was rejected row-by-row with `invalid_initial_state`, blocking the core
+data-migration path. Unlike the seed case it was visible (per-row errors), but it
+still functionally blocked a legitimate use.
+
+- **spec**: `ExecutionContext.skipStateMachine` — a general, server-set flag (the
+  seed-specific `seedReplay`'s sibling) that skips the `state_machine` rule for a
+  write; `ImportRequestSchema.treatAsHistorical` (default `false`) — the user-facing
+  import option.
+- **objectql**: the engine now skips the state machine for `seedReplay` OR
+  `skipStateMachine` (one helper), covering both seed replay and historical import.
+- **rest**: the import runner sets `skipStateMachine` on the write context iff the
+  request opts into `treatAsHistorical`; default off, so a normal import still walks
+  the FSM (the strict behavior is the default). Import **undo** now also carries
+  `skipStateMachine`, since restoring a prior snapshot re-writes an earlier state
+  that need not be a legal transition from where the row is now.
+- **platform-objects**: `sys_import_job.treat_as_historical` audit column (additive).
+
+Scope is identical to the seed exemption: ONLY the `state_machine` rule is skipped;
+field shape, `format`, `cross_field`, `script` all still run. The objectui import
+wizard checkbox is a separate follow-up.

content/docs/protocol/objectql/state-machine.mdx

@@ -100,6 +100,7 @@ transitions: {
 - The check is **lenient where it cannot reason**: if the prior state is not described by the table (e.g. legacy or externally-written data), it does not block.
 - Only a rule with `severity: 'error'` (the default) blocks the write; `warning`/`info` are logged.
 - **Seed writes are exempt** (#3433). Curated seed data — package bootstrap fixtures, marketplace templates, per-org replay, all loaded by `SeedLoaderService` — is a snapshot of established facts, not a record walking its lifecycle, so it bypasses the `state_machine` rule entirely: a seed may be born mid-lifecycle (a `completed` project, a `closed_won` opportunity) and neither `initialStates` (insert) nor `transitions` (update) is enforced. Every *other* validation still runs, so a seed must still satisfy field shape, `format`, `script`, and the rest. `os lint` warns when a seeded value is not a state the machine declares, so a typo is still caught before boot.
+- **A "historical" data import is exempt too** (#3479). Migrating established facts — a batch of already-`closed` tickets, `closed_won` deals — is the same "snapshot, not a lifecycle event" situation. Set `treatAsHistorical: true` on the import request (default **off**) and the runner puts `skipStateMachine` on the write context, so `initialStates` doesn't reject those mid-lifecycle rows. A normal import leaves it off and still walks the FSM — the strict behavior is the default, so the exemption is always an explicit opt-in.
 
 ### Conditional transitions
 

content/docs/references/api/export.mdx

@@ -82,6 +82,7 @@ const result = CreateExportJobRequest.parse(data);
 | **writeMode** | `Enum<'insert' \| 'update' \| 'upsert'>` | ✅ | insert / update / upsert semantics |
 | **matchFields** | `string[]` | optional | Fields that identify an existing record (required for update/upsert) |
 | **runAutomations** | `boolean` | ✅ | Fire triggers/hooks for each imported row (off by default for bulk) |
+| **treatAsHistorical** | `boolean` | ✅ | Import as established historical facts: skip the state_machine rule so mid-lifecycle rows (e.g. already-closed tickets, closed_won deals) are not rejected by initialStates (#3479). Off by default so a normal import still walks the FSM. |
 | **trimWhitespace** | `boolean` | ✅ | Trim leading/trailing whitespace from string cells |
 | **nullValues** | `string[]` | optional | Strings treated as null/blank (besides empty string) |
 | **createMissingOptions** | `boolean` | ✅ | Keep unmatched select values instead of failing the row |
@@ -370,6 +371,7 @@ Type: `{ sourceField: string; targetField: string; targetLabel?: string; transfo
 | **writeMode** | `Enum<'insert' \| 'update' \| 'upsert'>` | ✅ | insert / update / upsert semantics |
 | **matchFields** | `string[]` | optional | Fields that identify an existing record (required for update/upsert) |
 | **runAutomations** | `boolean` | ✅ | Fire triggers/hooks for each imported row (off by default for bulk) |
+| **treatAsHistorical** | `boolean` | ✅ | Import as established historical facts: skip the state_machine rule so mid-lifecycle rows (e.g. already-closed tickets, closed_won deals) are not rejected by initialStates (#3479). Off by default so a normal import still walks the FSM. |
 | **trimWhitespace** | `boolean` | ✅ | Trim leading/trailing whitespace from string cells |
 | **nullValues** | `string[]` | optional | Strings treated as null/blank (besides empty string) |
 | **createMissingOptions** | `boolean` | ✅ | Keep unmatched select values instead of failing the row |

content/docs/references/kernel/execution-context.mdx

@@ -68,6 +68,7 @@ const result = ExecutionContext.parse(data);
 | **skipTriggers** | `boolean` | optional |  |
 | **skipAutomations** | `boolean` | optional |  |
 | **seedReplay** | `boolean` | optional |  |
+| **skipStateMachine** | `boolean` | optional |  |
 | **oauthScopes** | `string[]` | optional |  |
 | **accessToken** | `string` | optional |  |
 | **transaction** | `any` | optional |  |

packages/objectql/src/engine.test.ts

@@ -907,6 +907,14 @@ describe('ObjectQL Engine', () => {
             expect(mockDriver.create).toHaveBeenCalledTimes(1);
         });
 
+        it('admits the same INSERT when the context carries skipStateMachine (#3479 historical import)', async () => {
+            // The general flag — set by the REST import runner for a "historical"
+            // import — takes the same engine path as seedReplay.
+            vi.mocked(SchemaRegistry.getObject).mockReturnValue(approvalObject as any);
+            await engine.insert('seed_approval', { status: 'approved' }, { context: { skipStateMachine: true } as any });
+            expect(mockDriver.create).toHaveBeenCalledTimes(1);
+        });
+
         it('still enforces non-state_machine rules under seedReplay (scoped exemption)', async () => {
             vi.mocked(SchemaRegistry.getObject).mockReturnValue({
                 ...approvalObject,

packages/objectql/src/engine.ts

@@ -219,6 +219,18 @@ function mergeReadContext(
   return { ...fromQuery, ...fromOptions };
 }
 
+/**
+ * True when this write is exempt from the `state_machine` validation rule —
+ * both the insert `initialStates` entry check and the update `transitions`
+ * check are skipped for it. Either the seed-specific `seedReplay` flag (#3433)
+ * or the general `skipStateMachine` flag (#3479, set by the REST import runner
+ * for a "historical" import) turns it off. Both are server-set, never
+ * client-supplied.
+ */
+function shouldSkipStateMachine(ctx?: ExecutionContext): boolean {
+  return ctx?.seedReplay === true || ctx?.skipStateMachine === true;
+}
+
 /**
  * Engine Middleware (Onion model)
  */
@@ -2569,7 +2581,7 @@ export class ObjectQL implements IDataEngine {
           try {
             normalizeMultiValueFields(schemaForValidation, rows[i]);
             validateRecord(schemaForValidation, rows[i], 'insert');
-            evaluateValidationRules(schemaForValidation as any, rows[i], 'insert', { logger: this.logger, currentUser: this.buildEvalUser(opCtx.context), skipStateMachine: opCtx.context?.seedReplay === true });
+            evaluateValidationRules(schemaForValidation as any, rows[i], 'insert', { logger: this.logger, currentUser: this.buildEvalUser(opCtx.context), skipStateMachine: shouldSkipStateMachine(opCtx.context) });
           } catch (e) {
             if (!partialMode) throw e;
             rowErrors[i] = e;
@@ -2859,7 +2871,7 @@ export class ObjectQL implements IDataEngine {
                    hookContext.input.data = stripReadonlyFields(updateSchema as any, preRo, suppliedKeys, this.logger) as any;
                    reportDroppedFields(preRo, hookContext.input.data as Record<string, unknown>, 'readonly');
                }
-               evaluateValidationRules(updateSchema as any, hookContext.input.data as Record<string, unknown>, 'update', { previous: priorRecord, logger: this.logger, currentUser: this.buildEvalUser(opCtx.context), skipStateMachine: opCtx.context?.seedReplay === true });
+               evaluateValidationRules(updateSchema as any, hookContext.input.data as Record<string, unknown>, 'update', { previous: priorRecord, logger: this.logger, currentUser: this.buildEvalUser(opCtx.context), skipStateMachine: shouldSkipStateMachine(opCtx.context) });
                result = await driver.update(object, hookContext.input.id as string, hookContext.input.data as Record<string, unknown>, hookContext.input.options as any);
            } else if (options?.multi && driver.updateMany) {
                await this.encryptSecretFields(object, hookContext.input.data as Record<string, unknown>, opCtx.context, hookContext.input.options);
@@ -2927,7 +2939,7 @@ export class ObjectQL implements IDataEngine {
                if (rulesNeedRows) {
                    for (const row of priorRows ?? []) {
                        try {
-                           evaluateValidationRules(updateSchema as any, hookContext.input.data as Record<string, unknown>, 'update', { previous: row, logger: this.logger, currentUser: bulkEvalUser, skipStateMachine: opCtx.context?.seedReplay === true });
+                           evaluateValidationRules(updateSchema as any, hookContext.input.data as Record<string, unknown>, 'update', { previous: row, logger: this.logger, currentUser: bulkEvalUser, skipStateMachine: shouldSkipStateMachine(opCtx.context) });
                        } catch (err) {
                            if (err instanceof ValidationError && row?.id != null) {
                                throw new ValidationError(err.fields.map((f) => ({ ...f, message: `${f.message} (record ${String(row.id)})` })));
@@ -2936,7 +2948,7 @@ export class ObjectQL implements IDataEngine {
                        }
                    }
                } else {
-                   evaluateValidationRules(updateSchema as any, hookContext.input.data as Record<string, unknown>, 'update', { previous: null, logger: this.logger, currentUser: bulkEvalUser, skipStateMachine: opCtx.context?.seedReplay === true });
+                   evaluateValidationRules(updateSchema as any, hookContext.input.data as Record<string, unknown>, 'update', { previous: null, logger: this.logger, currentUser: bulkEvalUser, skipStateMachine: shouldSkipStateMachine(opCtx.context) });
                }
                result = await driver.updateMany(object, ast, hookContext.input.data as Record<string, unknown>, hookContext.input.options as any);
            } else {

packages/platform-objects/src/audit/sys-import-job.object.ts

@@ -64,6 +64,7 @@ export const SysImportJob = ObjectSchema.create({
     ),
     dry_run: Field.boolean({ label: 'Dry Run', required: false, defaultValue: false, group: 'Request' }),
     run_automations: Field.boolean({ label: 'Run Automations', required: false, defaultValue: false, group: 'Request' }),
+    treat_as_historical: Field.boolean({ label: 'Treat As Historical', required: false, defaultValue: false, group: 'Request' }),
 
     // ── outcome ──
     error: Field.textarea({ label: 'Fatal Error', required: false, group: 'Outcome' }),

packages/rest/src/import-prepare.ts

@@ -173,6 +173,9 @@ export interface PreparedImport {
     matchFields: string[];
     dryRun: boolean;
     runAutomations: boolean;
+    /** #3479 — import established historical facts: skip the state_machine rule
+     *  so mid-lifecycle rows are not rejected by `initialStates`. */
+    treatAsHistorical: boolean;
     trimWhitespace: boolean;
     nullValues?: string[];
     createMissingOptions: boolean;
@@ -256,6 +259,11 @@ export async function prepareImportRequest(
     // flag until #2922), so opt-out must be explicit — matches platform
     // convention (Salesforce runs triggers on import by default).
     const runAutomations = body?.runAutomations !== false;
+    // Default OFF (opt-in): a normal import must still walk the state machine —
+    // only an explicit "historical" import skips it (#3479), so mid-lifecycle
+    // rows aren't rejected by `initialStates`. Unlike `runAutomations`, the safe
+    // default is the strict one.
+    const treatAsHistorical = body?.treatAsHistorical === true;
     const trimWhitespace = body?.trimWhitespace !== false;
     const nullValues: string[] | undefined = Array.isArray(body?.nullValues)
         ? body.nullValues.filter((v: any) => typeof v === 'string')
@@ -392,6 +400,7 @@ export async function prepareImportRequest(
         ok: true,
         prepared: {
             rows, metaMap, writeMode, matchFields, dryRun, runAutomations,
+            treatAsHistorical,
             trimWhitespace, nullValues, createMissingOptions, skipBlankMatchKey,
         },
     };

packages/rest/src/import-runner-historical.test.ts

@@ -0,0 +1,89 @@
+// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
+
+/**
+ * #3479 — a "historical" import carries curated established facts (a batch of
+ * already-`closed` tickets, `closed_won` deals), so it must skip the object's
+ * `state_machine` rule: otherwise `initialStates` rejects every mid-lifecycle
+ * row on insert. runImport owns the option→context mapping — it sets
+ * `skipStateMachine` on the write context iff `treatAsHistorical` is set. A
+ * normal import must NOT set it (the FSM still applies).
+ *
+ * The engine's actual exemption behavior (skipStateMachine → skip the rule) is
+ * tested in @objectstack/objectql (engine.test.ts); this pins the wiring the
+ * import runner owns.
+ */
+
+import { describe, it, expect, vi } from 'vitest';
+import { runImport, type ImportProtocolLike } from './import-runner';
+import type { ExportFieldMeta } from './export-format.js';
+
+const metaMap = new Map<string, ExportFieldMeta>([['name', { name: 'name', type: 'text' }]]);
+
+const baseOpts = {
+  objectName: 'ticket',
+  metaMap,
+  writeMode: 'insert' as const,
+  matchFields: [] as string[],
+  dryRun: false,
+  runAutomations: false,
+  trimWhitespace: true,
+  createMissingOptions: false,
+  skipBlankMatchKey: false,
+};
+
+/** Provider mock that records the write context every persist call received. */
+function makeProvider() {
+  const contexts: any[] = [];
+  let idc = 0;
+  const p: ImportProtocolLike = {
+    findData: vi.fn(async () => []),
+    createData: vi.fn(async (args: any) => {
+      contexts.push(args.context);
+      return { id: `d${++idc}`, ...args.data };
+    }),
+    updateData: vi.fn(async (args: any) => {
+      contexts.push(args.context);
+      return { id: args.id, ...args.data };
+    }),
+    createManyData: vi.fn(async (args: any) => {
+      contexts.push(args.context);
+      return { records: args.records.map((r: any) => ({ id: `d${++idc}`, ...r })) };
+    }),
+  };
+  return { p, contexts };
+}
+
+describe('runImport — historical import skips the state machine (#3479)', () => {
+  it('sets skipStateMachine on the write context when treatAsHistorical is true', async () => {
+    const { p, contexts } = makeProvider();
+    const summary = await runImport({
+      ...baseOpts, p, rows: [{ name: 'a' }, { name: 'b' }], treatAsHistorical: true,
+    });
+    expect(summary.created).toBe(2);
+    expect(contexts.length).toBeGreaterThan(0);
+    for (const ctx of contexts) expect(ctx?.skipStateMachine).toBe(true);
+  });
+
+  it('does NOT set skipStateMachine for a normal import (the FSM still applies)', async () => {
+    const { p, contexts } = makeProvider();
+    await runImport({ ...baseOpts, p, rows: [{ name: 'a' }], treatAsHistorical: false });
+    expect(contexts.length).toBeGreaterThan(0);
+    for (const ctx of contexts) expect(ctx?.skipStateMachine).toBeUndefined();
+  });
+
+  it('defaults to off when the option is omitted (safe default — walk the FSM)', async () => {
+    const { p, contexts } = makeProvider();
+    await runImport({ ...baseOpts, p, rows: [{ name: 'a' }] });
+    expect(contexts.length).toBeGreaterThan(0);
+    for (const ctx of contexts) expect(ctx?.skipStateMachine).toBeUndefined();
+  });
+
+  it('leaves the automation toggle independent (skipAutomations still tracks runAutomations)', async () => {
+    const { p, contexts } = makeProvider();
+    await runImport({ ...baseOpts, p, rows: [{ name: 'a' }], treatAsHistorical: true, runAutomations: true });
+    for (const ctx of contexts) {
+      expect(ctx?.skipStateMachine).toBe(true);
+      expect(ctx?.skipAutomations).toBe(false); // runAutomations:true ⇒ don't skip
+    }
+  });
+});

packages/rest/src/import-runner.ts

@@ -102,6 +102,11 @@ export interface RunImportOptions {
   matchFields: string[];
   dryRun: boolean;
   runAutomations: boolean;
+  /** #3479 — treat rows as established historical facts: the write context
+   *  carries `skipStateMachine`, so mid-lifecycle values aren't rejected by the
+   *  object's `state_machine` `initialStates` (insert) / `transitions` (update).
+   *  Optional here (runner default is off); `prepareImportRequest` always sets it. */
+  treatAsHistorical?: boolean;
   trimWhitespace: boolean;
   nullValues?: string[];
   createMissingOptions: boolean;
@@ -163,7 +168,7 @@ const yieldToEventLoop = (): Promise<void> =>
 export function runImport(opts: RunImportOptions): Promise<ImportRunSummary> {
   const {
     p, objectName, environmentId, context, rows, metaMap,
-    writeMode, matchFields, dryRun, runAutomations,
+    writeMode, matchFields, dryRun, runAutomations, treatAsHistorical,
     trimWhitespace, nullValues, createMissingOptions, skipBlankMatchKey,
     onProgress, shouldCancel, captureUndo,
   } = opts;
@@ -262,7 +267,14 @@ export function runImport(opts: RunImportOptions): Promise<ImportRunSummary> {
     return recs[0];
   };
 
-  const writeCtx = { ...(context ?? {}), skipAutomations: !runAutomations };
+  const writeCtx = {
+    ...(context ?? {}),
+    skipAutomations: !runAutomations,
+    // #3479 — a "historical" import carries curated established facts, so the
+    // engine skips the state_machine rule for these writes (initialStates on
+    // insert, transitions on update). Default off: a normal import walks the FSM.
+    ...(treatAsHistorical ? { skipStateMachine: true } : {}),
+  };
 
   // Sparse-indexed by row position `i` (not push-only): CREATE rows are
   // resolved immediately but their write is deferred to a later batch flush,

packages/rest/src/rest-server.ts

@@ -3831,6 +3831,7 @@ export class RestServer {
                         write_mode: prepared.writeMode,
                         dry_run: prepared.dryRun,
                         run_automations: prepared.runAutomations,
+                        treat_as_historical: prepared.treatAsHistorical,
                         created_at: createdAt,
                         ...(createdBy ? { created_by: createdBy } : {}),
                     };
@@ -4003,7 +4004,11 @@ export class RestServer {
                     const objectName = String(row.object_name ?? '');
                     const log = parseUndoLog(row.undo_log)!;
                     // Undo automations too: reversing writes shouldn't re-fire triggers.
-                    const writeCtx = { ...(context ?? {}), skipAutomations: true };
+                    // Skip the state machine as well (#3479): restoring a prior snapshot
+                    // re-writes the row's earlier state, which need not be a legal
+                    // transition from where it is now — an undo reinstates an established
+                    // fact, it does not walk the FSM.
+                    const writeCtx = { ...(context ?? {}), skipAutomations: true, skipStateMachine: true };
                     let deleted = 0, restored = 0, failed = 0;
 
                     // Delete created records first (they didn't exist before).