Skip to content

fix(objectql): resync the engine fallback autonumber counter — adopt exempt writers' numbers, re-seed on collision (#6806) - #6907

Merged
os-zhuang merged 6 commits into
mainfrom
claude/issue-6806-engine-autonumber-reseed
Aug 9, 2026
Merged

fix(objectql): resync the engine fallback autonumber counter — adopt exempt writers' numbers, re-seed on collision (#6806)#6907
os-zhuang merged 6 commits into
mainfrom
claude/issue-6806-engine-autonumber-reseed

Conversation

@os-zhuang

@os-zhuang os-zhuang commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Fixes #6806

Engine-side only. packages/objectql/src/engine.ts (the applyAutonumbers / seedAutonumber region), its tests, one changeset. No spec change; packages/drivers/** untouched (#5499 freeze; maintainer ruling 2026-08-06 = A).

The counter stays global — tenant-partitioning remains parked per #5495's disposition and is not relitigated here.


The card's real question: which resync shape is correct?

Candidates were (a) re-seed on collision, (b) re-seed on the first write after an exempt-writer insert is detectable, (c) both. Measured answer: (c) — but (b) is not "re-seed", it is "adopt", and that distinction is the whole measurement.

What the fallback path can observe, and what each candidate costs per insert

The decisive fact is that an exempt writer's insert passes through applyAutonumbers itself. isSystem seed replay, a preserveAudit import and a beforeInsert hook stamp are exactly the three writers #5503's strip exempts, and all three land on the if (current != null && current !== '') continue; branch. The value is in hand at that moment. So candidate (b) has two possible implementations with wildly different costs:

shape cost on a normal (generating) insert cost on an exempt insert cost on a collision
(b) as invalidate + re-seed on next write one full scope scan on the next generating insert (paging walk, AUTONUMBER_SEED_PAGE_SIZE rows/page) 0
(b) as adopt the value in place ← taken 0 one String.match, no query
(a) re-seed on collision ← taken 0 0 one scope scan + one re-issued create, bounded to 3 attempts

A seed replay of N rows interleaved with normal inserts costs N full scope scans under invalidate-and-reseed and zero queries under adopt. That measurement is what rules out the re-seed spelling of (b).

Why adoption is correct, not merely cheap — judged against #6468

Adoption reads the supplied value with readAutonumberCounter, which is the anchoring logic lifted verbatim out of seedAutonumber's scan loop and now shared by both callers (PR #6553 / #6468 landed it as the inline anchored branch; this PR only moves it). Same prefix check, same anchored-vs-unanchored rule, same suffix-stripped-when-it-matches-never-required, same linear /\d+/ forms (the CodeQL polynomial-ReDoS constraint). Two copies of that reading would be two different answers for one row, i.e. a duplicate record number — the harm the whole family exists to prevent.

The invariant this buys: a warm counter now answers what a cold restart would answer. Adoption applies exactly the reading the seeding scan would have applied to that row once it is persisted.

The two are disjoint, which is why both ship

  • Adoption covers drift the engine can observe but the store cannot report.
  • Collision re-seed covers drift the store does report but the engine could not observe (another instance, a direct driver write, a restore).

Neither subsumes the other, and neither costs anything on the happy path.

The dependency question: #6250 is satisfied, not blocking

No Blocked-by: line, deliberately. #6250 is closed completed and its predicate landed in @objectstack/types as isUniqueViolationError, with uniqueViolationColumn added by #6544. @objectstack/objectql already depends on @objectstack/types, so adopting them adds no edge. No dialect word-list is written in the engine (PD #12, precedent #5841).

One measurement that changed the retry gate. #6544's doc anticipates this caller ("#5495's autonumber-retry branch asks a yes/no question of the answer"), but its contract is deliberately narrow: a value comes back only when the driver printed a determinable column, never an index name. MongoDB prints E11000 ... index: doc_no_1 dup key: { doc_no: "D-0005" } — an index, so uniqueViolationColumn answers undefined. Gating the retry on column === <autonumber field> would therefore make the resync unreachable on driver-mongodb — the only in-repo driver that both takes this path and can raise a collision. The gate landed as:

isUniqueViolationError(error)
  && (uniqueViolationColumn(error) === undefined || <it names a field this insert issued>)

A conflict the dialect attributes to a different column is rethrown untouched — #5495's «非本字段的冲突原样上抛», honoured exactly.


⚠ The collision half is STORAGE-DEPENDENT — measured across all five drivers, by name

The collision branch is triggered by the store rejecting the duplicate, so it reaches only drivers that take this fallback path and enforce uniqueness. Rather than gesturing at "the storage layer", supports.autonumber was read from each of the five in-repo drivers:

driver supports.autonumber source fallback path? uniqueness on the autonumber column a collision appears as
driver-memory absent — supports = {} memory-driver.ts:174 yes none, ever nothing — silent duplicate
driver-mongodb absent — { batchSchemaSync: true } mongodb-driver.ts:94 yes single-field unique index (idx_<f>_unique) when the field declares unique E11000 duplicate key → re-seed + re-issue
driver-sql autonumber: true sql-driver.ts:2256 no
driver-sqlite-wasm inherited — extends SqlDriver, no supports override sqlite-wasm-driver.ts:67 no
driver-turso inherited — ...super.supports turso-driver.ts:180 no

This confirms the PM's sketch: the violation-triggered retry protects essentially ONE backend — driver-mongodb with a unique autonumber field. Two of five drivers take the path; one of those two can raise anything.

InMemoryDriver.create is a table.push() storing no constraints of any kind — its own docstring has said so since #4065, where it also stopped calling itself production-ready and started calling itself a WEAK oracle, for this exact reason.

Anchored to the existing ruled answer, not a second one

This is not a fresh prose claim. It is the reading the repo already ruled and gates, in scripts/driver-memory-census.ledger.json's disposition for packages/runtime/src/autonumber-seed-cross-side-parity.integration.test.ts — axis ruled-permanent, ruling «#6664 A, maintainer 2026-08-08 — inherits #5704 Q2 = B»:

the engine-fallback arm of the #6468 convergence pin: InMemoryDriver declares supports = {}, so the ENGINE's autonumber seeding owns the counter. No SQL backend can stand in — SqlDriver advertises the capability and its own sequence bootstrap answers instead, which is the arm this file already runs on the other side of the same toBe().

The engine doc comment, the test header and the changeset all cite that ruling rather than restating it. Authoring a second answer to "who owns the autonumber counter" is verbatim the one-contract-two-numbers shape graded on #6832, and this change does not create it while fixing a different one.

check:driver-memory-census — no unledgered arrival

The new test rig is a hand-rolled fake driver object; the file imports no driver package at all (driver-memory appears only in prose). So it adds no consumer, the ledger needs no entry, and no disposition was invented to silence a gate. pnpm check:driver-memory-census re-run after these edits: PASS.

The outcome, pinned rather than implied

test §(3b) pins the silent duplicate directly (two rows carrying D-0005, one create call, no error), so "collisions are handled" cannot read as true where it is not (PD #10 — never advertise a capability the runtime does not deliver). A companion pin shows adoption does hold on that driver, so the coverage statement is not empty there.

Follow-up finding for the drivers lane, not implemented here: the remaining silent-duplicate row can only be closed by uniqueness enforcement in driver-memory, which is out of scope under the #5499 freeze. It is explicitly not an engine-side pre-issue existence probe: that costs a query on every insert (the cost this design avoids) and is still TOCTOU-racy, so it would trade a silent duplicate for a rarer silent duplicate at double the read cost.

What an author gets on a batch collision

The batch path is re-seeded but never re-issued: bulkCreate may be partially applied by a driver without a transaction, so re-writing the batch could duplicate the rows that did land — strictly worse than the collision.

So, plainly: insert(object, rows[]) and insertMany reject with the driver's own duplicate-key error — never ERR_AUTONUMBER_COLLISION, which is the single-row identity for "re-issued and still refused". Whether any row was written is the driver's answer, not the engine's. The engine's guarantee is narrower and is the one that matters: the stale counter is dropped, so the caller's retry re-seeds instead of walking into the same collision one number at a time. Pinned for both entry points, including the negative assertion that the code is not ERR_AUTONUMBER_COLLISION.


Acceptance

1. Exempt-writer inserts no longer leave the counter stale. §(1), 4 tests — isSystem seed replay, preserveAudit import, beforeInsert hook stamp, and a batch of exempt rows (max wins, not last-seen). Each asserts the next generated number follows the supplied one (D-0009D-0010, not D-0005) and that it did so with one seeding scan for the whole sequence, i.e. by adoption rather than re-reading. Both fallback drivers' shapes are used as fixtures: driver-memory's (no uniqueness, §3b) and driver-mongodb's (E11000, §3).

§(2) pins adoption's four refusals — never lowers a counter; never seeds an unseeded one (that would skip the scan and answer from a single row — the defect arrived at from the fix); never adopts across scope (a past-date import cannot burn today's band); never throws (an exempt write was accepted before and still is), with the generating branch's {field}-empty refusal kept as a control.

2. A collision does not silently burn numbers. §(3), 9 tests. Observable outcome asserted, never a bare toThrow:

  • re-seed + succeed: the insert returns D-0008 after colliding on D-0005, with exactly 2 create calls and 2 seeding scans;
  • convergence: the following insert is D-0009 — the pre-fix storm was that the stale counter survived, so 0006 and 0007 collided in turn, one burned number and one raw driver error per insert;
  • refusal identity: after AUTONUMBER_COLLISION_ATTEMPTS (3) the error carries code: 'ERR_AUTONUMBER_COLLISION', names the field, says "No record was written", and keeps the driver's error as cause — the raw driver error is not the contract;
  • no burn on refusal: after that refusal the next successful insert is D-0004, i.e. the store decides, not the three advanced in-memory increments;
  • pass-through: a different-column conflict, a non-unique failure, and a collision on a value the engine did not issue are each rethrown untouched with exactly one create call.

3. #6114's read-failure discrimination survives. §(4), 3 tests: missing table still seeds from 0; an outage on the first seed still propagates and writes nothing; and — the new call site — an outage during the RE-seed propagates, is never swallowed into 0 or a stale value, attempts exactly one further create, and once the store is readable comes back from the real max (D-0008), not from 1.

4. Reverse verification — below.

Reverse verification (predicted BEFORE running)

Removal applied: applyAutonumbers' adopt branch reverted to a bare continue; the single-row path calls driver.create directly; the batch catch dropped. Tests unchanged, 24 total.

Predicted: 13 red / 11 green. Actual: 13 red / 11 green — same 13 tests, same 11 tests.

# test predicted signature actual
1 isSystem seed replay expected 'D-0005' to be 'D-0010' ✅ exact
2 preserveAudit import expected 'D-0005' to be 'D-0022' ✅ exact
3 beforeInsert hook stamp expected 'D-0005' to be 'D-0031' ✅ exact
4 batch of exempt rows expected 'D-0005' to be 'D-0015' ✅ exact
5 E11000 → write succeeds rejects with E11000 … D-0005 ✅ exact
6 converges rejects with E11000 … D-0005 ✅ exact
7 Postgres names the column rejects with duplicate key value violates unique constraint "doc_doc_no_key" ✅ exact
8 named refusal after bounded attempts expected 11000 to be 'ERR_AUTONUMBER_COLLISION' ✅ exact
9 burns no numbers when it refuses toMatchObject({code:'ERR_AUTONUMBER_COLLISION'}) vs code 11000 ✅ exact
10 batch drops the counter expected 'D-0006' to be 'D-0010' ⚠️ stronger: rejects with E11000 … D-0006
11 insertMany same way expected 'D-0006' to be 'D-0010' ⚠️ stronger: rejects with E11000 … D-0006
12 adoption holds on driver-memory expected 'D-0005' to be 'D-0010' ✅ exact
13 outage during the RE-seed expected … /Connection terminated/ but got 'E11000 …' ✅ exact

Green, as predicted: all five of §(2)'s refusals, the three §(3) pass-through cases, §(3b)'s silent-duplicate pin, and both unchanged §(4) read-failure controls.

The one deviation, reported honestly: #10 and #11 failed harder than predicted. I predicted an assertion diff on the follow-up insert; the follow-up insert instead threw, because without the counter drop it collides on D-0006 — a number the unobservable writer already holds. Same test, same cause, more severe signature. That deviation is itself a finding: while writing the prediction I noticed the original fixture (a single out-of-band row) made "drop the counter" and "merely advance past the collision" indistinguishable — both would have produced D-0006 and the test would have passed against the removal. The fixture was widened to a whole band (D-0005D-0009) before running, so the pin now discriminates. Commit 28aaa1b.

Test placement

New sibling packages/objectql/src/engine-autonumber-resync.test.ts, not a lone new file elsewhere and not folded into an existing one. The objectql autonumber coverage is already a one-file-per-concern familyengine-autonumber-{batch,defer,runtime-owned,seed-outage,seed-scan,seed-suffix}.test.ts — each named for the issue it pins. -resync is that family's next member and is where a future divergence in this behaviour gets caught. It reuses the #6468 suffix file's fake-driver rig deliberately, so the two files must keep agreeing about what the seeding scan is allowed to send.

Fake driver, real engine — so no engine write-verb dispatch contract is involved, no assertEngineDeleteDispatch fake engine is introduced, and no driver package is imported.

Gates

Enumerated fresh from .github/workflows/lint.yml and re-enumerated after each origin/main merge — it grew twice during this task (check:agent-model-declared, then check:tenant-chokepoint; 57 check:* scripts now). All run individually, all green:

  • pnpm lint
  • 57/57 check:* ✅ — including check:driver-memory-census (re-run after the doc changes), check:durability-log-level, check:engine-double-contract, check:type-check-coverage, check:driver-conformance, check:agent-model-declared, check:tenant-chokepoint, and the @objectstack/spec-filtered set
  • full build (turbo run build --filter='!@objectstack/docs' --concurrency=2) ✅
  • turbo run typecheck over packages/*, packages/*/*, apps/* — 120/120 ✅; tsc --noEmit -p packages/objectql
  • check:type-check-debt after that build ✅ — "none above its recorded number"; no ledger entry raised (the -N can be lowered lines are informational and pre-existing)

check:platform-checklist is maintainer-only on-demand per its own step comment and was not run.

packages/objectql: 159 files / 2752 tests green, no regressions; the new file is 24/24.

origin/main merged three times during the task (it kept moving under me); none of the merges touched packages/objectql.

claude added 3 commits August 9, 2026 01:30
…ion re-seed) (#6806)

The engine fallback autonumber path seeded `object.field.<scope>` once and then
incremented purely in memory, so it drifted below the store's real max in two
ways it could never recover from.

- Adopt an exempt writer's supplied record number into the counter (isSystem
  seed replay / preserveAudit import / beforeInsert hook stamp). Free: one
  string parse, no extra query. Read by #6468's anchoring rules, now shared
  with the seeding scan as `readAutonumberCounter` so the two cannot drift.
- Re-seed and re-issue on a unique violation attributable to an autonumber the
  engine issued, bounded to 3 attempts, then refuse with
  `code: 'ERR_AUTONUMBER_COLLISION'` carrying the driver error as `cause`.
  The predicates come from `@objectstack/types` (#6250 / #6544), never a
  dialect word-list of the engine's own.

Batch inserts drop the stale counter but are never re-issued (bulkCreate may be
partially applied). #6114's read-failure discrimination is unchanged and now
also covers the re-seed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015QW3F6hGmFkf1RpwGBthDo
…the batch outcome (#6806)

driver-memory enforces no uniqueness at all (its create is a table.push(),
#4065), so the collision branch is unreachable there and a duplicate lands
silently. Named and pinned rather than left implied (PD #10); adoption is the
half that covers that driver. Also pins what an author gets on a batch
collision: the driver's own error, never ERR_AUTONUMBER_COLLISION, with the
counter dropped so the caller's retry converges.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015QW3F6hGmFkf1RpwGBthDo
@vercel

vercel Bot commented Aug 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 9, 2026 4:12am

Request Review

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/objectql.

15 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/concepts/metadata-lifecycle.mdx (via @objectstack/objectql)
  • content/docs/data-modeling/formulas.mdx (via packages/objectql)
  • content/docs/deployment/migration-from-objectql.mdx (via @objectstack/objectql)
  • content/docs/deployment/vercel.mdx (via @objectstack/objectql)
  • content/docs/kernel/runtime-services/examples.mdx (via packages/objectql)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/objectql)
  • content/docs/kernel/services.mdx (via @objectstack/objectql)
  • content/docs/permissions/authentication.mdx (via @objectstack/objectql)
  • content/docs/permissions/system-context.mdx (via packages/objectql)
  • content/docs/plugins/index.mdx (via @objectstack/objectql)
  • content/docs/plugins/packages.mdx (via @objectstack/objectql)
  • content/docs/protocol/kernel/index.mdx (via @objectstack/objectql)
  • content/docs/protocol/objectql/query-syntax.mdx (via packages/objectql)
  • content/docs/protocol/objectql/state-machine.mdx (via @objectstack/objectql)
  • content/docs/releases/implementation-status.mdx (via @objectstack/objectql)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling labels Aug 9, 2026
claude added 2 commits August 9, 2026 04:11
…on half (#6806)

Measured `supports.autonumber` across all five in-repo drivers rather than
asserting "the storage layer": only driver-memory (`supports = {}`) and
driver-mongodb (bit absent) take the engine fallback path; driver-sql declares
`autonumber: true` and driver-sqlite-wasm / driver-turso inherit it via
`extends SqlDriver`. Of the two, only driver-mongodb can raise a unique
violation, so the collision retry protects essentially one backend.

Anchored to the reading the repo already ruled and gates —
scripts/driver-memory-census.ledger.json's `ruled-permanent` disposition for
autonumber-seed-cross-side-parity.integration.test.ts ("InMemoryDriver declares
`supports = {}`, so the ENGINE's autonumber seeding owns the counter") — rather
than authoring a second answer to who owns the counter (#6832's shape).

The test rig is a hand-rolled fake driver and imports no driver package, so
check:driver-memory-census sees no unledgered arrival; re-run to confirm.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015QW3F6hGmFkf1RpwGBthDo
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/xl tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Engine fallback autonumber: counter seeds once and never resyncs; collisions burn numbers with no recheck (residual split from #5495)

2 participants