Skip to content

Seed sites3.next_page_id after a full site import (fixes TyE306KSH4)#738

Open
IvanTheGeek wants to merge 1 commit into
debiki:mainfrom
IvanTheGeek:fix-next-page-id-after-import
Open

Seed sites3.next_page_id after a full site import (fixes TyE306KSH4)#738
IvanTheGeek wants to merge 1 commit into
debiki:mainfrom
IvanTheGeek:fix-next-page-id-after-import

Conversation

@IvanTheGeek

Copy link
Copy Markdown

The bug (TyE306KSH4)

After a full site import/restore, sites3.next_page_id is left at its default (1) instead of being advanced past the imported pages' ids. Page ids come from that stored counter (inc_next_page_id), unlike post & category ids which self-heal via max(id)+1. So the next new page starts at id 1, collides with the just-imported pages, and nextPageId() gives up after 100 laps → TyE306KSH4.

Real-world repro: a self-hosted site imported with pages up to id 145 while next_page_id stayed at 1 → creating any new topic failed.

The fix

In importCreateSite(), after inserting the pages, seed next_page_id to max(highest numeric imported page id + 1, dump's declared next_page_id) via a new bumpNextPageId() transaction method:

update sites3 set next_page_id = greatest(next_page_id, ?) where id = ?

GREATEST ⇒ idempotent, never lowers the counter. Non-numeric/special page ids and ids larger than Int are ignored (toIntOption), so it can't overflow.

This changes only the id handed to new pages created after the import. Existing pages keep their ids, so links/backlinks to them are unaffected. The upsertIntoExistingSite path already advances the counter via nextPageId() and is left untouched.

Tests

  • App-level regression test (SitePatcherAppSpec + a DumpMaker.importCreateSite helper): imports a dump with a stale counter, asserts next_page_id is seeded past the max page id, and creates a new page without TyE306KSH4. Verified it fails on unpatched code (1 was not equal to 4) and passes with the fix.
  • Full wdio7 e2e, baseline vs. change, run locally (139 specs, serial). Methodology + results in the working log linked below.

Heads-up: 3 e2e specs need their expected page ids updated (not a fix bug)

With the fix, 3 specs regress: api-upsert-posts.2br.d, alias-anons-basic-perm.2br.f, alias-anons-basic-temp.2br.f. The exact cause:

The e2e site-builder assigns page ids as opts.id + 19000 (tests/e2e-wdio7/utils/site-builder.ts:127), so test dumps carry pages at ids ~19002–19004 while declaring nextPageId = 100 — internally inconsistent, i.e. exactly the stale-counter condition this fix heals. On unpatched code the counter isn't seeded, so new pages gap-fill the free low ids (2, 3…), which these specs hard-code (c.SecondPageId = '2'). With the fix, the counter is correctly seeded past the max (→ 19005), so new pages get high ids and the hard-coded assertions fail.

Only pages created after import are affected — existing pages/links are not, and real-world sites (contiguous ids) behave identically to before. These 3 specs — or the builder's +19000/nextPageId inconsistency — just need their expected ids updated. I left them untouched since they encode intent about your test harness; happy to follow up whichever way you prefer.

Already-imported sites (one-time heal)

The fix prevents new damage; it doesn't retro-heal sites imported before it (like the repro). A safe, idempotent one-liner (no-op on healthy sites):

update sites3 s set next_page_id = greatest(s.next_page_id, coalesce((
  select least(max((p.page_id)::bigint) + 1, 2147483647)
  from pages3 p
  where p.site_id = s.id and p.page_id ~ '^[0-9]+$' and length(p.page_id) <= 9), 1));

Deliberately not shipped as a migration — a bulk row-update across all tenants at startup felt like your call, not something to run unasked.

Full working log & investigation

The complete diagnosis, the app-test proof, and the full e2e testing (including a fairly deep investigation into why the fix changes those 3 specs) are documented here, if useful:
https://forum.ivanthegeek.com/-189

Take it as-is, modify freely, or just use it as a pointer — whatever's most helpful. 🙂


Authored by Ivan (IvanTheGeek); the fix + analysis were done with AI assistance (Claude), as noted in the commit and the working log — flagging for transparency.

@IvanTheGeek
IvanTheGeek force-pushed the fix-next-page-id-after-import branch from fa89004 to 66bb46f Compare July 6, 2026 06:50
After importing/restoring a whole site, sites3.next_page_id was left at
its default (1) instead of being advanced past the imported pages' ids.
Page ids come from that stored counter (inc_next_page_id), unlike post
and category ids which self-heal via max(id)+1. So the next new page
started at id 1, collided with the just-imported pages, and nextPageId()
gave up after 100 laps [TyE306KSH4]. Real-world repro: a site imported
with pages up to id 145 while next_page_id stayed at 1 -> new-topic
creation broke.

Fix: in importCreateSite(), after inserting the pages, seed next_page_id
to max(highest numeric imported page id + 1, the dump's declared
next_page_id), via a new bumpNextPageId() transaction method:
`update sites3 set next_page_id = greatest(next_page_id, ?) where id = ?`
(GREATEST => idempotent, never lowers the counter). Non-numeric/special
page ids are ignored (toIntOption); the seed is computed in Long and
rejected at the Int page-id ceiling (throwBadRequestIf, TyEIMPPAGEIDMAX),
so it can't silently overflow if a page id is exactly Int.MaxValue.

This changes only the id handed to *new* pages created after the import
— existing pages keep their ids, so links/backlinks to them are
unaffected. The upsert path (upsertIntoExistingSite) already advances the
counter via nextPageId() and is left untouched.

Adds an app-level regression test (SitePatcherAppSpec) that reproduces
the bug (fails on unpatched code: "1 was not equal to 4") and a
DumpMaker.importCreateSite() helper for it.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

I, Ivan Rainbolt <talkyard@github-commits.ivanthegeek.com>, agree to the Contributor License Agreement, docs/CLA-v2.txt.
@IvanTheGeek
IvanTheGeek force-pushed the fix-next-page-id-after-import branch from 66bb46f to ab3d5b9 Compare July 6, 2026 06:52
@IvanTheGeek
IvanTheGeek marked this pull request as ready for review July 6, 2026 06:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant