Skip to content

docs: information modeling, a docs audit, and the code fixes it surfaced - #69

Merged
damusix merged 38 commits into
masterfrom
next
Aug 2, 2026
Merged

docs: information modeling, a docs audit, and the code fixes it surfaced#69
damusix merged 38 commits into
masterfrom
next

Conversation

@damusix

@damusix damusix commented Aug 2, 2026

Copy link
Copy Markdown
Collaborator

Three things, in order.

A new Information Modeling section documents ignatius, the planning half of the workflow. The site also gains a Why noorm page, a reworked top bar, and Apache 2.0 in place of MIT. The hero and footer had been claiming ISC, which nothing in either repo ever was.

Then thirteen audits, one per sidebar topic, checked every factual claim against the implementation. Most of what they found was ordinary rot. Some was not: several pages described safety properties the code does not have.

The final commits fix nine defects those audits surfaced. Look at these first: run file --dry-run executed the SQL, ci init marked production configs as test databases so requireTest passed against prod, and the transfer planner could copy the encrypted vault into a destination database.

Merging deploys the docs site and flips the public license to Apache 2.0.

damusix added 26 commits August 1, 2026 02:10
`executed_at` carries no time zone and noorm writes UTC into it, but `pg`
and `mysql2` both parse that naive text through the host's local zone. A
change applied a second ago came back four hours in the future on a UTC-4
host, which the TUI rendered as "Applied ... in 4 hours".

Corrected at the hydration boundary rather than in the driver config: the
column is read only for noorm's own tracking tables, so the fix cannot
shift a user's timestamps read through the SDK. Both drivers were measured
against live containers; mssql is left alone because tedious was not.
The TUI page's screens were hand-drawn ASCII that had already drifted from
the app — the DB screen's `[c] Create` claimed to build from SQL files when
it only creates the database and tracking tables, and the keyboard tables
listed a Home `k` that does not exist while omitting `+`, `v`, `u` and the
number shortcuts. All of it is now generated from the running TUI by VHS
tapes in docs/tapes/, so it regenerates instead of rotting.

Also fixes the mobile complaints: the nav did not stick (VitePress only
pins it above 960px) and the hero terminal rendered centre-aligned and
clipped. Adds og/twitter meta, which the site had none of, and trims the
front page down to the pitch with the relational-design essay moved to its
own page.

The tapes record against an isolated sandbox with HOME redirected, so
recording never touches a real ~/.noorm.
The site covered building a schema but not deciding what it should be,
so ignatius had no presence on noorm.dev at all.

These pages carry the narrative — intention, usage, key inheritance,
SSADM flows, the modeling skill, best practices — and link out to the
ignatius repo for the exhaustive rule catalogs. ignatius releases on its
own cadence, so duplicating its validation and folder-format references
here would drift within a release.

The home page gets a prose section rather than a seventh feature card:
VitePress switches the feature grid from 3-across to 2-across at seven
items, which would restructure the whole block.
The headless block listed `noorm config add`, which is TUI-only and exits 1
with "Interactive only — run: noorm ui" — so the first command a new user
copied failed, and the two after it could not work either, since `run build`
and `change ff` need a config that only the TUI can create. Replaced with the
real path, plus `ci init` for the case with no TUI to fall back on.

The SDK section described "consumers (queries), producers (mutations), and
guards (validation)". None of that is the SDK's vocabulary: the guide teaches
a domain class per schema area holding a typed Context, and the `guards`
module exports connection and access errors, not validators.
Export conditions are order-sensitive, so `types` after `import` resolves
only by luck — it works today because no `require` condition exists to
shadow it, and breaks silently the moment one is added. publint flags it as
an error; both packages now pass clean.
Upstream renamed `access.mcp` to `access.agent` and changed the form label
to "Agent Role (MCP/CLI access)", so every still and the walkthrough GIF
showed a field that no longer exists.

The tape was worse than stale. It pressed Up twice to move the role from
Admin to Viewer, which was right until DEFAULT_ACCESS started defaulting the
agent channel to viewer — the same two presses now wrap round to Admin, so
the recording would have handed an agent full access while narrating
read-only. The default is already what we want, so it just confirms.
The badge row advertised only @noormdev/cli, but the SDK is published and
versioned in lockstep with it. Bronze rather than ember so the CLI stays the
one loud thing in the row.
Home duplicated the logo link, and Dev Docs sent readers into a separate
sidebar tree from the top bar, which read like a peer of Guide and CLI
when it is contributor material. It now sits in the Reference group as
"Contributor Documentation", reachable without advertising itself to
people reading the product docs.
Environment-only mode was documented as a CI feature, so nothing told a
reader that it is also how a deployed application connects. The gap showed
up as "does the SDK offer a path to createContext with user/pass/dialect?" —
it does, via NOORM_CONNECTION_*, just nowhere a person looking to deploy
would find it.

Separates schema delivery from application runtime, since folding the former
into boot is what makes horizontal scaling hurt, and records the two limits
worth knowing up front: pool max is per-replica, and connection details come
from the process environment, so one process serves one database.
Getting Started and Reference stay pinned open as the entry and exit
points. Features and Information Modeling were open only because they
omitted `collapsed`, which in VitePress means "no toggle at all" rather
than "open by default", so neither could be closed and the sidebar
opened as a wall of links.
An 88-second walk through all three views on the modeling overview, which
until now described the app without showing it.

preload="none" plus a poster frame means the page costs 59 KB until
someone presses play, rather than pulling the 6.5 MB file on every visit.
The page names IDEF1X as the premise and then moves straight to the file
format, leaving the reader to learn the methodology elsewhere. The skill
covers the judgment the format cannot, so it is offered next to the
claim, framed as optional in both places it appears.
The page listed habits with nothing to check them against. It now sends
readers to the LLM memory model, the same one in the demo recording, and
names what to look at: full example coverage, the deliberate mix of
surrogate roots and inherited chains, and rules living in entity bodies.

It also says which two practices that model does not exercise, and sends
those to key-inherited instead, so the exemplar is not oversold.
The pieces existed separately: the MCP server, the agent access roles,
the explorer, and the skill's discover mode. Nothing connected them into
the one job people arrive with, which is modeling a database they
already run.

Spells out the catalog-permission step in particular. Postgres and SQL
Server both filter metadata by privilege and return fewer rows rather
than an error, so an under-privileged user yields a model that looks
finished and silently omits tables.
The viewer ships noorm's logo, title and copyright by default, which
reads as a claim on work it does not own. Branding exists so the model
carries your organization instead, and the export is the artifact that
goes to clients and approvers, so the page says so where that hand-off
happens.

Also rewrites the NOORM_CHANNEL caveat on the reverse-engineering page.
"An agent can set it too" described normal agent behaviour; the real
exposure is an agent talked into it by content it reads, which matters
here because extraction feeds it comments and procedure bodies nobody
on your team wrote.
The page read as a noorm workflow that happened to produce a model.
Extraction is the bottom-up path in IDEF1X, and the round trip through
new requirements is re-engineering, so the page now says so and carries
the three levels: physical documentation, the requirements a system
currently satisfies, and a business model that needs the business in
the room.

That framing is what the closing review step was already arguing for
without naming it. A model of what you have is not a model of what you
want, which is why an agent gets you to level 2 and no further.
The relationships section covered the mechanics and left the predicate
as a one-line aside, which is backwards: the keys give you cardinality
and identification, and the verb is the only part a tool cannot derive.

has many, belongs to and has one all restate something already on the
diagram. Names the replacements as different assertions rather than
synonyms, and gives the test: read it aloud in both directions, and if
a person in the business would not say it, the predicate or the
relationship is wrong. A sentence that rings true is the business case
for the entity carrying it.

Every predicate shown is taken from the reference models rather than
invented, so readers can go find it.
models/legacy appeared once inside the author-by-hand branch of step 1,
then step 7 ran commands against it as though it were established. A
reader who took the scaffold path had no such folder.

Step 1 now names it as the page's convention and says to substitute,
and notes the path can be dropped entirely since serve, export and
validate discover the model root from the current directory.
Two GitHub marks are indistinguishable, so each carries its repo name
beneath it. VPSocialLink renders its icon through v-html, which drops
any child node, so the caption is a pseudo-element keyed off the repo
path rather than markup.

Both links also get an explicit ariaLabel. The default is the icon name,
which would have announced them to a screen reader as "github" twice.
The pages stated the division of labour between the two tools but never
why it is two tools. A reader is told to install a second binary from a
different repo and is owed the reason.

It never opens a connection, it is useful with no noorm in the stack at
all, and its output is for the person approving a design rather than the
one deploying it. The separate release cadence is also why these pages
link out for reference material instead of copying it.
Named for Derek Ignatius Asirvadem, who taught the author IDEF1X. The
tribute needs the argument behind it or it reads as trivia, so the
section carries both: modeling first settles most of the design before
anyone opens an editor, and the part that does drift is cheap to change
while nothing is committed to code yet.

Also records why the tool exists at all. OmniGraffle made drawing the
bottleneck, and a picture cannot tell an LLM what the relationship
between two entities is.
The landing page argues migrations-versus-files, which is the product
case. It never said where the tool came from, and the origin makes a
different and more credible argument: noorm is the fifth attempt, and
each feature was excavated by a question the previous version could not
answer.

Also records what LLMs changed. Writing SQL stopped being the slow part,
so the tool grew safeguards, an agent skill, and eventually a planning
tool, which is why ignatius exists.
The hero eyebrow and the footer both claimed ISC. Nothing in either repo
is ISC: LICENSE, both package LICENSEs, all three package.json files,
and the README badge are MIT. The eyebrow sits above the fold, so it was
the most visible license claim on the site and the wrong one.

Footer now links to LICENSE rather than naming a licence in passing.
MIT gave users no patent grant and said nothing about trademarks, both
of which matter for a tool aimed at teams whose legal review asks, and
for a project whose name is also its brand.

Relicensing is a single commit while there is one copyright holder and
needs every contributor's consent afterwards, so it happens now.

LICENSE is the canonical text from apache.org, byte-identical across all
three copies. Copyright attribution moves to NOTICE, per the Apache
split, and NOTICE joins the published `files` because npm ships LICENSE
automatically but not NOTICE, which section 4(d) requires downstream.
@damusix damusix changed the title Next chore: docs and license Aug 2, 2026
damusix added 3 commits August 1, 2026 20:03
The CLI history ran backwards and implied the three frameworks each
failed to answer the questions below them. The order was minimist,
cmd-ts, oclif, each replacing the last for robustness and
discoverability, and all three were then dropped for citty and clack.
Ink is the missing piece: testing a TUI against the CLI is what produced
the split noorm still has, interactive on screen and automatable in the
CLI.

Questions move to a table, and the three with more behind them get
prose. Templates exist for seed data and for rendering DDL that is
miserable by hand, not for swapping hostnames. History records the
operator and the failure message, which is what makes a silent breakage
traceable, alongside changelog.md for the reason and manifests for
idempotent objects. Configs are exportable without credentials, stages
make that prescriptive, and the vault holds team secrets encrypted in
the database.
__noorm_vault__ is only the name on MySQL and SQLite. Schema migration
v2 moves every tracking table into a dedicated noorm schema on
PostgreSQL and MSSQL, so the vault is noorm.vault there.
Schema migration v2 moves every tracking table into a dedicated noorm
schema on PostgreSQL and SQL Server, and no-ops on MySQL and SQLite. The
docs still named the prefixed forms as though they were universal, so
half the supported dialects were documented wrong, including on the
teardown page where the reader is being told what survives.

Reference sections now give both forms. Passing mentions in dry-run and
history prose drop the names entirely, since the sentence was never
about where the rows live.

Two other errors found on the way: the lock table was written
__noorm_locks__ in two places and is singular, and the contributor data
model claimed five tracking tables and all-prefixed names when the vault
makes six.
damusix added 6 commits August 1, 2026 20:28
The vault page sent readers to the contributor docs to learn about
identity, which is the keypair that decides who can read a shared
secret. That is something a user has to understand before the vault
makes sense, not an implementation note.

A user-facing page already existed at /cli/identity with six inbound
links; it was just missing from the sidebar, so nobody browsing
Environments could find it. It now sits after Vault, beside the features
that depend on it.
The identity page walked through entry(), loadIdentityFromEnv() and
loadPrivateKey() in call order. A reader setting up CI needs to know the
variables are read once at startup and win over disk, both of which the
guarantees below already said. The function names told them nothing.

Contributor docs now carry the dialect rule too. Prose that names a
table gives both forms; pages whose tables appear in code samples get
one note above the samples rather than rewritten string literals, since
those are copied verbatim and the real code resolves names through
getNoormTables(dialect).
The page told contributors to reach for NOORM_TABLES. That constant is
marked deprecated in source and its own note says it "will produce
incorrect SQL when used with noormDb() on pg/mssql", because it only
ever returns the prefixed names. The replacement pair is
getNoormTables(dialect) for the name and noormDb(db, dialect) for the
schema, and neither works without the other.

The migration walkthrough had the same defect in a worse place. It
hardcoded __noorm_change__ and omitted the dialect argument that up()
and down() actually receive, so a migration written from it runs clean
on MySQL and SQLite and silently targets a nonexistent table on the
other two.

Stale numbers alongside it: the example migration was numbered v2, which
now collides with the real schema-move migration; CURRENT_VERSIONS was
printed as all ones when state is on 3; and the state example was
numbered v2 with three already written.
master carried the same seven changes as next under different SHAs, from
a cherry-pick, so git saw two independent edits to the same lines and
conflicted on four files.

All four resolve to next. Every master-only line was something removed
here on purpose: the Home nav entry, the Dev Docs top-bar entry, the
single GitHub social link, and the ISC footer.

master's one genuinely new commit was the 1.0.1 release. Its version
bumps and changelog entries merged cleanly and are preserved alongside
the Apache relicense.
Thirteen audits, one per sidebar topic, each verifying every factual
claim against the implementation and citing file:line.

Most of what turned up was ordinary rot: flags and subcommands that no
longer exist, JSON samples missing the success envelope or showing bare
arrays where the code returns named keys, exit codes predating the
0/1/2/3 split, TUI hotkeys pointing at screens that moved.

Two classes were worse. Code samples in the contributor docs had drifted
past compiling, with wrong field names, wrong arity and methods that
were removed. And several pages described safety properties the code
does not have: config export was said to strip passwords when it writes
them in plaintext, EXPLAIN was listed as viewer-safe when it is excluded
because EXPLAIN ANALYZE executes, the admin role was called frictionless
against nine confirm cells, and the config-sharing scheme was credited
with forward secrecy it cannot provide.

Also adds Project Discovery and Data Transfer to the dev sidebar, and
exempts dev/headless.md from the flag-placement check, which flagged a
deliberate counter-example the same way it already exempts two pages.
run file accepted --dry-run, threaded it through, and executed the SQL
anyway: only executeFiles checked the flag, and runFile routes through
executeSingleFile. Someone reviewing a destructive file with --dry-run
ran it. The branch now sits ahead of the tracker, matching executeFiles,
so a dry run also writes no history.

ci init stamped isTest: true on every config it created, production
included, which let the SDK's requireTest guard pass against a prod
database bootstrapped by the documented CI flow. Now opt-in via --test,
agreeing with env-only resolution, which already defaults it to false.

The transfer planner excluded tracking tables by the __noorm_ prefix
alone. Schema migration v2 moved them into a noorm schema under clean
names on postgres and mssql, where that test matches nothing, so a
transfer could copy the encrypted vault and the identity table into the
destination. Explore and teardown already exclude the schema.

transfer.plan() dropped the context channel that to() injects, so an
agent-channel context had its plan authorised as a human.

config validate and db teardown emitted success: true while exiting 1,
because toJsonEnvelope defaults success to true when the payload carries
neither success nor status. CI branching on .success read failure as a
pass.

Change dry-runs wrote rendered SQL with default permissions while the
runner path used 0700/0600, leaving resolved secrets world-readable.

Tracker, ChangeTracker and ChangeHistory defaulted dialect to sqlite
while every caller passed postgres. dialect is now required, so a direct
construction cannot silently query __noorm_*__ on a postgres connection.

Also fixes the sql repl redirect hint, which named `noorm sql --file`;
the argv rewriter cannot resolve that to `sql query`.
@damusix damusix changed the title chore: docs and license docs: information modeling, a docs audit, and the code fixes it surfaced Aug 2, 2026
damusix added 3 commits August 2, 2026 00:24
The command deliberately exports sensitive fields, and --output already
chmods 0600 because of it. The stdout path never got the same care: at
an interactive terminal it put a plaintext database password into
scrollback, onto the screen, and into any recorded session.

It now refuses a TTY and names the two safe forms. Redirects and pipes
are untouched, so `config export dev > dev.json` and every scripted use
keep working, but they warn on stderr: a shell redirect lands at the
caller's umask, so the one guarantee --output makes cannot be made there.
The existing test asserted isTest: true, which was the behaviour that
let requireTest pass against a production database. It now asserts the
default is false and that --test opts in.

CI caught this because these tests spawn the compiled dist/; a local run
without a rebuild exercises stale code and passes.
@damusix
damusix merged commit 11c7704 into master Aug 2, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant