Skip to content

Recertify DATA_MODEL.md against the live EF model (#1470) - #1629

Merged
Chris0Jeky merged 4 commits into
mainfrom
issue-1470/data-model-recert
Aug 12, 2026
Merged

Recertify DATA_MODEL.md against the live EF model (#1470)#1629
Chris0Jeky merged 4 commits into
mainfrom
issue-1470/data-model-recert

Conversation

@Chris0Jeky

Copy link
Copy Markdown
Owner

Problem

docs/architecture/DATA_MODEL.md had drifted from the live EF model, and the snapshot figure its mirrors advertised for it was stale.

Two things had changed since #1470 was filed. The two columns named in the issue title — AutomationProposal.ApprovedRevisionId and DeferredUntil — were already added by a later pass (the file's Last Verified: 2026-08-01 line). What remained was the recertification the issue actually scoped: the document carried its own boundary note admitting it was "not yet a complete inventory of all 51 live DbSet mappings" and named #1470 as the owner. It was documenting 41 of 51 mapped entities.

What was reconciled

Ground truth: backend/src/Taskdeck.Infrastructure/Persistence/TaskdeckDbContext.cs (the DbSet roster), backend/src/Taskdeck.Domain/Entities/*.cs (validation rules), and backend/src/Taskdeck.Infrastructure/Migrations/TaskdeckDbContextModelSnapshot.cs (persisted columns, widths, indexes, delete behavior).

Entities added — 10 mapped DbSets had no block at all

Entity Why it matters
ProposalRevision The revision chain ApprovedRevisionId pins. Undocumented while the column pointing at it was documented.
ProposalOutcome Content-free decision records; SourceType/RiskLevel are stored as strings, not the enums of the same name on AutomationProposal.
ProposalFeedback Has no free-text field by construction (the no-PII invariant); at most one row per (proposal, user).
ProposalProvenance Provenance chain head, one per proposal.
ProvenanceField ExtractiveQuote is required when Kind = Extractive and must be null otherwise.
ProvenanceEvidenceLink Source pointers. Distinct from the unmapped domain class EvidenceLink.
ConnectorCredential Encrypted credential storage — a security-relevant table absent from the canonical data-model doc.
DailySnapshot Day-sealing state.
TomorrowNote Next-day note.
McpToolHash Per-user MCP tool approval; lives in Taskdeck.Domain.Agents, not .Entities.

The Mermaid ERD and the Relationship Summary were extended with the corresponding edges (proposal-lifecycle cascades, the two ConnectorCredential FKs, and the logical per-user links).

Column-level drift corrected in existing blocks

  • LlmUsageRecord was missing two live columns. Status (Reserved/Committed) and ExpiresAt are what the atomic quota-reservation flow turns on — a reservation only counts toward quota while ExpiresAt > now. Its index list was absent too.
  • Card.Description said "Max 2000 chars". The column is 4000; 2000 is the domain rule. Now states both, matching the shape Board.Description already used ("Max 500 chars (DB); domain allows 1000").
  • The domain-only claim was wrong by six. The doc said AbuseActor and AbuseEvent are the unmapped Entity subclasses. There are eight: also EvidenceLink, IntentCandidate, IntentEnvelopeV1, SourceBlock, SourceSpan, TaskdeckProposalBatch.
  • "UpdatedAt used as optimistic concurrency token on key entities" was unfalsifiable. It now names the exact seven that configure IsConcurrencyToken; on every other entity UpdatedAt enforces nothing — a materially different fact for anyone reasoning about lost updates.
  • ApiKey gained the real DB constraints (KeyHash max 64 unique; KeyPrefix_ persists as column KeyPrefix, max 10) — the property/column rename was invisible before.
  • Card.BlockReason, ConnectorEvent, AutomationProposalOperation, AgentRunEvent gained missing width/index rows.
  • ERD legend now says the diagram maps relationships, so OAuthAuthCode, RegistrationBootstrap, and RegistrationInvite correctly have no line — previously their absence looked like an omission.

Count-figure decision

Dropped from all three mirrors; not reintroduced as a number anywhere.

Per the issue's own recommendation, the three dated ledger entries keep their as-delivered date and PR (#875/#917, 2026-04-22) but their shape claim is replaced by a pointer to the living document — docs/STATUS.md:407, docs/AUDIT.md:258, docs/IMPLEMENTATION_MASTERPLAN.md:1690. (The masterplan line has drifted from the :1623 recorded in the issue; AUDIT.md:258 is still exact.)

Instead of a refreshed count, DATA_MODEL.md now carries a self-checkable invariant: every DbSet on TaskdeckDbContext has a ### block, and the only non-DbSet blocks are the domain-only entities named under the ERD — with the comm one-liner that prints nothing when the doc is current. A count is a fact that rots silently; an invariant plus its check command does not.

Verification

node scripts/check-docs-governance.mjsDocs governance check passed. (exit 0)

Coverage invariant verified mechanically, not by eye — extracting both sets and diffing them:

DbSets: 51
### headings: 53
MAPPED BUT UNDOCUMENTED: []
HEADINGS NOT A DbSet: ['AbuseActor', 'AbuseEvent']
DUPLICATE HEADINGS: []

Spot-checks, doc line vs. source line:

  1. LlmUsageRecord.Status / .ExpiresAtDATA_MODEL.md:787-788LlmUsageRecord.cs:24 (Status, default Committed) and :31 (ExpiresAt), persisted at TaskdeckDbContextModelSnapshot.cs:1446 and :1427; index (Status, ExpiresAt) at snapshot :1464.
  2. Card.Description 4000 vs 2000DATA_MODEL.md:194 ⟵ column HasMaxLength(4000) at snapshot :681, domain throw at Card.cs:80-81 ("cannot exceed 2000 characters").
  3. ProvenanceField.ExtractiveQuote conditional requirementDATA_MODEL.md:441ProvenanceField.cs:63-68 (required for Extractive, rejected otherwise, max 2000); column at snapshot :2024.
  4. ApiKey.KeyPrefix_ → column KeyPrefixDATA_MODEL.md:570HasColumnName("KeyPrefix") with HasMaxLength(10) at snapshot :237-241; property at ApiKey.cs:25.
  5. Concurrency tokens are exactly sevenDATA_MODEL.md Persistence Notes ⟵ every IsConcurrencyToken() in the snapshot: :513 (AutomationProposal), :1144 (DailySnapshot), :1812 (ProposalFeedback), :1875 (ProposalOutcome), :1916 (ProposalProvenance), :2003 (ProvenanceEvidenceLink), :2040 (ProvenanceField).

Indexes and delete behaviors for every entity touched were taken from the snapshot's HasIndex/OnDelete calls, including the two ConnectorCredential cascades (snapshot :2538-2551) and the (ConnectorId, UserId) unique index (:1092).

NOT verified: no build or test run — this is a documentation-only change with no executable surface. The doc is reconciled against the EF model snapshot rather than a live .db schema; the snapshot is the generated source of truth for migrations, so a hand-edited database could still differ.

Residual risk

  • docs/STATUS.md:84 and docs/IMPLEMENTATION_MASTERPLAN.md:84 describe DATA_MODEL.md omits ApprovedRevisionId and DeferredUntil, and its snapshot figure is stale #1470 as an open residual and restate the stale figure. Both are out of this PR's ownership (a coordinator pass owns that content) and were deliberately left alone; they go stale when this merges and should be swept with the other residual-list updates.
  • The added blocks document constraints as they exist today; nothing enforces that a future migration updates this file. The coverage invariant catches a missing entity, not a changed column width — a CI check for that would be a separate, larger piece of work.

Closes #1470

Reconciled every entity block against TaskdeckDbContext, the domain entities,
and TaskdeckDbContextModelSnapshot.cs. Ten mapped DbSets had no block at all:
ProposalRevision, ProposalOutcome, ProposalFeedback, ProposalProvenance,
ProvenanceField, ProvenanceEvidenceLink, ConnectorCredential, DailySnapshot,
TomorrowNote, and McpToolHash. All 51 mapped entities are now documented, and
the ERD and Relationship Summary cover the new edges.

Column-level drift corrected:
- LlmUsageRecord was missing Status and ExpiresAt, the two columns the quota
  reservation flow turns on.
- Card.Description claimed max 2000; the column is 4000 and 2000 is the domain
  rule. Both are now stated, matching how Board.Description already reads.
- The domain-only note claimed two unmapped entities; there are eight.
- "UpdatedAt is a concurrency token on key entities" now names the exact seven
  that configure it, since it enforces nothing on the rest.

Replaced the hand-maintained "855 lines, 37 entities" figure in its three
historical delivery ledgers with a pointer to the living document, keeping the
as-delivered date and PR. DATA_MODEL.md instead carries a self-checkable
coverage invariant with the command to re-derive it.

Verified: node scripts/check-docs-governance.mjs passes.

Closes #1470

Signed-off-by: Chris0Jeky <jeky.tck@gmail.com>
@Chris0Jeky

Copy link
Copy Markdown
Owner Author

Worktree teardown note: git status --porcelain --ignored in the worktree lists exactly one gitignored file, .claude/settings.local.json, and it is byte-identical to the main checkout's copy (harness-generated permissions stub). Nothing needed to be copied out before removal — no artifact from this branch lives outside the commit.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7b5666525e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/STATUS.md
Comment thread docs/architecture/DATA_MODEL.md
Comment thread docs/architecture/DATA_MODEL.md Outdated
Comment thread docs/architecture/DATA_MODEL.md Outdated
Comment thread docs/architecture/DATA_MODEL.md Outdated
Comment thread docs/architecture/DATA_MODEL.md Outdated
Comment thread docs/architecture/DATA_MODEL.md Outdated
All seven were checked against the code before being applied; all seven
confirmed, none declined.

- STATUS.md:84 and IMPLEMENTATION_MASTERPLAN.md:84 still tracked #1470 as
  open ("DATA_MODEL omits ApprovedRevisionId/DeferredUntil, 855 lines/37
  entities is stale") while this same PR closes it. Both residual clauses
  now record the recertification instead. Masterplan Last Updated bumped.
- Added the missing User -> CardCommentMention FK edge to the ERD and the
  Relationship Summary. CardCommentMentionConfiguration.cs:39-42 and the
  model snapshot both define MentionedUserId as a cascading FK to User, so
  the summary's "lists every FK in the model snapshot" claim was false.
  An audit of all 49 snapshot FK edges found this to be the only gap.
  Delete behavior also added to the entity's own field table.
- The coverage self-check was one-directional: a lone `comm -23` cannot see
  an unexpected extra heading or a duplicate one, so it could not prove the
  invariant it was printed under. Now three checks, and the expected output
  of each is stated.
- Rewrote that recipe with `rg` per AGENTS.md:28 (native ripgrep for repo
  search), replacing two `grep -o` calls.
- Dropped "atomic" from the quota-reservation description. STATUS records
  the check-then-insert as an open over-admission race deferred to #1435,
  and all four LlmQuotaReservationConcurrencyTests carry
  Skip = "Reservation atomicity TOCTOU deferred to #1435". Replaced with an
  explicit "Not atomic" note that separates settlement (closed by #1427)
  from admission (open).
- Documented ConnectorCredential.AuthMethod as string-backed:
  ConnectorCredentialConfiguration.cs:21-24 applies HasConversion<string>()
  with HasMaxLength(50) and the snapshot stores TEXT. Added it to the
  enum-storage exceptions, which are now enumerated exhaustively (three
  HasConversion<string>() calls exist in Configurations/, no more).
- Corrected "last-specific-wins" to first-specific-wins.
  ProposalFeedbackService.cs guards the rewrite with
  `existing.Reason == Unspecified`, so Unspecified -> Irrelevant ->
  TooRisky ends at Irrelevant. Documented as a one-time upgrade out of
  Unspecified with that exact sequence as a worked table.

Verified: node scripts/check-docs-governance.mjs green; the newly
documented three-check recipe was executed against the edited document and
produced exactly the stated output (51 mapped entities, 53 headings, checks
1 and 3 empty, check 2 exactly AbuseActor and AbuseEvent).

Refs #1470

Signed-off-by: Chris0Jeky <jeky.tck@gmail.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6bdc12332e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/architecture/DATA_MODEL.md Outdated
Comment thread docs/architecture/DATA_MODEL.md Outdated
Comment thread docs/architecture/DATA_MODEL.md Outdated
Comment thread docs/architecture/DATA_MODEL.md Outdated
All four confirmed against code before applying:

- ProposalProvenance is at-most-one, not "Exactly one". AutomationProposalService
  writes it only when the optional IProposalProvenanceRepository was injected, the
  repository returns a nullable, and 20260425232031_AddProposalProvenanceForeignKey
  added the FK without a backfill. Prose now matches the ERD and Relationship Summary.
- Ground-truth glob broadened to Domain/**: McpToolHash is at Domain/Agents/McpToolHash.cs,
  the only mapped entity outside Domain/Entities (measured across all 51 DbSet names).
- User -> OAuthAuthCode added as a logical edge (ERD, legend, Relationship Summary,
  persistence note, field row). UserId is required in both flows and the link-flow
  exchange compares it to the caller; there is simply no DB FK.
- TomorrowNote documented as a same-day key: Paper Today saves and fetches under the
  current dossier date, so no X -> X+1 shift exists. The "tomorrow" framing is recorded
  as intent that the UI copy and the controller XML doc still assert.

Signed-off-by: Chris0Jeky <jeky.tck@gmail.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 50b8e5496f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/architecture/DATA_MODEL.md Outdated
Comment thread docs/architecture/DATA_MODEL.md Outdated
Comment thread docs/architecture/DATA_MODEL.md Outdated
Comment thread docs/architecture/DATA_MODEL.md Outdated
Comment thread docs/architecture/DATA_MODEL.md Outdated
Comment thread docs/architecture/DATA_MODEL.md Outdated
Comment thread docs/architecture/DATA_MODEL.md Outdated
…#1629 round 3)

Codex round-3 P2s: qualify prose that claimed guarantees the code does not enforce.

- ApprovedRevisionId: add a distinct logical (no-FK) edge row for the approved
  revision pin, opposite direction to the ProposalRevision.ProposalId cascade FK;
  MaterializeEffectiveProposalAsync reads it to pick what Apply executes.
- UpdatedAt: qualify "every other entity" to "every other entity that maps
  UpdatedAt"; name the ArtefactBlob/CardLabel/RegistrationBootstrap exceptions.
- OAuthAuthCode: narrow the caller-identity check to the authenticated link flow;
  the anonymous GitHub/OIDC login exchanges run no caller check.
- McpToolHash: describe the stored/intended re-approval gate; no MCP execution
  path calls RecordToolDefinitionAsync/IsToolApprovedAsync (runtime tracked #1154).
- Card.BlockReason: 500 is a model width only; Card.Block rejects blank only.
- ProposalOutcome: content-free is a caller convention, not an enforced guarantee
  (SourceType/RiskLevel/ModelId are free-form strings, nonblank+length only).
- LlmUsageRecord: Provider(100)/Model(200) are model widths, not runtime-enforced.

Signed-off-by: Chris0Jeky <jeky.tck@gmail.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6ea397d227

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/STATUS.md
Comment thread docs/architecture/DATA_MODEL.md
Comment thread docs/architecture/DATA_MODEL.md
Comment thread docs/architecture/DATA_MODEL.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

DATA_MODEL.md omits ApprovedRevisionId and DeferredUntil, and its snapshot figure is stale

1 participant