Skip to content

refactor(datagrid): migrate callers to TableRows + delete legacy stack#931

Merged
datlechin merged 31 commits intomainfrom
refactor/datagrid-phase4c2-migrate-callers
Apr 29, 2026
Merged

refactor(datagrid): migrate callers to TableRows + delete legacy stack#931
datlechin merged 31 commits intomainfrom
refactor/datagrid-phase4c2-migrate-callers

Conversation

@datlechin
Copy link
Copy Markdown
Member

Summary

Phase C.2 of the DataGrid refactor. Migrates every caller of RowBuffer / InMemoryRowProvider / RowDataStore to use TableRows / TableRowsStore / TableRowsController, then deletes the three old types. Mutations now emit Delta events and the controller drives NSTableView directly via insertRows / removeRows / reloadData(forRowIndexes:) — eliminating the SwiftUI counter-driven full reloads.

Builds on Phase C.1 (PR #930) which introduced TableRows, Row, and Delta.

Why

The old row-data system had three tightly-coupled types with inconsistent mutation paths (direct buffer writes, inout array refs, provider methods). Sort indices didn't auto-invalidate. Inserted rows lived in a parallel appendedRows array. The provider's weak ref to RowBuffer fell through to a shared empty buffer with no error. Every cell edit triggered a full SwiftUI reloadVersion bump.

After this PR:

  • Single source of truth (TableRows) for cell values and column metadata.
  • All mutations go through TableRows and return Delta.
  • The data grid coordinator applies Delta to NSTableView directly — partial reloads, surgical insert/remove animations.
  • Sort lives on the controller, keyed by RowID so it survives mutations cleanly.
  • Display cache lives on the coordinator, keyed by RowID.

Commits (12, app green at each step — read in order)

# Hash What
1 da111ec8 Introduce TableRowsStore alongside RowDataStore (12 unit tests)
2 3e98e3c9 Introduce TableRowsController (apply(_ delta:to:) mapping every Delta case to NSTableView; 11 unit tests)
3 b7bd7a4b Mirror result-delivery writes into TableRowsStore (parallel writes, no behavior change)
4a 9b62697f Switch JSON view and export to read TableRows
4b 5e1ba86e Switch sidebar reads to TableRows
4c a2f5a4ce Switch NSTableView delegate reads to TableRows
5 387dc8eb Route cell edits through tableRows.edit and Delta
6 b4c16432 RowOperationsManager mutates TableRows and returns Delta
7 45c51815 Undo replay returns Delta; add TableRows.insertInsertedRow(at:values:) for non-tail re-insertion
8 0b7ea835 Sort moves to controller keyed by Row.id
9 2d941b41 Display cache moves to coordinator keyed by Row.id
10 e190a7a2 Migrate Discard, tab switch, eviction; delete RowBuffer, InMemoryRowProvider, RowDataStore

Each commit was reviewed for the constraint that the app must continue to build at every step. The user opted out of a worktree for this PR — work was on the main repo's feature branch.

What was deleted in commit 10

  • TablePro/Models/Query/RowBuffer.swift
  • TablePro/Models/Query/RowProvider.swift (InMemoryRowProvider, TableRowData)
  • TablePro/Core/Services/Query/RowDataStore.swift
  • TablePro/Views/Results/RowProviderCache.swift
  • Six obsolete test files (RowBufferTests, RowProviderTests, DisplayValueCacheTests, RowProviderCacheTests, RowProviderSyncTests, RowDataStoreTests, TabEvictionTests). One new test added: closingTabRemovesOnlyThatEntry() in TableRowsStoreTests.

grep -rn "RowBuffer\|InMemoryRowProvider\|RowDataStore" TablePro/ TableProTests/ → empty.

Notable design decisions (locked from the planning round)

  • TableRows is a Sendable struct with ContiguousArray<Row> storage. Owns rows + columns + types + metadata.
  • Row.id is RowID.existing(Int) | .inserted(UUID) — replaces today's parallel insertedRowIndices set.
  • TableRows is not Equatable — full row equality on big tables is a footgun; DataGridIdentity keys on schemaVersion.
  • TableRows does not embed PendingChanges — composition stays at the coordinator level.
  • Mutators do not auto-update PendingChanges — caller's job (matches today's separation).
  • Sort under inserts: new rows append to the sorted view (sortedIDs). User can re-click the column header to re-sort.
  • Display cache invalidation: per-cell on cellChanged, prune-against-alive-IDs on rowsRemoved, full clear on columnsReplaced/fullReplace.

Test plan

This PR is large enough that manual verification matters more than the test suite alone. Suggested smoke tests on a local DB (tablepro_demo works in MariaDB or PostgreSQL; both have boolean columns and a 5000-row events table):

  • Open a table tab, edit a cell, save. Reload. Persisted.
  • Cell edit → undo → redo → undo. Yellow modified background clears on the final undo.
  • Add a new row (Cmd+N or button), edit cells, undo. Inserted row removed.
  • Multi-select 3 rows (mix of newly-inserted and existing), Delete, Cmd+Z. All 3 come back; inserted rows still marked inserted.
  • Sort a query result by a column, then add a new row. New row appears at the bottom of the sorted view.
  • Cell edit while sort is active. Edit lands on the correct underlying row (not the display index — this was a known gap fixed in commit 8).
  • Multi-cell paste (TSV from clipboard into a selected range). Single Cmd+Z reverts the whole paste.
  • Open multiple tabs, switch between them so eviction kicks in. Switch back, data re-fetches cleanly.
  • Multi-statement query. Switch between result sets. Pin one. Run another query. Pinned set survives.
  • Discard pending changes on a tab with a mix of edits, inserts, and deletes. All revert correctly.
  • Drag rows to TextEdit (TSV + HTML formats from PR refactor(datagrid): interaction fixes, window UndoManager, multi-cell paste #924).
  • Connect a database, open a 5000-row table (events). VoiceOver navigation works.

Verification commands

swiftlint lint --strict
xcodebuild -project TablePro.xcodeproj -scheme TablePro -configuration Debug build -skipPackagePluginValidation
xcodebuild -project TablePro.xcodeproj -scheme TablePro test -skipPackagePluginValidation

Follow-up parking lot (deferred, not in this PR)

  • TableViewCoordinator.cachedTableRows mirrors tableRowsProvider(). Two sources of truth in the coordinator. Could be reduced to one if the per-cell delegate path can afford a closure call.
  • MainEditorContentView.resolvedTableRowsForTab and resolvedTableRows(for:) overlap; consolidate.
  • sortCache (sync, small) and coordinator.querySortCache (async, large) cache the same shape with different keys; merge.
  • Tab close path: today native tabs are 1-window-1-tab so close == window close == teardown. If multi-tab-per-window returns, every tabManager.tabs.removeAll { ... } site needs to call tableRowsStore.removeTableRows(for: tabId).
  • Phase D follows: drop reloadVersion / lastIdentity / lastReapplyVersion now that mutations drive NSTableView through Delta directly.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e190a7a2f4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +572 to +573
if let rs = tab.display.activeResultSet, !rs.resultColumns.isEmpty {
provider = InMemoryRowProvider(
rowBuffer: rs.rowBuffer,
sortIndices: sortIndicesForTab(tab),
columns: rs.resultColumns,
columnDefaults: rs.columnDefaults,
columnTypes: rs.columnTypes,
columnForeignKeys: rs.columnForeignKeys,
columnEnumValues: rs.columnEnumValues,
columnNullable: rs.columnNullable
)
} else {
let buffer = coordinator.rowDataStore.buffer(for: tab.id)
provider = InMemoryRowProvider(
rowBuffer: buffer,
sortIndices: sortIndicesForTab(tab),
columns: buffer.columns,
columnDefaults: buffer.columnDefaults,
columnTypes: buffer.columnTypes,
columnForeignKeys: buffer.columnForeignKeys,
columnEnumValues: buffer.columnEnumValues,
columnNullable: buffer.columnNullable
)
return rs.tableRows
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Source active grid rows from TableRowsStore

resolvedTableRowsForTab returns activeResultSet.tableRows, but all edit/insert/delete paths now mutate tableRowsStore (via tableRowsMutator in the same view). Because TableRows is a struct, the result-set copy and store copy diverge after the first mutation, so the grid can render stale data while row deltas are applied to a different backing state; in practice this can make edits appear to revert and can desynchronize numberOfRows from inserted/removed row operations.

Useful? React with 👍 / 👎.

@datlechin datlechin changed the title refactor(datagrid): migrate callers to TableRows + delete legacy stack (Phase C.2) refactor(datagrid): migrate callers to TableRows + delete legacy stack Apr 28, 2026
Load More appended rows to the store but ResultSet.tableRows stayed
frozen at applyPhase1Result time. The data grid read through the active
ResultSet, so newly fetched rows showed empty cells.

Route every TableRows mutation through mutateActiveTableRows on
MainContentCoordinator. The helper updates the store, then writes the
new value back into the active ResultSet. Result-set switches go
through switchActiveResultSet so the store tracks the new active
snapshot. Read paths simplify to reading the store directly.
…a across reloads

Two related visual regressions:

1. Edit cell -> yellow modified background -> undo -> value reverts but
   highlight persists. applyDataUndo updated pending state but did not
   bump reloadVersion, so the visual state cache was gated and the
   stale modifiedColumns set survived. Bumping reloadVersion forces a
   rebuild on the next render.

2. FK column arrow and dropdown chevron toggle visible/hidden on each
   reload of a table tab. applyPhase1Result rebuilt TableRows from
   scratch and only populated columnDefaults / columnForeignKeys /
   columnNullable / columnEnumValues when a fresh schema fetch ran.
   When isMetadataCached returned true (because metadata was in the
   previous TableRows), no fetch ran and the new TableRows wiped the
   metadata, which then caused the next reload to refetch -- so every
   other reload had FK info and every other one didn't. Carry the
   existing metadata over when the schema fetch is skipped.
Inserting or undoing a row pegged the CPU at 100%. Each mutation went
through mutateActiveTableRows, which wrote the live TableRows back into
the active ResultSet's @observable tableRows property. That triggered a
full SwiftUI re-render of MainEditorContentView (which reads
rs.resultColumns / rs.errorMessage / etc), on top of the existing
observation triggers from changeManager.reloadVersion and
tabManager.tabs. Fast key-repeat undo or paste cascaded re-renders.

Move the per-ResultSet snapshot into a save-on-switch model:
mutateActiveTableRows now only writes the store, and
switchActiveResultSet saves the outgoing snapshot then loads the
incoming one. Edits in a pinned result set still survive switching
back, but routine inserts / undos / cell edits no longer cross the
@observable boundary on the ResultSet.

Also short-circuit the inserted-rows scan in
DataGridCoordinator.rebuildVisualStateCache: when the grid is unsorted,
read changeManager.insertedRowIndices directly instead of iterating
every row in TableRows. The full scan only runs in the sorted case
where display indices differ from storage indices.
… evicted

`evictInactiveRowData` deliberately skips the currently selected tab
("kept in memory so the user sees no refresh flicker"), but the migrated
tests added a tab and immediately called eviction — the new tab was the
selected tab, so eviction was a no-op and the assertions failed.

Fix: add a second tab so the first becomes background, then assert
eviction on the background tab.
…ing call

addNewRow / duplicateSelectedRow now call coordinator.beginEditing(displayRow:column:) synchronously after applyDelta. The editingCell SwiftUI binding plumbing across 7 files is removed since no caller sets it to non-nil anymore. Each row-add press fully completes (commit prior edit, mutate model, apply delta, focus new cell) before the next press fires, so rapid Cmd+Shift+N keeps focus on the latest appended row instead of getting trapped in queued Tasks.
setActiveTableRows now dispatches applyFullReplace to the active grid after every full row swap, routing through a single mutation surface instead of seven direct tableRowsStore.setTableRows callers across navigation, query helpers, multi-statement, FK navigation, and sidebar actions. Without the dispatch, the coordinator's RowID-keyed displayCache survived table switches and returned the previous table's formatted cell values for matching RowIDs, even though the cell views themselves had rebuilt with the new column set.
… add dispatch regression tests

The protocol now exposes commitActiveCellEdit and beginEditing alongside the row-delta methods, so its name no longer matches its scope. Renaming to TableViewCoordinating tracks the conforming class TableViewCoordinator and the field DataTabGridDelegate.tableViewCoordinator.

TableRowsMutationTests verifies that setActiveTableRows dispatches applyFullReplace exactly once for the active tab and skips background tabs, locking in the displayCache invalidation contract that was missing before commit 0e967c2.
@datlechin datlechin merged commit 9ef7f19 into main Apr 29, 2026
1 check passed
@datlechin datlechin deleted the refactor/datagrid-phase4c2-migrate-callers branch April 29, 2026 03:27
datlechin added a commit that referenced this pull request Apr 29, 2026
…vider() reads (#934)

Removes the cachedTableRows stored property on TableViewCoordinator. It mirrored what tableRowsProvider() returns and was kept in sync via four writers (initial load, updateNSView, updateCache, releaseData). The two-sources-of-truth setup was a parking-lot item from PR #931 — under it, every reader had to trust the cache had been refreshed, and forgetting to refresh produced stale-value bugs.

Each reader now captures rows once at the top of its scope:
- persistColumnLayoutToStorage
- updateCache (the only true cache update; cachedRowCount/Count still derived)
- releaseData (just drops the now-removed field)
- DataGridView+CellPaste (anchor column count check)
- DataGridView+Sort (sortDescriptorsDidChange + menuNeedsUpdate header context menu)
- DataGridView+RowActions (undoInsertRow no longer needs the explicit refresh)

Net -6 LOC, one fewer field on the coordinator, and no possibility of cache/source drift.

Smoke-tested: column header context menu, click-to-sort, cell paste, undo-insert all behave as before.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant