CS-11170: Include FileDefs in CardsGrid#4865
Open
lukemelia wants to merge 12 commits into
Open
Conversation
Contributor
Preview deploymentsHost Test Results 1 files ±0 1 suites ±0 1h 45m 12s ⏱️ +37s Results for commit 0faa65e. ± Comparison against earlier commit 75a7fea. Realm Server Test Results 1 files ±0 1 suites ±0 8m 19s ⏱️ +9s Results for commit 0faa65e. ± Comparison against earlier commit 75a7fea. |
Surfaces FileDef instances alongside CardDef instances in the CardsGrid
sidebar. Implements the six-step plan from the Linear project description.
- realm_meta.value partitions into { instances, files } via a per-type
aggregation in index-writer; normalizeRealmMetaValue tolerates the
legacy array shape during rollout.
- File rows now carry display_names from a FileDef class-chain walk in
FileDefAttributesExtractor, parallel to the card-side meta route.
- _types endpoint emits a flat list with a kind: 'instance' | 'file'
discriminator; CardsGrid partitions by kind to render an All Files
group beside All Cards (hidden when the realm has no files).
- detectStackItemTypeForTarget consults knownFileMetaUrls so files
clicked from a freshly-loaded CardsGrid resolve to file stack items
before the file-meta resource lands in the store; an Open in Code
Mode menu entry is added to the file kebab in interact context.
- Coverage: kind discriminator asserted on the existing types endpoint
test, new index-writer case for file-row partitioning, and a
normalize-realm-meta-value shared test for backward compat.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A host test run produces hundreds of KB of output that's lost if you only pipe through tail/grep. Re-running to recover detail is slow and the original failure may not reproduce. Capture once with tee, then grep the file as needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
lukemelia
force-pushed
the
cs-11170-implement-include-filedefs-in-cardsgrid
branch
from
2cba8dd to
7d1e29c
Compare
The previous implementation made `filterOptions` a TrackedArray and called setupFilterOptions() — which does splice+push — from the component constructor. The constructor runs during render, so mutating tracked state there fires Glimmer's "you attempted to update X but it had already been used in this computation" assertion the moment a parent re-renders the CardsGrid card. The crash propagates up through OperatorModeStackItem and tears down the whole interact-mode view. Switch to a @cached getter that derives the group list from `isPersonalRealm` and `fileTypeFilters.length`. The per-group wrappers (highlightFilter, allCardsFilter, allFilesFilter) are also @cached so their object identity stays stable across re-computations — FilterList's isSelected is an identity comparison and would otherwise lose the active highlight after the first render. No imperative array mutation remains, so the assertion can't fire. Removed setupFilterOptions and its call site in loadFilterList; the getter re-runs naturally when fileTypeFilters.length transitions from 0 to >0 as `_types` results stream in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CardsGrid's All Files group lists .gts/.ts file rows, but the fused visit currently skips the FileRender pass for executable modules (visit-file.ts:144), so their fitted_html / embedded_html / etc. are NULL. Without a fallback, <card.component /> renders nothing and the content area looks empty even though the search returned 137 rows. Expose `hasHtml` on PrerenderedCard (and PrerenderedCardLike) and render a minimal `<file icon + filename>` placeholder in CardList when it's false. The placeholder lives inside the existing clickable <li>, so clicking it still routes through viewCard into interact-mode (where the FileDef stack item can take over and the kebab offers "Open in Code Mode"). This is a workaround for CS-11171. The real fix is to drop the `!isModule` gate so executable modules get a FileRender pass — see that issue for the plan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bring the FileDef kebab menu in interact context closer to CardDef's so users get a consistent set of actions: - Copy as Markdown — reuses CopyCardAsMarkdownCommand, which fetches the URL with `Accept: text/markdown` and copies the response. Works the same way for files as it does for cards; what it returns depends on whether the file row's `markdown` column is populated (null today for .gts/.ts pending CS-11171). - Delete — gated on canEdit, marked dangerous. Routes through cardCrudFunctions.deleteCard which calls store.delete(id); the realm's source-delete endpoint doesn't care whether the URL points at a card JSON or another file kind. Tests cover both the new entries and the canEdit gate (Delete hidden when canEdit is false; Copy as Markdown and Open in Code Mode remain available read-only). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CardsGrid's "Copy as Markdown" on a `.md` (or any non-card file) URL was 415-ing because `getCardMarkdown` only looked the path up as an instance entry. With no instance row to match, the handler fell through to the `nonJsonFileExists` branch and returned `Unsupported Media Type` — even though the file row in `boxel_index` had a perfectly serviceable `markdown` column. Look the path up as a file entry when the instance lookup misses, and serve `fileEntry.markdown` with the same `text/markdown` content-type. Falls back to 415/404 only if neither index row exists, matching the original semantics. Test: a `.md` file seeded into the markdown-test fixture now round-trips through `GET ... Accept: text/markdown` and returns the indexed markdown body with the right content-type. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three small fixes flagged by CI:
- packages/host/app/components/prerendered-card-search.gts: move
`../lib/known-file-meta-urls` above the `../resources/...` imports
so import/order is happy. Also moves the re-export and
OWNER_DESTROYED_ERROR const to sit alongside their natural
neighbors after the import block.
- packages/host/tests/unit/file-def-menu-items-test.ts: prettier
reformat (`pnpm eslint --fix`).
- packages/realm-server/handlers/handle-federated-types.ts: the
federated handler still iterated `fetchCardTypeSummary()` as an
array — it now returns the partitioned `{ instances, files }`
RealmMetaValue shape. Iterate both arms and stamp each entry's
`kind` accordingly so the federated response carries the same
discriminator the per-realm `_types` endpoint already emits.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Surfaces FileDef summaries alongside CardDef summaries so CardsGrid can show file types, search/open file rows, and expose file-specific menu actions.
Changes:
- Partitions
realm_meta.valueinto{ instances, files }and normalizes legacy values. - Adds file summary
kinddiscrimination through_types/ federated types and CardsGrid filters. - Adds FileDef display names, file markdown support, file click detection, fallback rendering, and interact-mode file menu actions.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
packages/runtime-common/tests/normalize-realm-meta-value-test.ts |
Adds shared normalization tests. |
packages/runtime-common/realm.ts |
Reads markdown from file index rows when no card instance exists. |
packages/runtime-common/prerendered-card-search.ts |
Adds hasHtml to prerendered card shape. |
packages/runtime-common/index.ts |
Adds FileExtractResponse display names. |
packages/runtime-common/index-writer.ts |
Writes partitioned instance/file type summaries. |
packages/runtime-common/index-structure.ts |
Defines partitioned realm meta shape and normalizer. |
packages/runtime-common/index-runner/file-indexer.ts |
Persists file display names. |
packages/runtime-common/index-query-engine.ts |
Returns normalized realm meta summaries. |
packages/runtime-common/document-types.ts |
Emits flat summary docs with kind. |
packages/realm-server/tests/types-endpoint-test.ts |
Updates _types expectations for kind. |
packages/realm-server/tests/realm-endpoints/markdown-test.ts |
Adds FileDef markdown regression test. |
packages/realm-server/tests/normalize-realm-meta-value-test.ts |
Registers shared normalization tests. |
packages/realm-server/tests/index.ts |
Includes the new normalization test file. |
packages/realm-server/handlers/handle-federated-types.ts |
Federates instance and file summaries. |
packages/host/tests/unit/index-writer-test.ts |
Updates realm meta helper and adds file partition test. |
packages/host/tests/unit/file-def-menu-items-test.ts |
Tests new interact file menu items. |
packages/host/tests/helpers/realm-server-mock/routes.ts |
Updates mock federated type entry typing. |
packages/host/app/utils/file-def-attributes-extractor.ts |
Extracts FileDef display names. |
packages/host/app/services/realm-server.ts |
Parses summary kind from federated types. |
packages/host/app/lib/stack-item.ts |
Uses known file URLs when detecting stack item type. |
packages/host/app/lib/known-file-meta-urls.ts |
Adds shared file URL registry. |
packages/host/app/components/prerendered-card-search.gts |
Moves/re-exports file registry and implements hasHtml. |
packages/base/file-menu-items.ts |
Adds interact-mode FileDef actions. |
packages/base/components/card-list.gts |
Adds fallback rendering for empty prerendered HTML. |
packages/base/cards-grid.gts |
Adds All Files grouping and file type filters. |
AGENTS.md |
Adds test-output capture guidance. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Two latent bugs collided to make CardsGrid's "All Files" group appear only on whichever realm happened to hit a Postgres row-order lottery winner: 1. fetchCardTypeSummary had no ORDER BY / LIMIT on its realm_meta read. Multiple rows per realm_url exist (one per realm_version) when prune misses, and Postgres returned an arbitrary one. 2. pruneObsoleteEntries used `realm_version < this.realmVersion`, which is correct for monotonically forward-marching incremental indexing but silently wrong after a from-scratch reindex: the reindex resets the realm_version to a low number, so every older high-version row survives the predicate and lingers forever. A single from-scratch in a realm's lifetime is enough to poison future _types reads. Fix both: - index-query-engine.ts: SELECT ... ORDER BY rm.realm_version DESC LIMIT 1 so the read always returns the freshest realm_meta row, regardless of how many stale ones are still in the table. - index-writer.ts: DELETE ... WHERE realm_version != this.realmVersion, so the prune sweeps every row that isn't the one we just wrote — forward, backward, or sideways. The unique (realm_url, realm_version) constraint guarantees we never end up with two current rows. Existing realms with stale rows clean up on their next batch.done() under the new prune predicate. Until then the new SELECT picks the latest by version, so the user-visible bug is resolved immediately. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous fix used ORDER BY realm_version DESC LIMIT 1 to pick the freshest realm_meta row. That works for monotonically-marching incremental indexing, but it's exactly *wrong* after a from-scratch reindex: the fresh row gets a LOW realm_version (the reindex resets to 1 and counts up), while stale rows linger at higher numbers (e.g., good-times has v178/v127/v43/v26 from previous incremental runs and the fresh row at v2). DESC ordering then picks v178 — which is the *oldest* row by wall time, in the legacy array shape. realm_versions.current_version is the authoritative pointer for "which realm_meta row is current." JOIN against it so the read returns the matching row regardless of which version number it happens to carry. realm_versions is updated inside the same transaction as realm_meta in batch.done(), so concurrent readers under READ COMMITTED see a consistent (version, value) pair. Verified locally: good-times realm now returns the v2 object-shape row (6 files) instead of one of the stale v26+ array rows. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds tests for the recent realm_meta fixes that were going untested:
- fetchCardTypeSummary returns the realm_meta row at
realm_versions.current_version even when higher-numbered stale rows
are present (the from-scratch-reset case that broke "All Files"
before the JOIN went in).
- done() prunes realm_meta rows in any direction. Seeds high-version
stale rows, runs a batch at v1, asserts only v1 remains — the
scenario the legacy `realm_version < new` predicate silently failed.
- /_federated-types stamps kind: instance | file on entries from both
arms. Seeds a `.md` file alongside the existing card so the federated
response must include at least one of each kind.
Also fixes a transient-state bug surfaced by the user: when a realm
has been partially re-indexed across a code change (some file rows
populate display_names, others still have []), the SQL aggregation
`GROUP BY display_names->>0, types->>0` produced two summary rows
for the same code_ref. CardsGrid then rendered two sidebar entries
("Markdown" and the codeRef-name fallback "MarkdownDef") that
resolved to identical searches.
Fix: group by code_ref only, aggregate display_name with `MAX()`
(SQL MAX skips NULLs, so the populated label wins). New regression
test covers the mixed-display_names rollup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Six fixes against the Copilot review on PR #4865: 1. stack-item.ts: the unconditional `id.replace(/\.json$/, '')` was designed for CardDef ids (canonical form has no extension). For FileDef stack items the canonical URL keeps the extension — especially `.json` for JsonFileDef rows — so stripping it loses the file's identity in the store. Gate the strip on `type === 'card'`. 2. card-list.gts: the null-html fallback used to fire for any row with `hasHtml === false`, including error rows (PrerenderedCard builds a dedicated error component for those, also with empty html). Add a `shouldRenderFallback` helper that also requires `!card.isError`, so error rows still get their error UI. 3. cards-grid.gts: bare `FileDef` rows are intentionally excluded from the leaf list (they'd duplicate the All Files group's own root), but tying group visibility to `fileTypeFilters.length` meant a realm with only bare FileDef rows lost the entire group. Track `hasAnyFileSummary` separately and gate the group on that. 4. realm-server.ts: the federated types `kind` discriminator was only in the local `json` cast, not the declared `fetchCardType Summaries` return type. Callers downcast to the declared shape and lose `kind`. Add it (optional, for back-compat with legacy responses) so consumers can partition card vs file summaries. 5. index-structure.ts: comment pointed to the wrong file for the defensive reader. `normalizeRealmMetaValue` is defined directly below in the same file, not in `index-query-engine.ts`. 6. types-endpoint-test.ts: the misleading "see dedicated FileDefs test below" comment referenced a test that doesn't exist. Replace with a real assertion that the response contains at least one `kind: 'file'` entry — the realistic fixture seeds `sample.md` and `card-refs.md`, so this anchors endpoint-level coverage that would catch a regression dropping the `files` arm before makeCardTypeSummaryDoc. All 52 touched tests (37 index-writer + 7 file-def menu + 8 prerendered-card-search) still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Filter list grows by one ("All Files" group) and the type picker
total grows by two (file-type summaries) once the realm scan exposes
the FileDef partitions added in this PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linear: CS-11170
Project: Include FileDefs in CardsGrid
Summary
Surfaces FileDef instances alongside CardDef instances in the CardsGrid sidebar. Implements the six-step plan from the Linear project description. The sidebar now shows two top-level groups — All Cards and All Files — each expandable to specific subtypes. Clicking a file opens it as an isolated stack item in interact-mode; editing remains in code-mode (a new "Open in Code Mode" affordance is added to the file kebab).
What landed
realm_meta.valueis now a{ instances, files }object instead of a bare array.index-writerruns the per-type aggregation via a new#fetchTypeSummaryhelper.normalizeRealmMetaValuetolerates the legacy array shape so this is a hot rollout, no DB migration, no required reindex (the next index run upgrades each realm in place).FileExtractResponsecarriesdisplayNameswalked from the resolved FileDef class chain (newgetDisplayNameshelper inFileDefAttributesExtractor, mirroring the card-side meta route).file-indexerpersists those toboxel_index.display_names._typesendpoint —makeCardTypeSummaryDocemits a flat JSON:API list with akind: 'instance' | 'file'discriminator.FederatedCardTypeSummaryEntryand host-side typings updated to match.cards-grid.gtspartitions_typesresults bykind, buildsfileTypeFilters, and adds anallFilesFiltergroup that's hidden when the realm has no files. BaseFileDefis excluded from the leaf list to avoid duplicating the group label.knownFileMetaUrlstohost/app/lib/known-file-meta-urls.ts(back-compat re-export fromprerendered-card-search.gts).detectStackItemTypeForTargetconsults the registry as a fallback so file URLs clicked from a freshly-loaded CardsGrid resolve to'file'stack items even before the file-meta resource has landed in the store. Added an "Open in Code Mode" entry to the interact-context file kebab menu.Test plan
runtime-common,host,base,realm-serverfor touched files.realm-server/tests/types-endpoint-test.tsupdated to require thekinddiscriminator on every response entry.host/tests/unit/index-writer-test.tscase asserts file rows partition intorealm_meta.value.files, collapse duplicates, and don't bleed into the instances arm.normalize-realm-meta-value-testcovers the legacy array, partitioned object, partial shape, null/undefined, and unrecognized JSONB./_typesand verify each entry hasattributes.kind. Open CardsGrid in the realm and verify the All Files group appears, expanded clicks build a file-typed query, and clicking a result opens a file stack item with no edit button but with "Open in Code Mode" available.realm_meta.valueis still an array) and verify CardsGrid still renders All Cards correctly, no All Files group, no errors.Deferred (per project plan)
TextFileDef/JsonFileDef/MarkdownDefpolished; the rest land behind a follow-up.Screen.Recording.2026-05-18.at.6.32.25.PM.mov
🤖 Generated with Claude Code