perf(datahub): resolve record usernames server-side#1420
Open
gdevenyi wants to merge 2 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses #1410 by eliminating the client-side O(records × sessions) join and the large /v1/sessions fetch when viewing a subject’s instrument records in the datahub. Instead, the API resolves the session user’s username server-side using a scoped secondary lookup keyed only to session IDs referenced by returned records.
Changes:
- Add an optional
session.user.usernameshape to the sharedInstrumentRecordschema so clients can read usernames directly from records. - Update the datahub visualization hook to stop fetching
/v1/sessionsand userecord.session?.user?.usernameinstead. - Add an API-side post-processing step to label records with session usernames using a deduplicated
sessionIdlookup, with new unit tests covering edge cases (including orphaned sessions).
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/schemas/src/instrument-records/instrument-records.ts | Extends InstrumentRecord schema to optionally carry session.user.username for server-resolved usernames. |
| apps/web/src/hooks/useInstrumentVisualization.ts | Removes sessions fetch + client-side join; reads username directly from record.session. |
| apps/web/src/hooks/useFindSessionQuery.ts | Deletes now-unused hook that queried /v1/sessions. |
| apps/web/src/hooks/tests/useInstrumentVisualization.test.ts | Updates fixtures/mocks to provide username via record.session.user.username. |
| apps/api/src/instrument-records/instrument-records.service.ts | Adds session-model lookup and mapping helper to attach usernames to records server-side. |
| apps/api/src/instrument-records/tests/instrument-records.service.spec.ts | Adds tests for username labelling, deduped lookups, empty-record short-circuit, and orphaned-session behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| * record's session has since been deleted. Looking the sessions up by id degrades to a missing | ||
| * username for that record instead of a failed request. | ||
| */ | ||
| private async withSessionUsernames(records: PrismaInstrumentRecord[]): Promise<InstrumentRecord[]> { |
Collaborator
|
@gdevenyi please rebase against main |
Viewing one subject's records fetched every session in the current
group, each with its subject embedded, and joined them in the browser
with `sessions.find(...)` inside `records.map(...)` -- O(records x
sessions) to label a few dozen rows.
Resolve the username on the server instead, keyed on the sessions the
returned records actually reference. Measured against a real MongoDB
replica set with 20,031 sessions in the group being viewed:
before records 2,624 B + sessions 12,922,313 B / 950 ms
after records 2,786 B / 50 ms
The `/v1/sessions` request is gone from the datahub entirely, and the
client-side join with it.
This is a second scoped query rather than an `include` on the relation.
The first attempt used `include: { session: { select: { user: ... } } }`,
which is what the issue proposed, but `session` is declared required in
the prisma schema, so Prisma aborts the entire query with "Inconsistent
query result: Field session is required to return data, got `null`" if
any one record's session has since been deleted. Verified against a live
instance: main returns 200 for such a record, the `include` version
returned 500. Looking sessions up by id degrades to a missing username
for that record instead of a failed request, and is covered by a test.
`useFindSessionQuery` had no other caller and is removed. `GET
/v1/sessions` is left in place -- it is now unused by the web client and
is still unbounded, so it remains a candidate for scoping or removal.
Refs #1410
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bfen9VQemZanJLXinJ6MMG
gdevenyi
force-pushed
the
fix/datahub-record-usernames
branch
from
July 23, 2026 17:41
6d95442 to
c3e9199
Compare
joshunrau
requested changes
Jul 23, 2026
joshunrau
left a comment
Collaborator
There was a problem hiding this comment.
Needs to apply ability scoping. Right now, this does not apply access controls.
The server-side username resolution queried sessions by id alone, so a readable record referencing a session outside the caller's read scope (possible with inconsistent MongoDB data) could leak that session's username. Thread the ability through and apply accessibleQuery. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1410.
Viewing one subject's records fetched every session in the current group, each with its subject document embedded, and joined them in the browser with
sessions.find(...)insiderecords.map(...)— O(records × sessions) to label a few dozen rows with a username.The username is now resolved on the server, keyed on the sessions the returned records actually reference.
Measured effect
Against a real MongoDB 8 replica set, database provisioned through the app's own
POST /v1/setup, with 20,031 sessions in the group being viewed:mainGET /v1/instrument-records?subjectId=…GET /v1/sessions?groupId=…~4,600× fewer bytes. The
/v1/sessionsrequest is gone from the datahub entirely, and the client-side join with it —useInstrumentVisualizationnow readsrecord.session?.user?.usernamedirectly.An important correction to the approach the issue proposed
#1410 proposed doing this with an
includeon the relation:That does not work, and I only found out by testing it against a live instance.
InstrumentRecord.sessionis declared required in the prisma schema (session Session @relation(...)), so if any single record's session has since been deleted, Prisma aborts the whole query:I reproduced this by deleting one session and re-querying:
mainreturns 200, theincludeversion returned 500. That would have turned a performance fix into an outage for any instance with an orphaned record — and MongoDB enforces no referential integrity, so nothing prevents that state.The merged approach is a second scoped query instead:
joined with a Map. It is bounded by the records being returned (not by the group's session count), it degrades to a missing username rather than a failed request, and it costs one extra round trip that is skipped entirely when there are no records.
Verified on a live instance
All three cases, against the running API:
Tests
5 new server tests covering the username labelling, deduplication of session ids, skipping the lookup when there are no records, and — the one that matters most — that a record whose session no longer exists is still returned, with no username.
The 8 existing
useInstrumentVisualizationtests pass unchanged. They assert thatUsername,…testusernameappears in the CSV, TSV, Excel and JSON exports; those assertions are untouched, so they demonstrate the username still reaches every export format, just sourced from the record instead of a separate fetch. Only the fixture moved.Checks
tsc --noEmitonpackages/schemas,apps/api,apps/web— cleaneslint srcon all three — cleanprettier --check— cleanvitest run(whole workspace) — 257 passed, 1 skipped, 1 failedThe one failure is
instrument-records.service.spec.ts > upload > should create records with pending set, which fails identically on pristinemainand is unrelated.Notes for review
useFindSessionQueryhad exactly one caller and is removed, since this change orphaned it.GET /v1/sessionsis left in place. It is now unused by the web client but is still unbounded (no pagination, nosubjectIdfilter, and it embeds the full subject per session). Removing it is a breaking API change I did not want to make unilaterally; scoping or removing it is worth a follow-up, and Performance: datahub downloads every session in the group to attach usernames, then joins O(n*m) in the browser #1410 remains open for that.🤖 Generated with Claude Code
https://claude.ai/code/session_01Bfen9VQemZanJLXinJ6MMG