Skip to content

bulk-submit: multi-file manifests collide in bulk_entry_results — the PK has no file discriminator, so every file after the first fails at its first line #457

Description

@angela-helios

Found during the #448 manual sanity pass (SQLite, main @ 2026-07-30), submitting a Bulk Export Manifest with one NDJSON file per resource type (20 files, 11,286 resources from 17 Synthea patients).

What happens

The first file ingests fine (Patient, 17/17). Every subsequent file dies wholesale at its first entry:

failed to ingest file http://.../ImagingStudy.ndjson: internal error in sqlite:
Failed to store entry result: UNIQUE constraint failed:
bulk_entry_results.tenant_id, .submitter, .submission_id, .manifest_id, .line_number

Why

bulk_entry_results's primary key is (tenant_id, submitter, submission_id, manifest_id, line_number) — no file/output discriminator — so line 1 of the second file collides with line 1 of the first. Both the SQLite and Postgres schemas carry the same key shape.

A single-file manifest never trips it, which is presumably how it survived: the smoke tests submit one file.

Fix sketch

Add the output file (url or index) to the table, the PK, the insert, and the FK-dependent aggregation queries (countSeverity, status output/outcome grouping already carry per-file manifestUrl — they need the per-file identity too), plus a migration for both backends.

Part of the #448 sanity pass. The remaining count checks after the failure: Patient 27 ✓ (10 transaction + 17 bulk), other bulk types partially or not ingested.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions