Working POC: Series-Level DICOM Album Pipeline with Query Reproducibility

[rishav-singh15]

Hi @pradeeban and @anbhimi,

I've implemented a working pipeline to better understand the album workflow.

Repo: https://github.com/rishav-singh15/diomede-albums

Current features:

• Metadata extraction using pydicom (PatientID, StudyInstanceUID, SeriesInstanceUID, Modality, StudyDate)
• Series-level metadata store in SQLite
• Query engine filtering by modality, patient ID, and date range
• Album creation using series-level grouping
• Albums store the query definition used to generate them (for reproducibility)
• JSON export structured for Kheops API compatibility

The pipeline is designed to sit downstream of Niffler's meta-extraction, treating Niffler's output as the ingestion source rather than reimplementing extraction from scratch.

Tested using pydicom sample datasets.

Two questions on next steps:

1. For Kheops integration, should the utility authenticate per-user (researcher-level tokens) or use a single service account for the local deployment context?
2. I've structured the export as a two-tier approach; manifest first, then direct API push. Does this staged rollout align with the expected 90-hour scope, or should direct API integration be prioritized?

Would appreciate any feedback on whether this direction aligns.

[pradeeban]

1. Per-user sounds good as it allows sharing of albums, which is the intended goal.
2. Sounds good.

[rishav-singh15]

Hi @pradeeban and @anbhimi,

Quick update, I’ve opened a PR adding a DICOM ingestion validator focused on album creation workflows.

It validates the presence and consistency of series-level metadata before files enter the album pipeline, and flags incomplete or corrupted inputs instead of silently passing them through.

PR: #75

Moving on to the proposal now.

[rishav-singh15]

Also opened PR #78 adding a configurable DICOM metadata validator — tag criticality is user-defined rather than hardcoded, so researchers can specify which tags are critical for their specific workflow (e.g. anonymized datasets can exclude PatientID from critical tags). PR :- #78