Skip to content

Refactor scan manifest semantics to canonical scans.tsv with long-form entity columns and participant metadata flow#12

Merged
akhanf merged 13 commits into
mainfrom
copilot/refactor-naming-metadata-inputs
Jul 10, 2026
Merged

Refactor scan manifest semantics to canonical scans.tsv with long-form entity columns and participant metadata flow#12
akhanf merged 13 commits into
mainfrom
copilot/refactor-naming-metadata-inputs

Conversation

Copilot AI commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

SPIMpack used datasets.tsv for scan-level rows, which conflicted with BIDS “dataset” semantics and overloaded metadata responsibilities. This PR makes scans.tsv the canonical scan manifest, removes deprecated scan-manifest aliases in alpha, and introduces participant-level metadata ingestion/output via participants.tsv.

  • Canonical scan manifest (alpha simplification)

    • Uses scans.tsv as the canonical scan manifest file.
    • Removes deprecated datasets_tsv / datasets.tsv compatibility handling.
    • Resolves scan table paths via manifest-relative path checks.
  • Removal of deprecated scan-level fields/aliases

    • Removes scan-level compatibility traces for deprecated names like dataset_id.
    • Keeps scan processing stable via row iteration without legacy dataset-id assumptions.
  • Participant-level metadata support

    • Adds optional participants.tsv ingestion with required participant_id.
    • Uses subject as the canonical participant linkage in scans.tsv when participants are supplied.
    • Validates that scan subjects are present in participants.tsv.
    • Extends manifest model to carry participant records and metadata.
  • Long-form scan TSV entity headers for readability

    • Uses long-form column headers in scans.tsv: subject, session, sample, acquisition.
    • Keeps short forms internal for BIDS filename construction (sub, ses, acq tokens in paths).
  • BIDS output alignment

    • Generates top-level BIDS participants.tsv from participant metadata when provided.
    • Preserves existing scan packaging behavior and sidecar generation.
    • Keeps scan-specific metadata in scan flow while participant attributes remain participant-scoped.
  • UI + docs migration updates

    • Renames UI defaults/copy from datasets to scans (scans.tsv, scans_tsv).
    • Updates UI-generated scan rows and labels to use long-form entity names.
    • Updates README schema/examples to reflect canonical scans.tsv and long-form scan headers.
# manifest.yml (canonical form)
dataset_description:
  Name: Demo
  BIDSVersion: 1.9.0
  DatasetType: raw
  License: CC0
scans_tsv: scans.tsv
participants_tsv: participants.tsv  # optional

Copilot AI changed the title [WIP] Refactor naming and metadata inputs for BIDS compliance Refactor manifest semantics to scans.tsv, deprecate dataset_id, and add participant-level metadata flow Jul 9, 2026
Copilot finished work on behalf of akhanf July 9, 2026 20:26
Copilot AI requested a review from akhanf July 9, 2026 20:26
@akhanf

akhanf commented Jul 9, 2026

Copy link
Copy Markdown
Member

@copilot: the subject column and participant_id column now duplicate each other.

Copilot AI commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

@copilot: the subject column and participant_id column now duplicate each other.

Addressed in c94d42c (behavior/UI/tests) and 28e982a (README alignment). sub is now the canonical participant linkage in scans, and legacy participant_id is optional compatibility-only.

Copilot finished work on behalf of akhanf July 9, 2026 21:52
@akhanf

akhanf commented Jul 10, 2026

Copy link
Copy Markdown
Member

@copilot: don't worry about keeping backwards compatibility for deprecated variables/names, this code is still alpha and can have major api changes.. you can just remove any trace of things like dataset_id. also would be good to replace the shorthand names for the column headers (eg sub -> subject, ses -> session, acq -> acquisition), keeping track of the short and long form names so the short ones can still be used when constructing filenames, but long ones used for column headers to improve readability.

Copilot AI commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

@copilot: don't worry about keeping backwards compatibility for deprecated variables/names, this code is still alpha and can have major api changes.. you can just remove any trace of things like datas...

Done in af4b20e (+ follow-up fixes 84f6ffe, 3b6990d, 44891df). Deprecated scan aliases/compatibility paths were removed, and scans headers now use long names (subject, session, sample, acquisition) while short forms remain internal for BIDS filename construction.

Copilot AI changed the title Refactor manifest semantics to scans.tsv, deprecate dataset_id, and add participant-level metadata flow Refactor scan manifest semantics to canonical scans.tsv with long-form entity columns and participant metadata flow Jul 10, 2026
@akhanf akhanf marked this pull request as ready for review July 10, 2026 12:26
@akhanf akhanf merged commit 256f13d into main Jul 10, 2026
@akhanf akhanf deleted the copilot/refactor-naming-metadata-inputs branch July 10, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants