Skip to content

Fix unnecessary quotes showing up in tsv/csv files - #28

Merged
ferponcem merged 1 commit into
mainfrom
bad_tsv
Aug 5, 2026
Merged

Fix unnecessary quotes showing up in tsv/csv files#28
ferponcem merged 1 commit into
mainfrom
bad_tsv

Conversation

@man-shu

@man-shu man-shu commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

To reproduce, try:

import ibc_api.utils as ibc
db = ibc.get_info("raw")
participants = db[db["path"] == "participants.tsv"]
downloaded_db = ibc.download_data(filtered_db)

The downloaded participants.tsv would have unnecessary "" around each row:

"participant_id	age	sex	handedness score"

This fails BIDS validation.

@man-shu
man-shu requested review from ferponcem and a lite review from Copilot August 5, 2026 20:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes incorrect CSV/TSV handling during downloads that caused TSV output to be re-serialized with unnecessary quoting (e.g., the entire header/row wrapped in ""), which breaks BIDS validation.

Changes:

  • Determine delimiter (sep) based on the source file extension (.csv vs .tsv) when reading downloaded tabular data.
  • Reuse the same delimiter when writing the local output via DataFrame.to_csv(...), eliminating the accidental “single-column TSV” re-serialization that triggered quoting.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@ferponcem
ferponcem merged commit a2bf648 into main Aug 5, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants