Various errors with upload-bids

[jeffrey-phillips]

I'm trying to identify and fix metadata problems with our legacy data. When trying to upload changes using upload-bids, I get KeyErrors because the "session.label" and "subject.label" fields are not in my csv file. Example error follows.

If I manually add subject.label and session.label to both the original and modified csv files, I then get an exception regarding the info_BIDS_valid column--see the second traceback below. Not sure if this is a problem with my csv files or a code issue, but I'm attaching my modified csv file as a spreadsheet--feedback would be appreciated.

Thanks,

Jeff

##############################

Traceback #1:

$ upload-bids -orig BOLD_20190619.csv -mod BOLD_20190619_mod.csv
Traceback (most recent call last):
File "/usr/local/bin/upload-bids", line 11, in
load_entry_point('flywheel-bids-tools==0.0.11', 'console_scripts', 'upload-bids')()
File "/usr/local/lib/python2.7/site-packages/flywheel_bids_tools/upload_bids.py", line 284, in main
df_original = read_flywheel_csv(args.original)
File "/usr/local/lib/python2.7/site-packages/flywheel_bids_tools/utils.py", line 41, in read_flywheel_csv
df[col] = df[col].astype(str)
File "/usr/local/lib/python2.7/site-packages/pandas/core/frame.py", line 2927, in getitem
indexer = self.columns.get_loc(key)
File "/usr/local/lib/python2.7/site-packages/pandas/core/indexes/base.py", line 2659, in get_loc
return self._engine.get_loc(self._maybe_cast_indexer(key))
File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1601, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1608, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'session.label'

##############################

Traceback #2:

$ upload-bids -orig BOLD_20190619.csv -mod BOLD_20190619_mod.csv
Traceback (most recent call last):
File "/usr/local/bin/upload-bids", line 11, in
load_entry_point('flywheel-bids-tools==0.0.11', 'console_scripts', 'upload-bids')()
File "/usr/local/lib/python2.7/site-packages/flywheel_bids_tools/upload_bids.py", line 291, in main
res = validate_on_unequal_cells(unequal, df_modified)
File "/usr/local/lib/python2.7/site-packages/flywheel_bids_tools/upload_bids.py", line 165, in validate_on_unequal_cells
changed_df.columns[pair[1]]
File "/usr/local/lib/python2.7/site-packages/flywheel_bids_tools/upload_bids.py", line 147, in change_checker
raise Exception("Column {0} not recognised!".format(column))
Exception: Column info_BIDS_valid not recognised!

BOLD_20190619_mod.xlsx

[TinasheMTapera]

Hi @jeffrey-phillips , I've added a commit that will hopefully fix these issues for you. The downside is that for columns that come out in the CSV as nested objects, users can only modify the data using that same nomenclature. For example:

acquisition.label	classification_Intent	classification_Measurement
foo	['Functional']	''
bar	['Fieldmap']	['B0']

When editing classification_Intent or classification_Measurement, the tool will not allow you to upload something like:

acquisition.label	classification_Intent	classification_Measurement
foo	['Functional']	'B0'
bar	['Fieldmap']	['B0']

The user must supply a list in the same form as was output.

Please update flywheel-bids-tools from pip with:

pip install flywheel-bids-tools -U