BIDS json sidecar population with implicitly inherited information

[leej3]

I was exploring how to get AFNI to support BIDS metadata a little more cleanly. @jbteves and I discussed this a little in See: afni/afni#132

I was wondering whether pybids and its internal metadata model could be used to generate a view of the dataset (in the derivatives folder) that had all metadata propagated to each leaf. This could be bidirectional and seems like something that could be added as methods of BIDS_Layout. Something like "increase_metadata_redundancy" and "decrease_metadata_redundancy" will move metadata down to each leaf json sidecar up back up the tree as far as it can go to maximize the reduction of redundant information in the file system tree. Choosing fields for pushing up and down the tree might make the task easier (or at least not spend time/disk space on fields that are not of interest).

A related discussion regarding write the parsed metadata into a single file is also worth considering in this context: #243

[tyarkoni]

Can you give an example and/or elaborate on what you mean by redundancy? It's already the case that every BIDSFile should be associated with all of the metadata in any JSON files that apply to it via inheritance. If you're not seeing that, it's probably a bug (or maybe you need to make sure index_metadata=True in the layout init?).

(Note that pybids currently does not read any metadata from image headers, or really anything other than JSON sidecars. That's something I'd be happy to accept a PR for.)

[leej3]

every BIDSFile should be associated with all of the metadata in any JSON files that apply to it via inheritance

I didn't realize this.

The feature request would be to use that information to generate a tree on disk in the derivatives directory in which each bids json has all that inherited information. I beg forgiveness if that is sacrilege!

[tyarkoni]

BIDS-Derivatives datasets are treated exactly like ordinary BIDS datasets by pybids, so that behavior should also already be there... the only catch is that I'm not sure you can directly get the inherited values for a JSON file itself—you might need to do it through a BIDSFile the target JSON applies to. Have you played around with this to see what it's doing? I'm happy to take a look myself when I have time...

[tyarkoni]

Note that there's already a get_associations() method that should work for any file (including JSONs), so in the worst case, it wouldn't be much work to add a method to the BIDSJSONFile that returns inherited contents and not just its own contents.

EDIT: Actually a more sensible approach would be to add a inherit flag to BIDSJSONFile.get_dict().

[leej3]

Ok, I'll take a look. I'll try add an inherit flag to BIDSJSONFile.get_dict() and add a method to write a tree and see what it looks like. Thanks.