Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] BEP 003: Common Derivatives #265

Open
wants to merge 174 commits into
base: master
from
Open

[ENH] BEP 003: Common Derivatives #265

wants to merge 174 commits into from

Conversation

@effigies
Copy link
Collaborator

effigies commented Jul 9, 2019

This PR is the first to replace #207, as discussed in #254.

In this PR, we will focus on the (mostly) modality-agnostic components of the derivatives specification, and the necessary changes to the broader spec. This should hopefully keep the scope to a level that several people can read it all the way through, with a particular lookout for:

  1. Backwards-incompatible changes
  2. Excessive restriction that will make it difficult for apps to produce valid datasets with as-yet-unspecified data types (related to discussion in #264)
  3. Inconsistencies with how the modality-specific sub-BEPs have been treating an issue

There are a lot of conversations on #207. If any of them are relevant to the common derivatives, we should link to the old conversation, ideally with a summary of the state of discussion.

The branch is being rendered on https://bids-specification.readthedocs.io/en/common-derivatives/.


9/18 State of the BEP (by: @effigies)

I consider the following PRs essential to the completion of this BEP:

The following PR into master is a pre-condition for merging this BEP:

The following issues are worth considering in this context:

8/22 State of the BEP (by: @franklin-feingold)

Remaining PRs to meet at a consensus. This will enable the finalization of merging the common derivatives extensions.
#300
#301 (offshoot discussion in #309)
#306
#307
#310

chrisgorgo and others added 30 commits Dec 16, 2018
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
…into enh/derivatives
…into enh/derivatives
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
Co-Authored-By: chrisfilo <krzysztof.gorgolewski@gmail.com>
effigies added 3 commits Oct 8, 2019
ENH: Clarify the position toward non-compliant derivative datasets and files
CPernet and others added 6 commits Oct 23, 2019
#345  discuss splitting a pipeline into multiple derivatives with/without nesting
Co-Authored-By: Chris Markiewicz <markiewicz@stanford.edu>
Co-Authored-By: Chris Markiewicz <markiewicz@stanford.edu>
Co-Authored-By: Chris Markiewicz <markiewicz@stanford.edu>
Co-Authored-By: Chris Markiewicz <markiewicz@stanford.edu>
specify further the pipeline following #345
@robertoostenveld

This comment has been minimized.

Copy link
Collaborator

robertoostenveld commented Nov 1, 2019

could anyone tell me whether https://bids-specification.readthedocs.io/en/derivatives/ which states "v1.3.0-dev" is up to date with this PR?

@effigies

This comment has been minimized.

Copy link
Collaborator Author

effigies commented Nov 1, 2019

@robertoostenveld common-derivatives is a fork of derivatives with most of the modality-specific bits removed. I'm not making an effort at this point to push our changes back to that branch, but we probably could if that seems worthwhile. I can also remove the branch from readthedocs, if we just want to reduce confusion.

@robertoostenveld

This comment has been minimized.

Copy link
Collaborator

robertoostenveld commented Nov 1, 2019

@robertoostenveld common-derivatives is a fork of derivatives with most of the modality-specific bits removed. I'm not making an effort at this point to push our changes back to that branch, but we probably could if that seems worthwhile. I can also remove the branch from readthedocs, if we just want to reduce confusion.

I don't want it specifically on readthedocs, but would appreciate BEP003 being rendered in a format that allows non-github gurus (like me) to be able to read it, form an opinion and contribute. Parsing the diffs to 16 files that are in a PR to a branch, which itself is a PR to another branch, stresses my mental capacity and also exceeds my available BIDS-quality time.

As a reminder, it was discussed here prior to moving to github that conceptual (rather than technical) discussions might better happen on google docs to ensure that the discussions are inclusive to people with different skills and backgrounds. If it were easy to render a branch on readthedocs using some CI magic, then that would also be fine with me; it is not that the comments have to be in a google doc, just that the BEP is directly readable.

@effigies

This comment has been minimized.

Copy link
Collaborator Author

effigies commented Nov 1, 2019

Oh, sorry. If you want to see the current rendering of this PR, the URL is https://bids-specification.readthedocs.io/en/common-derivatives/.

@robertoostenveld

This comment has been minimized.

Copy link
Collaborator

robertoostenveld commented Nov 1, 2019

Does this work in general for all branches of this repo? It does not seem to work e.g. for the satra-units2ascii branch. Is the readthedocs-rendered version always up to date?

@effigies

This comment has been minimized.

Copy link
Collaborator Author

effigies commented Nov 1, 2019

Does this work in general for all branches of this repo? It does not seem to work e.g. for the satra-units2ascii branch.

It needs to be activated for specific branches. Since derivatives is so large and also being fairly widely implemented, this seemed like a good replacement for the draft Google doc when referencing it.

Is the readthedocs-rendered version always up to date?

Within a couple minutes of a commit to the branch, yes.

Also note that there's a "build_docs artifact" in the checks for each PR:

Screen Shot 2019-11-01 at 11 44 12

If you click "details", you can see the rendering of any PR, whether or not it's on RTD.

@effigies

This comment has been minimized.

Copy link
Collaborator Author

effigies commented Nov 4, 2019

Merged master post-#306. @oesteban can you check the merge commit (45d93f9) with particular attention to src/99-appendices/08-coordinate-systems.md?

Feel free to push fixes directly to this branch.

@effigies effigies force-pushed the common-derivatives branch from 03d0793 to 1a7d012 Nov 4, 2019
@effigies effigies force-pushed the common-derivatives branch from 1a7d012 to c200cff Nov 4, 2019
Copy link

sebastientourbier left a comment

  • Corrected 2 typos

  • Suggestion of an example to illustrate the use of _desc-<label> keyworkd value to distinguish segmentations at multiple scales in the case of a multi-scale atlas.

<pipeline_name>/
sub-<participant_label>/
anat/
<source_keywords>[_hemi-{L|R}][_space-<space>]_dseg.{label.gii|dlabel.nii}

This comment has been minimized.

Copy link
@sebastientourbier

sebastientourbier Nov 12, 2019

Typo:
<source_keywords>[_hemi-{L|R}][space-]dseg.{label.gii|dlabel.nii}
should be
<source_keywords>[_hemi-{L|R}][space-]dparc.{label.gii|dlabel.nii}

sub-001_hemi-R_dparc.label.gii
```

The REQUIRED extension for CIFTI parcellations is `.dlabel.nii`. For example:

This comment has been minimized.

Copy link
@sebastientourbier

sebastientourbier Nov 12, 2019

Would not it be:
The REQUIRED extension for NIFTI parcellations is .dlabel.nii. For example:
?

This comment has been minimized.

Copy link
@oesteban

oesteban Dec 5, 2019

Collaborator

NIfTI does not impose any naming, CIFTI strongly recommends them and it is good to adhere to that.

sub-001_space-orig_dseg.nii.gz
sub-001_space-orig_dseg.json
```

This comment has been minimized.

Copy link
@sebastientourbier

sebastientourbier Nov 12, 2019

In the case of a multi-scale atlas (such as in https://doi.org/10.1016/j.jneumeth.2011.09.031) that provides a discrete segmentation of cortical regions for each single scale, and if we follow the description (https://github.com/bids-standard/bids-specification/pull/265/files#diff-0361044f2e50dc58ce5ac1949c9cf93e line 235) we can use the desc keyword to distinguish them.

To illustrate it, I would suggest to add a little paragraph to illustrate this example such as:

In the case of a multi-scale atlas, the _desc-<label> keyword-value should be used to distinguish the segmentations generated for each scale.

Example:

pipeline/
    sub-001/
        anat/
            sub-001_space-orig_desc-scale1_dseg.nii.gz
            sub-001_space-orig_desc-scale1_dseg.json
            sub-001_space-orig_desc-scale2_dseg.nii.gz
            sub-001_space-orig_desc-scale2_dseg.json

This comment has been minimized.

Copy link
@oesteban

oesteban Dec 5, 2019

Collaborator

By multi-scale you refer to several resolutions? If so, please have a look into recent changes (#301)

@robertoostenveld

This comment has been minimized.

Copy link
Collaborator

robertoostenveld commented Nov 26, 2019

There is mention of a multi-scale atlas in the comments. That seems better suited for another pull request, since according to this comment "In this PR, we will focus on the (mostly) modality-agnostic components of the derivatives specification, and the necessary changes to the broader spec.". I don't think multi-scale atlases are modality agnostic.

* [FIX] Revise template-generated coordinate systems

This PR supersedes #213, and should be rebased with master after #306 is
merged.

W.r.t. master it only adds the table of _Nonstandard template
identifiers_

W.r.t. `common-derivatives`, the changes are:

  - [x] Removed the distinction between surface templates and volumetric
        templates. After all, to properly apply a surface template one
        should first spatially normalize to a particular volumetric
        template. Although it is possible to create mixed templates
        (e.g. using ``MNI152Lin`` to generate a mapping for volumetric
        coordinates and then, say, ``fsLR`` for the surface after some
        surface mapping process), this option should be discouraged.
        Anyways, this discussion does not affect how the names of BIDS
        entities should be specified, since one can always describe how
        the template coordinates are generated with the information
        described in this PR.
  - [x] Removed ``MNI152NLin6AsymConte69`` because it does not define
        any new coordinate system (see
        #207 (comment)),
  - [x] Removed ``FS305`` as it is now covered by ``fsaverage``.
  - [x] Reorganized subsections in "standard" vs "nonstandard" spaces.

* fix: added the concept of ``scanner`` as an Image-based coordinate system

- [x] Describes more clearly the situation when files do not have
``space-`` definition.
- [x] Remove the ``ReferenceIndex`` as an index does not generate a
coordinate system (the concept here is mixed up with that of  image
registration, which is just a process).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.