[ENH] Dynamic citation boilerplate #1024

oesteban · 2018-03-05T19:06:32Z

Compose a citation boilerplate in a per-workflow basis, with up-to-date software versions and generated based on the actual workflows being run.

This PR is more of a request for comments from @effigies, @chrisfilo.

Maybe this can be done already with duecredit (cc @yarikoptic), but the basic idea here is to place a mechanism to build literate descriptions of the workflow. My plan would be then collect all the references with duecredit.

How it works. Workflows in FMRIPREP will have a __predesc__ and a __postdesc__ attributes (in this version the pre- dunder is just __desc__ as I was just prototyping). When writing a new workflow, these fields can be used to add the corresponding literate description of what that workflow does (allowing to dynamically replace version strings, optional parameters, etc).

Let me know if you think this is worth exploring. The most challenging culprit happens with inhomogeneous datasets (e.g. fieldmap has IntendedFor only on 3 out of 5 runs). In that case the citation boilerplate will have two descriptions, as there's is not just one.

Any comments will be very welcome.

EDIT

Results

This is how it looks

Links to more results

In the context of the reports

Compose a citation boilerplate in a per-workflow basis, with up-to-date software versions and generated based on the actual workflows being run.

…rplate

chrisgorgo · 2018-03-05T19:12:47Z

I have a feeling this would be very challenging to do if the goal is to provide a snippet of text ready to paste into the manuscript. A solution based on conditional statements mirroring the one we did on the website might be easier to implement (even though harder to maintain).

oesteban · 2018-03-05T19:30:32Z

IMHO implementation is fairly natural since each Workflow factory function can (this is not mandatory) set the __predesc__ and __postdesc__ attributes and they are automatically picked up the same way nipype generates the graph. Having them set right after the workflow instantiation feels pretty natural, just like adding another docstring.

For this PR I didn't get to the bottom and generate the exact citation boilerplate we have currently online just for laziness, but this prototype is able to do that already.

I think we could leverage duecredit to generate the lookup table of references (@yarikoptic can correct me if I'm wrong).

On the other hand, I see some positive side effects:

Trivial to append the boilerplate to the report (minimizing work for the user)
Trivial to maintain the citing.rst. Also, the citing.rst can be generated automatically, becoming an actual test for this feature.
Automatically include these new dunder fields when generating the documentation will make it very easy to highlight these excerpts with their corresponding citation in documentation proper.

I'll let this sit here while I focus on more urgent issues.

…rplate

oesteban · 2018-07-25T17:54:32Z

Hey @satra, what do you think about this extension to nipype?. Particularly the magic:

https://github.com/oesteban/fmriprep/blob/b349694455eac1fbbaa7c6ee1b6a133b71c980e4/fmriprep/engine/workflows.py#L14-L49

satra · 2018-07-26T01:01:53Z

@oesteban - i think this a reasonable starting point that could then be edited by a human. i agree with @chrisfilo that this can get complicated. but this is at least a simple mechanism by which certain workflows could be described. how about adding a flag to allow ignoring nested workflows.

oesteban · 2018-07-26T17:35:13Z

how about adding a flag to allow ignoring nested workflows

Seems like a good idea.

…rplate

oesteban · 2018-07-29T19:36:53Z

Ready for review!

effigies

The display looks off for Markdown and LaTeX:

https://2366-53608443-gh.circle-artifacts.com/0/tmp/ds005/derivatives/fmriprep/sub-01.html#boilerplate

Also, the text indicates that CIFTI files were created for all of the test datasets, which I think we only do for one dataset. So we may not be handling some conditionals properly.

effigies · 2018-07-30T14:04:17Z

docs/contributors.rst

+Once all the sub-workflows of a given workflow have
+been visited, then the ``__postdesc__`` attribute is appended
+and the execution pops out to higher level workflows.
+The dunder attributes are written in Markup language, and may contain


effigies · 2018-07-30T14:10:38Z

.circleci/ds005_outputs.txt

@@ -1,5 +1,7 @@
 fmriprep
 fmriprep/logs
+fmriprep/logs/CITATION.html
+fmriprep/logs/CITATION.md


Add CITATION.tex to expected outputs.

effigies · 2018-07-30T14:18:00Z

fmriprep/workflows/bold/resampling.py

+    workflow = Workflow(name=name)
+    workflow.__desc__ = """\
+The BOLD time-series were resampled on {tpl} standard space,
+generating a *preprocessed BOLD run on {tpl} space*.


effigies · 2018-07-30T14:18:15Z

fmriprep/workflows/bold/resampling.py

-    workflow = pe.Workflow(name=name)
+    workflow = Workflow(name=name)
+    workflow.__desc__ = """\
+The BOLD time-series were resampled on {tpl} standard space,


oesteban · 2018-07-30T15:09:02Z

The display looks off for Markdown and LaTeX

Yep, I'm working this out right now.

oesteban · 2018-07-30T17:35:04Z

Okay, with my last commit this should be ready to go.

Fixed visualization
Fixed @effigies' minor comments
Fixed @effigies' major comment about the grayordinates file - I have actually added the HCP pipelines reference and cited the paper in two contexts: the grayordinates files and the epi unwarp with phase difference fieldmap (since that workflow is inspired in HCP pipelines).

This is how it looks now (locally, awaiting for tests in circle):

and

effigies · 2018-07-30T17:41:31Z

fmriprep/workflows/bold/resampling.py

@@ -92,6 +92,8 @@ def init_bold_surf_wf(mem_gb, output_spaces, medial_surface_nan, name='bold_surf
        workflow.__desc__ = """\
 The BOLD time-series, were resampled to surfaces on the following
 spaces: {out_spaces}.
+*Grayordinates* files [@hcppipelines], which combine surface-sampled
+data and volume-sampled data, were also generated.


Do we want to make this conditional on cifti_output? In which case, it might go better back in init_func_preproc_wf, which has access to that variable.

Thanks, I've just added it to init_func_preproc_wf 👍

effigies

I'm good with this. You might want to update the GitHub Project, so that things done after 1.1.0 don't get accidentally labeled as such.

oesteban · 2018-07-30T19:33:52Z

I'm merging this.

@chrisfilo
We need to figure out how to cite the right version here. Especially considering that the zenodo handle is created after the release is made.

I'm assuming this comment is not meant to block merging the PR. I've opened #1229 to follow up on this problem.

oesteban added 2 commits March 5, 2018 10:59

[ENH,WIP] Dynamic citation boilerplate

927db19

Compose a citation boilerplate in a per-workflow basis, with up-to-date software versions and generated based on the actual workflows being run.

Merge remote-tracking branch 'upstream/master' into enh/dynamic-boile…

b234d15

…rplate

oesteban added 4 commits March 21, 2018 18:30

Merge remote-tracking branch 'upstream/master' into enh/dynamic-boile…

917e195

…rplate

Merge remote-tracking branch 'upstream/master' into enh/dynamic-boile…

86bdeb8

…rplate

add space

773a2d2

Merge remote-tracking branch 'upstream/master' into enh/dynamic-boile…

b349694

…rplate

fix nipype import

1342c0f

oesteban added 10 commits July 26, 2018 15:29

full boilerplate, write to report, add bibliography

724d5ca

fix errors, generate html boilerplate

91db8de

fix errors, install pandoc

5e59f81

fix report generation

98173d9

pandoc as a new dependency

f6c1084

Merge remote-tracking branch 'upstream/master' into enh/dynamic-boile…

677acd9

…rplate

mv CITATION.html generation to start

5fb45ce

migrate reports.py to pathlib

eee4e11

use version placeholders when version not available

7e316f9

minor improvements

cf2f560

oesteban added this to To do in 1.3.0 via automation Jul 27, 2018

oesteban moved this from To do to In progress in 1.3.0 Jul 27, 2018

oesteban added 6 commits July 27, 2018 13:55

add bibliography file to installation

232a52c

add missing reference for FSL flirt

d7419e6

fixing citations

44c4f36

minor fixes

08db86e

fix div class

85ad7ca

fix citations

7575275

oesteban changed the title ~~[ENH,WIP] Dynamic citation boilerplate~~ [ENH] Dynamic citation boilerplate Jul 29, 2018

oesteban requested review from effigies and rwblair July 29, 2018 19:36

oesteban mentioned this pull request Jul 29, 2018

[DOC] Add ME descriptions #1227

Closed

oesteban added 6 commits July 29, 2018 17:47

fix epidewarp name

db370fa

we know whether sdc was performed

012a6c8

ensure all caps when needed

6eb4ab7

improvements to the text

f24d414

show tabs for boilerplate

c5f599e

working out different languages

d2d7631

effigies reviewed Jul 30, 2018

View reviewed changes

oesteban added 6 commits July 30, 2018 08:34

update bootstrap to 4.0+, fixes visualization of tabs

6f82497

fix navbar after bootstrap upgrade, improve styling of boilerplate

0bcd88c

add new citation LaTeX output (fix tests)

fe2970c

fix typo Markdown

5bada5e

fix typos

748330a

add citation to HCP pipelines, fix grayordinates

8bfe2c9

effigies reviewed Jul 30, 2018

View reviewed changes

grayordinates sentence in the right place

fdf14ad

effigies approved these changes Jul 30, 2018

View reviewed changes

oesteban mentioned this pull request Jul 30, 2018

[ENH] Zenodo citation on boilerplate #1229

Closed

oesteban merged commit 38d6e04 into nipreps:master Jul 30, 2018

1.3.0 automation moved this from In progress to Done Jul 30, 2018

oesteban deleted the enh/dynamic-boilerplate branch July 30, 2018 19:34

oesteban mentioned this pull request Jul 30, 2018

ENH: initial support for duecredit #608

Closed

emdupre mentioned this pull request Jul 30, 2018

Zenodo DOI for tedana ME-ICA/tedana#57

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ENH] Dynamic citation boilerplate #1024

[ENH] Dynamic citation boilerplate #1024

oesteban commented Mar 5, 2018 •

edited

Loading

chrisgorgo commented Mar 5, 2018

oesteban commented Mar 5, 2018

oesteban commented Jul 25, 2018 •

edited

Loading

satra commented Jul 26, 2018

oesteban commented Jul 26, 2018

oesteban commented Jul 29, 2018

effigies left a comment

effigies Jul 30, 2018

effigies Jul 30, 2018

oesteban Jul 30, 2018

effigies Jul 30, 2018

effigies Jul 30, 2018

oesteban commented Jul 30, 2018

oesteban commented Jul 30, 2018

effigies Jul 30, 2018

oesteban Jul 30, 2018

effigies left a comment

oesteban commented Jul 30, 2018 •

edited

Loading

[ENH] Dynamic citation boilerplate #1024

[ENH] Dynamic citation boilerplate #1024

Conversation

oesteban commented Mar 5, 2018 • edited Loading

Results

Links to more results

chrisgorgo commented Mar 5, 2018

oesteban commented Mar 5, 2018

oesteban commented Jul 25, 2018 • edited Loading

satra commented Jul 26, 2018

oesteban commented Jul 26, 2018

oesteban commented Jul 29, 2018

effigies left a comment

Choose a reason for hiding this comment

effigies Jul 30, 2018

Choose a reason for hiding this comment

effigies Jul 30, 2018

Choose a reason for hiding this comment

oesteban Jul 30, 2018

Choose a reason for hiding this comment

effigies Jul 30, 2018

Choose a reason for hiding this comment

effigies Jul 30, 2018

Choose a reason for hiding this comment

oesteban commented Jul 30, 2018

oesteban commented Jul 30, 2018

effigies Jul 30, 2018

Choose a reason for hiding this comment

oesteban Jul 30, 2018

Choose a reason for hiding this comment

effigies left a comment

Choose a reason for hiding this comment

oesteban commented Jul 30, 2018 • edited Loading

oesteban commented Mar 5, 2018 •

edited

Loading

oesteban commented Jul 25, 2018 •

edited

Loading

oesteban commented Jul 30, 2018 •

edited

Loading