New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added skip_citation_process flag to skip processing citation.md #1876
Conversation
causes jobs to fail due to pandoc using too much vmem
Thanks for opening this pull request! It looks like this is your first time contributing to fMRIPrep. 😄 "name": "Contributor, New FMRIPrep",
"affiliation": "Department of fMRI prep'ing, Open Science Made-Up University",
"orcid": "<your id>"
}, ```
Of course, if you want to opt-out this time there is no problem at all with adding your name later. You will be always welcome to add it in the future whenever you feel it should be listed. |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. This should be moved to a different argument group.
I also wonder if there's a clearer name, but my brain isn't coming up with anything better. Something about "process" feels vague, and once it exists, it'll be annoying to change. So in case someone has a better idea, now would be a good time to change it.
fmriprep/cli/run.py
Outdated
@@ -73,6 +73,9 @@ def get_parser(): | |||
g_bids.add_argument('--skip_bids_validation', '--skip-bids-validation', action='store_true', | |||
default=False, | |||
help='assume the input dataset is BIDS compliant and skip the validation') | |||
g_bids.add_argument('--skip_citation_process', '--skip-citation-process', action='store_true', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
g_bids.add_argument('--skip_citation_process', '--skip-citation-process', action='store_true', | |
g_bids.add_argument('--skip-citation-process', action='store_true', |
Usually the _
one is there for backwards compatibility with old submission scripts. -
is preferred, so we can use it on new options.
And this should go in g_perfm
, not g_bids
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will update the PR. Testing the update now.
could 'skip-pandoc' or 'no-pandoc' be a more direct/intuitive flag name? Given the brief help text, I think it would still be pretty clear to users. |
I'm hesitant to mention What about |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Any further comments?
Changes proposed in this pull request
This PR workarounds the issue of pandoc running out of vmem at the end of the fmriprep pipeline which then terminates the workflow on our HPC environement. #1870
Here is Neurostart discussion about this issue.
Our workflow doesn't just run a single fmriprep, so if fmriprep fails at the end, it will kill the entire workflow. For us, it's important that fmriprep survives the post-processing. The memory issue with pando has been a known issue since 2016 according to this thread (jgm/pandoc#3169) ..