Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a subcommand to generate a nf-params.yml template #2362

Merged
merged 14 commits into from
Jul 19, 2023

Conversation

fbdtemme
Copy link
Contributor

@fbdtemme fbdtemme commented Jul 4, 2023

This PR adds a new subcommand to generate a nf-params.yml template.

Sometimes it is easier to manually edit a parameter file than to use the web interface or interactive commandline wizard
provided by nf-core launch, for example when running a pipeline with many options on a remote server without a graphical interface.

This command takes one argument - either the name of a nf-core pipeline which will be pulled automatically,
or the path to a directory containing a Nextflow pipeline (can be any pipeline, doesn't have to be nf-core).

The generated YAML file contains all parameters set to the pipeline default value along with their description in comments.
This template can then be used by uncommenting and modifying the value of parameters you want to pass to a pipline run.

First lines for eg rnaseq

## ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
##     nf-core/rnaseq
## ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
##     This is an example parameter file to pass to the `-params-file` option
##     of nextflow run with the nf-core/rnaseq pipeline.
##      
##     Uncomment lines with a single '#' if you want to pass the parameter to
##     the pipeline.
## -----------------------------------------------------------------------------

## =============================================================================
## Input/output options
## =============================================================================
## Define where the pipeline should find input data and save output data.

## -----------------------------------------------------------------------------
## input
## -----------------------------------------------------------------------------
##     Path to comma-separated file containing information about the samples
##     in the experiment.
##     Type: string
## -----------------------------------------------------------------------------
# input = null

Command help

Usage: nf-core params-template [OPTIONS] <pipeline name>                                           
                                                                                                    
 Build a parameter file template for a pipeline.                                                    
 Uses the pipeline schema file to generate a YAML parameters file. Parameters are set to the        
 pipeline defaults and descriptions are shown in comments. After the output file is generated, it   
 can then be edited as needed before passing to nextflow using the `-params-file` option.           
 Run using a remote pipeline name (such as GitHub `user/repo` or a URL), a local pipeline           
 directory.                                                                                         
                                                                                                    
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────╮
│ --revision     -r  TEXT        Release/branch/SHA of the pipeline (if remote)                    │
│ --output       -o  <filename>  Output filename. Defaults to `nf-params.yml`.                     │
│ --force        -f              Overwrite existing files                                          │
│ --show-hidden  -x              Show hidden params which don't normally need changing             │
│ --help         -h              Show this message and exit.                                       │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯

I am not 100% convinced about the params-template name, since the template part may be easy to confuse with with the pipeline template.

Maybe nf-core params-file is a better option?

PR checklist

  • This comment contains a description of changes (with reason)
  • CHANGELOG.md is updated
  • If you've fixed a bug or added code that should be tested, add tests!
  • Documentation in docs is updated

@fbdtemme fbdtemme mentioned this pull request Jul 4, 2023
6 tasks
@codecov
Copy link

codecov bot commented Jul 4, 2023

Codecov Report

Merging #2362 (aa88750) into dev (5e5b41a) will increase coverage by 0.36%.
The diff coverage is 91.11%.

@@            Coverage Diff             @@
##              dev    #2362      +/-   ##
==========================================
+ Coverage   72.81%   73.17%   +0.36%     
==========================================
  Files          78       79       +1     
  Lines        8891     9026     +135     
==========================================
+ Hits         6474     6605     +131     
- Misses       2417     2421       +4     
Impacted Files Coverage Δ
nf_core/__main__.py 57.50% <72.72%> (+0.26%) ⬆️
nf_core/params_file.py 92.74% <92.74%> (ø)

... and 2 files with indirect coverage changes

Copy link
Member

@ewels ewels left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super nice, thank you @fbdtemme! 🎉

Just chatting with @mashehu and @mirpedrol about an alternative command name, as we try to reserve the word "template".. I'll let them comment about that though.

README.md Outdated Show resolved Hide resolved
@mashehu
Copy link
Contributor

mashehu commented Jul 4, 2023

Very nice new feature, thanks for contributing!

What do you think about create-params-file for this command?

@fbdtemme
Copy link
Contributor Author

fbdtemme commented Jul 4, 2023

Yeah, create-params-file is better.

README.md Outdated Show resolved Hide resolved
Copy link
Member

@mirpedrol mirpedrol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very nice, thanks!
Remember to update the CHANGELOG.md

@fbdtemme
Copy link
Contributor Author

fbdtemme commented Jul 5, 2023

CHANGELOG is updated now.

Copy link
Member

@mirpedrol mirpedrol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@mashehu
Copy link
Contributor

mashehu commented Jul 5, 2023

I had now time to play with it:

  • could we have . as the default pipeline argument so it will run without prompting (similar to other commands) by just running nf-core create-params-file inside a pipeline repo? Nevermind, looks like we only have that when the argument is the directory not a pipeline name. So no need then.
  • for future reference it would be good to have the pipeline version also printed on the top of file next to the name

nf_core/params_file.py Outdated Show resolved Hide resolved
nf_core/__main__.py Outdated Show resolved Hide resolved
nf_core/params_file.py Outdated Show resolved Hide resolved
nf_core/params_file.py Outdated Show resolved Hide resolved
@fbdtemme
Copy link
Contributor Author

fbdtemme commented Jul 6, 2023

The version is now printed along with the pipeline name.

@fbdtemme
Copy link
Contributor Author

Is there anything preventing this PR from being merged?

@mirpedrol
Copy link
Member

Looks like some tests are failing now. Once they are fixed, sound ok to me to merge, do you have permissions?

@fbdtemme fbdtemme merged commit 5a6d2b0 into nf-core:dev Jul 19, 2023
20 checks passed
fbdtemme added a commit to nf-core/pixelator that referenced this pull request Aug 9, 2023
This functionality is now merged in nf-core/tools (nf-core/tools#2362) so we can remove this.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants