Skip to content

Commit

Permalink
Merge pull request #374 from maxulysse/dev_2.7.1
Browse files Browse the repository at this point in the history
Prepare release 2.7.1
  • Loading branch information
maxulysse committed Jun 14, 2021
2 parents 7ccfb36 + 3c98945 commit 68b9930
Show file tree
Hide file tree
Showing 44 changed files with 1,406 additions and 1,474 deletions.
20 changes: 10 additions & 10 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ jobs:
environment:
GENOME: GRCh37
SNPEFF_CACHE_VERSION: "75"
SAREK_TAG: 2.7
SAREK_TAG: 2.7.1
steps:
- checkout
- setup_remote_docker
Expand All @@ -20,28 +20,28 @@ jobs:
environment:
GENOME: GRCh38
SNPEFF_CACHE_VERSION: "86"
SAREK_TAG: 2.7
SAREK_TAG: 2.7.1

snpeffgrcm38:
<< : *buildsnpeff
environment:
GENOME: GRCm38
SNPEFF_CACHE_VERSION: "86"
SAREK_TAG: 2.7
SAREK_TAG: 2.7.1

snpeffcanfam3_1:
<< : *buildsnpeff
environment:
GENOME: CanFam3.1
SNPEFF_CACHE_VERSION: "86"
SAREK_TAG: 2.7
SAREK_TAG: 2.7.1

snpeffwbcel235:
<< : *buildsnpeff
environment:
GENOME: WBcel235
SNPEFF_CACHE_VERSION: "86"
SAREK_TAG: 2.7
SAREK_TAG: 2.7.1

vepgrch37: &buildvep
docker:
Expand All @@ -50,7 +50,7 @@ jobs:
GENOME: GRCh37
SPECIES: homo_sapiens
VEP_VERSION: "99"
SAREK_TAG: 2.7
SAREK_TAG: 2.7.1
steps:
- checkout
- setup_remote_docker
Expand All @@ -67,31 +67,31 @@ jobs:
GENOME: GRCh38
SPECIES: homo_sapiens
VEP_VERSION: "99"
SAREK_TAG: 2.7
SAREK_TAG: 2.7.1

vepgrcm38:
<< : *buildvep
environment:
GENOME: GRCm38
SPECIES: mus_musculus
VEP_VERSION: "99"
SAREK_TAG: 2.7
SAREK_TAG: 2.7.1

vepcanfam3_1:
<< : *buildvep
environment:
GENOME: CanFam3.1
SPECIES: canis_familiaris
VEP_VERSION: "99"
SAREK_TAG: 2.7
SAREK_TAG: 2.7.1

vepwbcel235:
<< : *buildvep
environment:
GENOME: WBcel235
SPECIES: caenorhabditis_elegans
VEP_VERSION: "99"
SAREK_TAG: 2.7
SAREK_TAG: 2.7.1

workflows:
version: 2
Expand Down
1 change: 1 addition & 0 deletions .github/.dockstore.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ version: 1.2
workflows:
- subclass: nfl
primaryDescriptorPath: /nextflow.config
publish: True
4 changes: 2 additions & 2 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ If you wish to contribute a new step, please use the following coding standards:
2. Write the process block (see below).
3. Define the output channel if needed (see below).
4. Add any new flags/options to `nextflow.config` with a default (see below).
5. Add any new flags/options to `nextflow_schema.json` with help text (with `nf-core schema build .`)
5. Add any new flags/options to `nextflow_schema.json` with help text (with `nf-core schema build .`).
6. Add any new flags/options to the help message (for integer/text parameters, print to help the corresponding `nextflow.config` parameter).
7. Add sanity checks for all relevant parameters.
8. Add any new software to the `scrape_software_versions.py` script in `bin/` and the version command to the `scrape_software_versions` process in `main.nf`.
Expand All @@ -87,7 +87,7 @@ Once there, use `nf-core schema build .` to add to `nextflow_schema.json`.

### Default processes resource requirements

Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/%7B%7Bcookiecutter.name_noslash%7D%7D/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.
Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.

The process resources can be passed on to the tool dynamically within the process with the `${task.cpu}` and `${task.memory}` variables in the `script:` block.

Expand Down
9 changes: 4 additions & 5 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@ name: Bug report
about: Report something that is broken or incorrect
title: "[BUG]"
labels: bug
assignees: MaxUlysse

assignees: maxulysse
---

<!--
Expand All @@ -21,7 +20,7 @@ Please delete this text and anything that's not relevant from the template below
I have checked the following places for your error:

- [ ] [nf-core website: troubleshooting](https://nf-co.re/usage/troubleshooting)
- [ ] [nf-core/sarek pipeline documentation](https://nf-co.re/nf-core/sarek/usage)
- [ ] [nf-core/sarek pipeline documentation](https://nf-co.re/sarek/usage)

## Description of the bug

Expand Down Expand Up @@ -58,9 +57,9 @@ Have you provided the following extra information/files:

## Container engine

- Engine: <!-- [e.g. Conda, Docker, Singularity or Podman] -->
- Engine: <!-- [e.g. Conda, Docker, Singularity, Podman, Shifter or Charliecloud] -->
- version: <!-- [e.g. 1.0.0] -->
- Image tag: <!-- [e.g. nfcore/sarek:2.6] -->
- Image tag: <!-- [e.g. nfcore/sarek:2.7] -->

## Additional context

Expand Down
3 changes: 1 addition & 2 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@ name: Feature request
about: Suggest an idea for the nf-core/sarek pipeline
title: "[FEATURE]"
labels: enhancement
assignees: MaxUlysse

assignees: maxulysse
---

<!--
Expand Down
7 changes: 4 additions & 3 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,15 @@ Remember that PRs should be made against the dev branch, unless you're preparing
Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/sarek/tree/master/.github/CONTRIBUTING.md)
-->
<!-- markdownlint-disable ul-indent -->

## PR checklist

- [ ] This comment contains a description of changes (with reason).
- [ ] If you've fixed a bug or added code that should be tested, add tests!
- [ ] If you've added a new tool - add to the software_versions process and a regex to `scrape_software_versions.py`
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/sarek/tree/master/.github/CONTRIBUTING.md)
- [ ] If necessary, also make a PR on the nf-core/sarek _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
- [ ] If you've added a new tool - add to the software_versions process and a regex to `scrape_software_versions.py`
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/sarek/tree/master/.github/CONTRIBUTING.md)
- [ ] If necessary, also make a PR on the nf-core/sarek _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
- [ ] Make sure your code lints (`nf-core lint .`).
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker`).
- [ ] Usage Documentation in `docs/usage.md` is updated.
Expand Down
5 changes: 2 additions & 3 deletions .github/markdownlint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,8 @@ no-duplicate-header:
siblings_only: true
no-inline-html:
allowed_elements:
- a
- details
- img
- kbd
- p
- kbd
- details
- summary
17 changes: 10 additions & 7 deletions .github/workflows/awsfulltest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,16 @@ on:
types: [completed]
workflow_dispatch:


env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}


jobs:
run-awstest:
name: Run AWS full tests
Expand All @@ -27,13 +37,6 @@ jobs:
# Add full size test data (but still relatively small datasets for few samples)
# on the `test_full.config` test runs with only one set of parameters
# Then specify `-profile test_full` instead of `-profile test` on the AWS batch command
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
run: |
aws batch submit-job \
--region eu-west-1 \
Expand Down
20 changes: 13 additions & 7 deletions .github/workflows/awstest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,16 @@ name: nf-core AWS test
on:
workflow_dispatch:


env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}


jobs:
run-awstest:
name: Run AWS tests
Expand All @@ -20,13 +30,9 @@ jobs:
- name: Install awscli
run: conda install -c conda-forge awscli
- name: Start AWS batch job
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}
# TODO nf-core: You can customise CI pipeline run tests as required
# For example: adding multiple test runs with different parameters
# Remember that you can parallelise this by using strategy.matrix
run: |
aws batch submit-job \
--region eu-west-1 \
Expand Down
16 changes: 13 additions & 3 deletions .github/workflows/branch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
- name: Check PRs
if: github.repository == 'nf-core/sarek'
run: |
{ [[ ${{github.event.pull_request.head.repo.full_name}} == nf-core/sarek ]] && [[ $GITHUB_HEAD_REF = "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
{ [[ ${{github.event.pull_request.head.repo.full_name }} == nf-core/sarek ]] && [[ $GITHUB_HEAD_REF = "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
# If the above check failed, post a comment on the PR explaining the failure
Expand All @@ -23,14 +23,24 @@ jobs:
uses: mshick/add-pr-comment@v1
with:
message: |
## This PR is against the `master` branch :x:
* Do not close this PR
* Click _Edit_ and change the `base` to `dev`
* This CI test will remain failed until you push a new commit
---
Hi @${{ github.event.pull_request.user.login }},
It looks like this pull-request is has been made against the ${{github.event.pull_request.head.repo.full_name}} `master` branch.
It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `master` branch.
The `master` branch on nf-core repositories should always contain code from the latest release.
Because of this, PRs to `master` are only allowed if they come from the ${{github.event.pull_request.head.repo.full_name}} `dev` branch.
Because of this, PRs to `master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch.
You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page.
Note that even after this, the test will continue to show as failing until you push a new commit.
Thanks again for your contribution!
repo-token: ${{ secrets.GITHUB_TOKEN }}
allow-repeats: false

25 changes: 14 additions & 11 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ on:
release:
types: [published]

# Uncomment if we need an edge release of Nextflow again
# env: NXF_EDGE: 1

jobs:
test:
name: Run workflow tests
Expand All @@ -33,13 +36,13 @@ jobs:
environment.yml
- name: Build new docker image
if: env.MATCHED_FILES
run: docker build --no-cache . -t nfcore/sarek:2.7
run: docker build --no-cache . -t nfcore/sarek:2.7.1

- name: Pull docker image
if: ${{ !env.MATCHED_FILES }}
run: |
docker pull nfcore/sarek:dev
docker tag nfcore/sarek:dev nfcore/sarek:2.7
docker tag nfcore/sarek:dev nfcore/sarek:2.7.1
- name: Install Nextflow
env:
Expand Down Expand Up @@ -71,8 +74,8 @@ jobs:
NXF_VER: '20.04.0'
- name: Pull docker image
run: |
docker pull nfcore/sarek:2.7
docker pull nfcore/sarek${{ matrix.tools }}:2.7.${{ matrix.species }}
docker pull nfcore/sarek:2.7.1
docker pull nfcore/sarek${{ matrix.tools }}:2.7.1.${{ matrix.species }}
- name: Run annotation test
run: nextflow run ${GITHUB_WORKSPACE} -profile test_annotation,docker --tools ${{ matrix.tools }}

Expand All @@ -93,7 +96,7 @@ jobs:
# Only check Nextflow pipeline minimum version
NXF_VER: '20.04.0'
- name: Pull docker image
run: docker pull nfcore/sarek:2.7
run: docker pull nfcore/sarek:2.7.1
- name: Get test data
run: git clone --single-branch --branch sarek https://github.com/nf-core/test-datasets.git data
- name: Run germline test
Expand Down Expand Up @@ -121,9 +124,9 @@ jobs:
# Only check Nextflow pipeline minimum version
NXF_VER: '20.04.0'
- name: Pull docker image
run: docker pull nfcore/sarek:2.7
run: docker pull nfcore/sarek:2.7.1
- name: Run test for minimal genomes
run: nextflow run ${GITHUB_WORKSPACE} -profile test,docker --skipQC all --genome ${{ matrix.genome }} ${{ matrix.intervals }} --tools Manta,mpileup,Strelka,FreeBayes
run: nextflow run ${GITHUB_WORKSPACE} -profile test,docker --skip_qc all --genome ${{ matrix.genome }} ${{ matrix.intervals }} --tools Manta,mpileup,Strelka,FreeBayes

profile:
env:
Expand All @@ -142,7 +145,7 @@ jobs:
# Only check Nextflow pipeline minimum version
NXF_VER: '20.04.0'
- name: Pull docker image
run: docker pull nfcore/sarek:2.7
run: docker pull nfcore/sarek:2.7.1
- name: Run ${{ matrix.profile }} test
run: nextflow run ${GITHUB_WORKSPACE} -profile ${{ matrix.profile }},docker

Expand All @@ -163,7 +166,7 @@ jobs:
# Only check Nextflow pipeline minimum version
NXF_VER: '20.04.0'
- name: Pull docker image
run: docker pull nfcore/sarek:2.7
run: docker pull nfcore/sarek:2.7.1
- name: Run ${{ matrix.profile }} test
run: nextflow run ${GITHUB_WORKSPACE} -profile test,docker --aligner ${{ matrix.aligner }}

Expand Down Expand Up @@ -194,6 +197,6 @@ jobs:
# Only check Nextflow pipeline minimum version
NXF_VER: '20.04.0'
- name: Pull docker image
run: docker pull nfcore/sarek:2.7
run: docker pull nfcore/sarek:2.7.1
- name: Run ${{ matrix.tool }} test
run: nextflow run ${GITHUB_WORKSPACE} -profile test_tool,docker --tools ${{ matrix.tool }} ${{ matrix.intervals }}
run: nextflow run ${GITHUB_WORKSPACE} -profile test_tool,docker --tools ${{ matrix.tool }} ${{ matrix.intervals }}

0 comments on commit 68b9930

Please sign in to comment.