Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Configure testing with Podman and Mamba #2675

Closed
wants to merge 8 commits into from
Closed

Configure testing with Podman and Mamba #2675

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
8bc4531
configure tests for podman
Dec 16, 2022
e533358
add profile podman to GitHub Actions tests
Dec 16, 2022
9ebb309
Merge branch 'master' of ssh://github.com/nf-core/modules into podman
Dec 16, 2022
b645cd9
creates separate tests for conda and mamba
Dec 16, 2022
0bdc301
remove params.enable_conda as deprecated
TomKellyGenetics Jan 18, 2023
39dff42
cherry-pick changes from https://github.com/nf-core/modules/pull/2740
muffato Jan 17, 2023
ddd8e90
cherry-pick changes from https://github.com/nf-core/modules/pull/2659
maxulysse Dec 14, 2022
a8e4c1b
Merge branch 'podman' of ssh://tkg-github.com/tomkellygenetics/nf-cor…
Jan 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions .github/workflows/pytest-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
fail-fast: false
matrix:
tags: ["${{ fromJson(needs.changes.outputs.modules) }}"]
profile: ["docker", "singularity", "conda"]
profile: ["docker", "podman", "singularity", "conda", "mamba"]
exclude:
- profile: "conda"
tags: bases2fastq
Expand Down Expand Up @@ -121,6 +121,11 @@ jobs:
with:
singularity-version: 3.7.1

- name: Set up Podman
if: matrix.profile == 'podman'
run: |
sudo ln -sf /usr/bin/podman /usr/bin/docker
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this line. Why override /usr/bin/docker ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Required by GitHub Actions as discussed here. actions/runner#505 (comment)


- name: Set up miniconda
if: matrix.profile == 'conda'
uses: conda-incubator/setup-miniconda@v2
Expand All @@ -129,13 +134,17 @@ jobs:
channels: conda-forge,bioconda,defaults
python-version: ${{ matrix.python-version }}

- name: Conda clean
- name: Conda setup
TomKellyGenetics marked this conversation as resolved.
Show resolved Hide resolved
if: matrix.profile == 'conda'
run: conda clean -a

- name: Install mamba
if: matrix.profile == 'conda'
if: matrix.profile == 'mamba'
TomKellyGenetics marked this conversation as resolved.
Show resolved Hide resolved
run: conda install mamba -n base -c conda-forge
run: |
conda clean -a
conda install -n base conda-libmamba-solver
conda config --set solver libmamba
TomKellyGenetics marked this conversation as resolved.
Show resolved Hide resolved

# Test the module
- name: Run pytest-workflow
Expand Down
6 changes: 3 additions & 3 deletions modules/nf-core/nanoplot/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@ process NANOPLOT {
tag "$meta.id"
label 'process_low'

conda "bioconda::nanoplot=1.40.0"
conda "bioconda::nanoplot=1.41.0"
container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
'https://depot.galaxyproject.org/singularity/nanoplot:1.40.0--pyhdfd78af_0' :
'quay.io/biocontainers/nanoplot:1.40.0--pyhdfd78af_0' }"
'https://depot.galaxyproject.org/singularity/nanoplot:1.41.0--pyhdfd78af_0' :
'quay.io/biocontainers/nanoplot:1.41.0--pyhdfd78af_0' }"
TomKellyGenetics marked this conversation as resolved.
Show resolved Hide resolved

input:
tuple val(meta), path(ontfile)
Expand Down
4 changes: 4 additions & 0 deletions tests/config/nextflow.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,12 @@ if ("$PROFILE" == "singularity") {
} else if ("$PROFILE" == "mamba") {
conda.enabled = true
conda.useMamba = true
} else if ("$PROFILE" == "podman") {
podman.enabled = true
podman.runOptions = "--runtime /usr/bin/crun --userns=keep-id --platform linux/x86_64"
} else {
docker.enabled = true
docker.fixOwnership = true
docker.userEmulation = true
docker.runOptions = "--platform linux/x86_64"
}
Expand Down
5 changes: 0 additions & 5 deletions tests/modules/nf-core/nanoplot/nextflow.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,3 @@ process {
publishDir = { "${params.outdir}/${task.process.tokenize(':')[-1].tokenize('_')[0].toLowerCase()}" }

}

// Conda takes too long to resolve the dependencies
conda {
useMamba = true
}
TomKellyGenetics marked this conversation as resolved.
Show resolved Hide resolved