New module Ilastik/multicut

[FloWuenne]

PR checklist

Closes #2865
Uses the same tool (ilastik) as in #2875

• This comment contains a description of changes (with reason).
• If you've fixed a bug or added code that should be tested, add tests!
• If you've added a new tool - have you followed the module conventions in the contribution docs
• If necessary, include test data in your PR.
• Remove all TODO statements.
• Emit the versions.yml file.
• Follow the naming conventions.
• Follow the parameters requirements.
• Follow the input/output options guidelines.
• Add a resource label
• Use BioConda and BioContainers if possible to fulfil software requirements.
• Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
  • PROFILE=docker pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware
  • PROFILE=singularity pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware
  • PROFILE=conda pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware

Description of changes

This PR adds ilastik Multicut as a DSL2 module. The module takes as input an image in .h5 format, a multicut ilastik project and a probability map from a previously trained ilastik pixel classification. It outputs a multicut segmentation mask in .tiff format.

Things to do before merging

• Add test data to module branch of https://github.com/nf-core/test-datasets
• Determine whether ilastik-forge conda channel can be used for conda

[nvnieuwk]

Can this be added to conda?

[FloWuenne]

So it's a bit tricky with ilastik. It is already on conda, but on it's own channel called ilastik-forge. There is also a biocontainer, but I don't see from where it is built : https://hub.docker.com/r/biocontainers/ilastik/tags. Additionally, the biocontainer is not the most recent release candidate. This is problematic, since MacOS users can't use the latest stable release to train models but can use the release candidate. That's why for now I put our mcmicro container, which we can update and uses the latest release.

[nvnieuwk]

Wow I've never seen a tool have it's own specific conda channel. Is it impossible to use this channel in Nextflow?

[FloWuenne]

I actually have no idea. @drpatelh is a custom conda channel like ilastik-forge useable in the conda definition of modules?

[FloWuenne]

I am gonna check with the ilastik developers, whether they plan to have a maintained container on dockerhub that tracks the newest version. In the meantime, I think having the ilastik modules run off the latest mcmicro-ilastik container is the best way to go. That way we can update the container whenever we need to.

[nvnieuwk]

Allright, can you also ask whether or not they would want to add their tool to biocontainers instead of that separate channel? I think it would be best for everyone (although I don't know what their reason is so we'll have to see)

[FloWuenne]

Any idea why the singularity build still fails? Is it due to this part of the nextflow.config in test?

profiles {
    standard {
        docker {
            enabled = true
            userEmulation = false
        }
    }
}

I had to add userEmulation = false to the config, because otherwise the workflow gives errors about puid tracking within the container.

[FloWuenne]

Any idea why the singularity build still fails? Is it due to this part of the nextflow.config in test?

profiles {
    standard {
        docker {
            enabled = true
            userEmulation = false
        }
    }
}

I had to add userEmulation = false to the config, because otherwise the workflow gives errors about puid tracking within the container.

It seems that having enabled = true for docker as the standard profile was the issue. I removed it and now the singularity container also builds correctly!

[nvnieuwk]

Can you also add a check for pipelines running with conda like this?

    // Exit if running this module with -profile conda / -profile mamba
    if (workflow.profile.tokenize(',').intersect(['conda', 'mamba']).size() >= 1) {
        exit 1, "ILASTIK_MULTICUT module does not support Conda. Please use Docker / Singularity / Podman instead."
    }

Otherwise LGTM

[FloWuenne]

Before merging, the ilastik authors have just released a "stable" release version 1.4.0. We might wanna update the mcmicro ilastik container and this here to that version. I will look into that now!

[FloWuenne]

Should be good to merge now!

[maxulysse]

I'm hoping I fixed the merge conflicts properly

[maxulysse]

@nf-core-bot fix linting pretty please 🙏