nf-core failure to attain singularity containers #168
Comments
|
Hi @bcardus, our Docker images are not hosted on quay.io. I have not used the If you want to simply pull our images with Singularity, you should be able to accomplish this by removing the reference to quay.io: Note the command will first pull the Docker image in order to build the Singularity image, as we do not host a registry of native Singularity images. |
|
Hi Sam,
Thank you for the pointers and apologies if I submitted this to the wrong github page.
The modifier -l docker.io solved the registry issue noted above.
For anyone who finds this thread in the future, I still got different errors after that (creating the SIF file failed for unclear reasons). However, I was able to pull the images from Singularity one by one as you suggested. I had some issues with running out of disc space as I could not figure out how to set the cache to the right folder, and in the end I used the --disable-cache option. This all worked in the end, and I managed to run the workflow.
Thank you again!
Best wishes,
Bea
From: Sam Nicholls ***@***.***>
Sent: Wednesday, April 3, 2024 10:22 PM
To: epi2me-labs/wf-human-variation ***@***.***>
Cc: Cardus, Beatrix (RTH) OUH ***@***.***>; Mention ***@***.***>
Subject: Re: [epi2me-labs/wf-human-variation] nf-core failure to attain singularity containers (Issue #168)
Hi @bcardus<https://github.com/bcardus>, our Docker images are not hosted on quay.io. I have not used the nf-core download command but from a cursory glance at the docs<https://nf-co.re/tools#downloading-pipelines-for-offline-use> it looks like you might need to supply -l docker.io to override the registry. For any further support on the nf-core tools I would ask the nf-core community<https://nf-co.re/join>.
If you want to simply pull our images with Singularity, you should be able to accomplish this by removing the reference to quay.io:
$ singularity pull --name ontresearch-modkit-shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb.img docker://ontresearch/modkit:shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb
Note the command will first pull the Docker image in order to build the Singularity image, as we do not host a registry of native Singularity images.
-
Reply to this email directly, view it on GitHub<#168 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A4KFTDOXHQ4SZ66MCKNVYMLY3RXGPAVCNFSM6AAAAABFWB2GDGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZVGYYTMNRTGE>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Operating System
CentOS 7
Other Linux
CentOS Linux 7 (Core)
Workflow Version
v2.1.0
Workflow Execution
Command line (Cluster)
Other workflow execution
No response
EPI2ME Version
No response
CLI command run
[hzs682@rescomp2 hzs682]$ nf-core download epi2me-labs/wf-human-variation --force
? Select release / branch: v2.1.0 [release]
? Include the nf-core's default institutional configuration files into the download? Yes
In addition to the pipeline code, this tool can download software containers.
? Download software container images: singularity
If you are working on the same system where you will run Nextflow, you can amend the downloaded images to the ones
in the$NXF_SINGULARITY_CACHEDIR folder, Nextflow will automatically find them.However if you will transfer the
downloaded files to a different system then they should be copied to the target folder.
? Copy singularity images from $NXF_SINGULARITY_CACHEDIR to the target folder or amend new images to the cache? co
py
If transferring the downloaded files to another system, it can be convenient to have everything compressed in a
single file.
This is not recommended when downloading Singularity images, as it can take a long time and saves very little
space.
? Choose compression type: none
Workflow Execution - CLI Execution Profile
None
What happened?
I am trying to use nf-core to download the wf-human-variation workflow onto a cluster entry node so that I can run it on a compute node which does not have internet access. I input the above code and then run through the options below it to select the version, container images etc. However, when it tries to pull the singularity images, it fails and crashes. It seems that the command tries to look at an address in quay.io for the singularity image but can't find it, and when I go to that address on my browser there is nothing there either. When I run the individual command to pull the singularity image, as suggested in the error report, I get another error saying it failed to get the checksum and I am not authorised to access it. I did find docker images in different locations when searching manually e.g. at a different quay.io address and at a Docker hub address, but I don't know how to change this in the command. I am also not sure why it is pulling Docker images when I have asked for Singularity images.
Relevant log output
[hzs682@rescomp2 hzs682]$ nf-core download epi2me-labs/wf-human-variation --force ,--./,-. ___ __ __ __ ___ /,-._.--~\ |\ | |__ __ / ` / \ |__) |__ } { | \| | \__, \__/ | \ |___ \`-._,-`-, `._,._,' nf-core/tools version 2.10 - https://nf-co.re There is a new version of nf-core/tools available! (2.13.1) ? Select release / branch: v2.1.0 [release] ? Include the nf-core's default institutional configuration files into the download? Yes In addition to the pipeline code, this tool can download software containers. ? Download software container images: singularity If you are working on the same system where you will run Nextflow, you can amend the downloaded images to the ones in the$NXF_SINGULARITY_CACHEDIR folder, Nextflow will automatically find them.However if you will transfer the downloaded files to a different system then they should be copied to the target folder. ? Copy singularity images from $NXF_SINGULARITY_CACHEDIR to the target folder or amend new images to the cache? co py If transferring the downloaded files to another system, it can be convenient to have everything compressed in a single file. This is not recommended when downloading Singularity images, as it can take a long time and saves very little space. ? Choose compression type: none WARNING Deleting existing output directory: 'epi2me-labs-wf-human-variation_v2.1.0' INFO Saving 'epi2me-labs/wf-human-variation' Pipeline revision: 'v2.1.0' Use containers: 'singularity' Container library: 'quay.io' Using $NXF_SINGULARITY_CACHEDIR': /well/thompson/users/hzs682/singularity-cashdir' Output directory: 'epi2me-labs-wf-human-variation_v2.1.0' Include default institutional configuration: 'True' INFO Downloading centralised configs from GitHub INFO Downloading workflow files from GitHub INFO Processing workflow revision v2.1.0, found 9 container images in total. Pulling singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0% • 0/9 completed ontresearch/modkit:shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ERROR "ontresearch/modkit:shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb" failed for unclear reasons. Pulling singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0% • 0/9 completedINFO Pulling of "ontresearch/modkit:shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb" failed. Please troubleshoot the command "singularity pull --name /well/thompson/users/hzs682/singularity-cashdir/ontresearch-modkit-shaeedb131a939d3eea2f9bd4dbecec805c0fa 20bdb.img docker://quay.io/ontresearch/modkit:shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb" manually. ERROR "ontresearch/modkit:shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb" failed for unclear reasons. ERROR Pulling of "ontresearch/modkit:shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb" failed. Please troubleshoot the command "singularity pull --name /well/thompson/users/hzs682/singularity-cashdir/ontresearch-modkit-shaeedb131a939d3eea2f9bd4dbecec805c0fa 20bdb.img docker://quay.io/ontresearch/modkit:shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb" manually. Pulling singularity images ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0% • 0/9 completed ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ /apps/eb/2022b/skylake/software/nf-core/2.10-foss-2022b/lib/python3.10/site-packages/nf_core/dow │ │ nload.py:1066 in get_singularity_images │ │ │ │ 1063 │ │ │ │ │ │ # Iteration happens over a copy of self.container_library[:], as │ │ 1064 │ │ │ │ │ │ for library in self.container_library[:]: │ │ 1065 │ │ │ │ │ │ │ try: │ │ ❱ 1066 │ │ │ │ │ │ │ │ self.singularity_pull_image(*container, library, progres │ │ 1067 │ │ │ │ │ │ │ │ # Pulling the image was successful, no ContainerError wa │ │ 1068 │ │ │ │ │ │ │ │ break │ │ 1069 │ │ │ │ │ │ │ except ContainerError.ImageExists as e: │ │ │ │ /apps/eb/2022b/skylake/software/nf-core/2.10-foss-2022b/lib/python3.10/site-packages/nf_core/dow │ │ nload.py:1285 in singularity_pull_image │ │ │ │ 1282 │ │ │ # something went wrong with the container retrieval │ │ 1283 │ │ │ if any("FATAL: " in line for line in lines): │ │ 1284 │ │ │ │ progress.remove_task(task) │ │ ❱ 1285 │ │ │ │ raise ContainerError( │ │ 1286 │ │ │ │ │ container=container, │ │ 1287 │ │ │ │ │ registry=library, │ │ 1288 │ │ │ │ │ address=address, │ │ │ │ /apps/eb/2022b/skylake/software/nf-core/2.10-foss-2022b/lib/python3.10/site-packages/nf_core/dow │ │ nload.py:1616 in __init__ │ │ │ │ 1613 │ │ log.debug(f'Failed command:\n{" ".join(singularity_command)}') │ │ 1614 │ │ log.debug(f'Singularity error messages:\n{"".join(error_msg)}') │ │ 1615 │ │ │ │ ❱ 1616 │ │ raise self.error_type │ │ 1617 │ │ │ 1618 │ class RegistryNotFound(ConnectionRefusedError): │ │ 1619 │ │ """The specified registry does not resolve to a valid IP address""" │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ OtherError: ('[bold red]"ontresearch/modkit:shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb" failed for unclear reasons.[/]\n', 'Pulling of "ontresearch/modkit:shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb" failed.\nPlease troubleshoot the command \n"singularity pull --name /well/thompson/users/hzs682/singularity-cashdir/ontresearch-modkit-shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb.img docker://quay.io/ontresearch/modkit:shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb" manually.\n', ContainerError()) During handling of the above exception, another exception occurred: ╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ /apps/eb/2022b/skylake/software/nf-core/2.10-foss-2022b/bin/nf-core:8 in <module> │ │ │ │ 5 from nf_core.__main__ import run_nf_core │ │ 6 if __name__ == '__main__': │ │ 7 │ sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) │ │ ❱ 8 │ sys.exit(run_nf_core()) │ │ 9 │ │ │ │ /apps/eb/2022b/skylake/software/nf-core/2.10-foss-2022b/lib/python3.10/site-packages/nf_core/__m │ │ ain__.py:111 in run_nf_core │ │ │ │ 108 │ │ │ log.debug(f"Could not check latest version: {e}") │ │ 109 │ │ stderr.print("\n") │ │ 110 │ # Launch the click cli │ │ ❱ 111 │ nf_core_cli(auto_envvar_prefix="NFCORE") │ │ 112 │ │ 113 │ │ 114 @click.group(context_settings=dict(help_option_names=["-h", "--help"])) │ │ │ │ /apps/eb/2022b/skylake/software/Python/3.10.8-GCCcore-12.2.0/lib/python3.10/site-packages/click/ │ │ core.py:1130 in __call__ │ │ │ │ 1127 │ │ │ 1128 │ def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any: │ │ 1129 │ │ """Alias for :meth:`main`.""" │ │ ❱ 1130 │ │ return self.main(*args, **kwargs) │ │ 1131 │ │ 1132 │ │ 1133 class Command(BaseCommand): │ │ │ │ /apps/eb/2022b/skylake/software/nf-core/2.10-foss-2022b/lib/python3.10/site-packages/rich_click/ │ │ rich_command.py:126 in main │ │ │ │ 123 │ │ try: │ │ 124 │ │ │ try: │ │ 125 │ │ │ │ with self.make_context(prog_name, args, **extra) as ctx: │ │ ❱ 126 │ │ │ │ │ rv = self.invoke(ctx) │ │ 127 │ │ │ │ │ if not standalone_mode: │ │ 128 │ │ │ │ │ │ return rv │ │ 129 │ │ │ │ │ # it's not safe to `ctx.exit(rv)` here! │ │ │ │ /apps/eb/2022b/skylake/software/Python/3.10.8-GCCcore-12.2.0/lib/python3.10/site-packages/click/ │ │ core.py:1657 in invoke │ │ │ │ 1654 │ │ │ │ super().invoke(ctx) │ │ 1655 │ │ │ │ sub_ctx = cmd.make_context(cmd_name, args, parent=ctx) │ │ 1656 │ │ │ │ with sub_ctx: │ │ ❱ 1657 │ │ │ │ │ return _process_result(sub_ctx.command.invoke(sub_ctx)) │ │ 1658 │ │ │ │ 1659 │ │ # In chain mode we create the contexts step by step, but after the │ │ 1660 │ │ # base command has been invoked. Because at that point we do not │ │ │ │ /apps/eb/2022b/skylake/software/Python/3.10.8-GCCcore-12.2.0/lib/python3.10/site-packages/click/ │ │ core.py:1404 in invoke │ │ │ │ 1401 │ │ │ echo(style(message, fg="red"), err=True) │ │ 1402 │ │ │ │ 1403 │ │ if self.callback is not None: │ │ ❱ 1404 │ │ │ return ctx.invoke(self.callback, **ctx.params) │ │ 1405 │ │ │ 1406 │ def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]: │ │ 1407 │ │ """Return a list of completions for the incomplete value. Looks │ │ │ │ /apps/eb/2022b/skylake/software/Python/3.10.8-GCCcore-12.2.0/lib/python3.10/site-packages/click/ │ │ core.py:760 in invoke │ │ │ │ 757 │ │ │ │ 758 │ │ with augment_usage_errors(__self): │ │ 759 │ │ │ with ctx: │ │ ❱ 760 │ │ │ │ return __callback(*args, **kwargs) │ │ 761 │ │ │ 762 │ def forward( │ │ 763 │ │ __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any # noqa: B902 │ │ │ │ /apps/eb/2022b/skylake/software/nf-core/2.10-foss-2022b/lib/python3.10/site-packages/nf_core/__m │ │ ain__.py:344 in download │ │ │ │ 341 │ │ container_cache_index, │ │ 342 │ │ parallel_downloads, │ │ 343 │ ) │ │ ❱ 344 │ dl.download_workflow() │ │ 345 │ │ 346 │ │ 347 # nf-core licences │ │ │ │ /apps/eb/2022b/skylake/software/nf-core/2.10-foss-2022b/lib/python3.10/site-packages/nf_core/dow │ │ nload.py:234 in download_workflow │ │ │ │ 231 │ │ if self.tower: │ │ 232 │ │ │ self.download_workflow_tower() │ │ 233 │ │ else: │ │ ❱ 234 │ │ │ self.download_workflow_static() │ │ 235 │ │ │ 236 │ def download_workflow_static(self): │ │ 237 │ │ """Downloads a nf-core workflow from GitHub to the local file system in a self-c │ │ │ │ /apps/eb/2022b/skylake/software/nf-core/2.10-foss-2022b/lib/python3.10/site-packages/nf_core/dow │ │ nload.py:261 in download_workflow_static │ │ │ │ 258 │ │ │ │ self.find_container_images(os.path.join(self.outdir, revision_dirname)) │ │ 259 │ │ │ │ │ │ 260 │ │ │ │ try: │ │ ❱ 261 │ │ │ │ │ self.get_singularity_images(current_revision=item[0]) │ │ 262 │ │ │ │ except OSError as e: │ │ 263 │ │ │ │ │ raise DownloadError(f"[red]{e}[/]") from e │ │ 264 │ │ │ │ /apps/eb/2022b/skylake/software/nf-core/2.10-foss-2022b/lib/python3.10/site-packages/nf_core/dow │ │ nload.py:1095 in get_singularity_images │ │ │ │ 1092 │ │ │ │ │ │ │ │ # Try other registries │ │ 1093 │ │ │ │ │ │ │ │ log.error(e.message) │ │ 1094 │ │ │ │ │ │ │ │ log.error(e.helpmessage) │ │ ❱ 1095 │ │ │ │ │ │ │ │ if e.error_log.absoluteURI: │ │ 1096 │ │ │ │ │ │ │ │ │ break # there no point in trying other registries i │ │ 1097 │ │ │ │ │ │ │ │ else: │ │ 1098 │ │ │ │ │ │ │ │ │ continue │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ AttributeError: 'ContainerError' object has no attribute 'absoluteURI' [hzs682@rescomp2 hzs682]$ singularity pull --name /well/thompson/users/hzs682/singularity-cashdir/ontresearch-modk it-shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb.img docker://quay.io/ontresearch/modkit:shaeedb131a939d3eea2f9bd4db ecec805c0fa20bdb FATAL: While making image from oci registry: error fetching image to cache: failed to get checksum for docker://quay.io/ontresearch/modkit:shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb: reading manifest shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb in quay.io/ontresearch/modkit: unauthorized: access to the requested resource is not authorizedApplication activity log entry
No response
Were you able to successfully run the latest version of the workflow with the demo data?
no
Other demo data information
No response
The text was updated successfully, but these errors were encountered: