Error when using singularity - /etc/resolv.conf doesn't exist in container

[yuifu]

Check Documentation

I have checked the following places for your error:

• nf-core website: troubleshooting
• nf-core/fetchngs pipeline documentation

Description of the bug

Command:

nextflow run nf-core/fetchngs --input ids.txt -profile singularity

Error:

Error executing process > 'NFCORE_FETCHNGS:FETCHNGS:SRA_IDS_TO_RUNINFO (SRR7667101)'

Caused by:
  Process `NFCORE_FETCHNGS:FETCHNGS:SRA_IDS_TO_RUNINFO (SRR7667101)` terminated with an error exit status (1)

Command executed:

  echo SRR7667101 > id.txt
  sra_ids_to_runinfo.py \
      id.txt \
      SRR7667101.runinfo.tsv \

Command exit status:
  1

Command output:
  (empty)

executor >  local (6)
[e5/c7daaf] process > NFCORE_FETCHNGS:FETCHNGS:SRA_IDS_TO_RUNINFO (SRR7667099) [100%] 6 of 6, failed: 6, retries: 4 ✘
[-        ] process > NFCORE_FETCHNGS:FETCHNGS:SRA_RUNINFO_TO_FTP              -
[-        ] process > NFCORE_FETCHNGS:FETCHNGS:SRA_FASTQ_FTP                   -
[-        ] process > NFCORE_FETCHNGS:FETCHNGS:SRA_TO_SAMPLESHEET              -
[-        ] process > NFCORE_FETCHNGS:FETCHNGS:SRA_MERGE_SAMPLESHEET           -
[e1/a4ddc5] NOTE: Process `NFCORE_FETCHNGS:FETCHNGS:SRA_IDS_TO_RUNINFO (SRR7667101)` terminated with an error exit status (1) -- Execution is retried (1)
[ab/aab919] NOTE: Process `NFCORE_FETCHNGS:FETCHNGS:SRA_IDS_TO_RUNINFO (SRR7667099)` terminated with an error exit status (1) -- Execution is retried (1)
[b3/e42c10] NOTE: Process `NFCORE_FETCHNGS:FETCHNGS:SRA_IDS_TO_RUNINFO (SRR7667101)` terminated with an error exit status (1) -- Execution is retried (2)
[ee/bafd63] NOTE: Process `NFCORE_FETCHNGS:FETCHNGS:SRA_IDS_TO_RUNINFO (SRR7667099)` terminated with an error exit status (1) -- Execution is retried (2)
Error executing process > 'NFCORE_FETCHNGS:FETCHNGS:SRA_IDS_TO_RUNINFO (SRR7667101)'

Caused by:
  Process `NFCORE_FETCHNGS:FETCHNGS:SRA_IDS_TO_RUNINFO (SRR7667101)` terminated with an error exit status (1)

Command executed:

  echo SRR7667101 > id.txt
  sra_ids_to_runinfo.py \
      id.txt \
      SRR7667101.runinfo.tsv \

Command exit status:
  1

Command output:
  (empty)

Command error:
  WARNING: While bind mounting '/lustre7/home/lustre3/harukao/work/dc/30737a68a705c5750ef5205813c04e:/lustre7/home/lustre3/harukao/work/dc/30737a68a705c5750ef5205813c04e': destination is already in the mount point list
  WARNING: Skipping mount /opt/pkg/singularity/3.7.1/var/singularity/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container
  HTTPSConnectionPool(host='www.ebi.ac.uk', port=443): Max retries exceeded with url: /ena/portal/api/returnFields?dataPortal=ena&format=tsv&result=read_run (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x2aaaac67ae50>: Failed to establish a new connection: [Errno -2] Name or service not known'))

Work dir:
  /lustre7/home/lustre3/harukao/work/dc/30737a68a705c5750ef5205813c04e

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

$ cat ids.txt
SRR7667099
SRR7667101

Steps to reproduce

Steps to reproduce the behaviour:

1. Command line: As above
2. See error: As above

Expected behaviour

• Get FASTQ files for ids in ids.txt.

Log

.nextflow.log https://www.dropbox.com/sh/fiie8ian20zt17f/AACBfnMn1QiMjUi9778NL_jka?dl=0

System

$ cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

Nextflow Installation

$ nextflow -version

      N E X T F L O W
      version 21.04.1 build 5556
      created 14-05-2021 15:20 UTC (15-05-2021 00:20 JDT)
      cite doi:10.1038/nbt.3820
      http://nextflow.io

Container engine

$ singularity --version
singularity version 3.7.1

Additional context

[yuifu]

$ cat /lustre7/home/lustre3/harukao/work/dc/30737a68a705c5750ef5205813c04e/.command.sh
#!/bin/bash -euo pipefail
echo SRR7667101 > id.txt
sra_ids_to_runinfo.py \
    id.txt \
    SRR7667101.runinfo.tsv \

[drpatelh]

Hi @yuifu ! I have pushed a fix that might work for you on a separate branch so you can test it out. Can you try and run the commands below and see if it works for you please?

nextflow pull nf-core/fetchngs
nextflow run --input ids.txt -r requests -profile singularity

[drpatelh]

This should be fixed in v1.1 now @yuifu. Please feel free to re-open if the issue persists.

[ImNotaGit]

@drpatelh I think I ran into a similar error just by running the test as given in the "Quick Start":

nextflow run nf-core/fetchngs -profile test,singularity

Error message:

Error executing process > 'NFCORE_FETCHNGS:FETCHNGS:SRA_IDS_TO_RUNINFO (SRR11140744)'

Caused by:
  Process `NFCORE_FETCHNGS:FETCHNGS:SRA_IDS_TO_RUNINFO (SRR11140744)` terminated with an error exit status (1)

Command executed:

  echo SRR11140744 > id.txt
  sra_ids_to_runinfo.py \
      id.txt \
      SRR11140744.runinfo.tsv \

Command exit status:
  1

Command output:
  (empty)

Command error:
  WARNING: destination ....../fetchngs_test/work/29/11b616463fdee227008f69e864fde0 already in mount list: destination is already in the mount point list
  HTTPSConnectionPool(host='www.ebi.ac.uk', port=443): Max retries exceeded with url: /ena/portal/api/returnFields?dataPortal=ena&format=tsv&result=read_run (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f065d2fdd90>: Failed to establish a new connection: [Errno 111] Connection refused',))

This error doesn't occur when using conda:

nextflow run nf-core/fetchngs -profile test,conda

Nextflow version: 21.04.0
Singularity version: 3.5.3

Thanks!