2.13.0 dockerhub image fails

[keiranmraine]

Hi,

I'm getting the following on downloading the latest release on dockerhub:

$ echo $IMAGE_NAME
gridss/gridss:2.13.0
$ docker run --rm -v $PWD/work:/data/work:rw $IMAGE_NAME gridss --steps preprocess,assemble,call --externalaligner --reference work/ref/genome.fa --blacklist work/ref/gridss/blacklist-2011-05-04-ENCFF001TDO.bed --threads 8 --labels $WT --skipsoftcliprealignment --assembly work/result/$WT.assembly.bam --output work/result/$WT.gridss.vcf.gz --workingdir work/workspace work/$WT.sample.dupmarked.bam
Using working directory "work/workspace"
Mon Dec  6 11:45:13 UTC 2021: Full log file is: work/workspace/gridss.full.20211206_114513.eede6af1f171.1.log
Mon Dec  6 11:45:13 UTC 2021: Found /usr/bin/time
Mon Dec  6 11:45:13 UTC 2021: Using GRIDSS jar /opt/gridss/gridss-2.13.0-gridss-jar-with-dependencies.jar
...
Mon Dec  6 11:45:13 UTC 2021: samtools version: 1.10+htslib-1.10.2-3
Mon Dec  6 11:45:13 UTC 2021: samtools 1.13 or later is required.
"exit $EX_CONFIG" command completed with exit code 78.
*****
The underlying error message can be found in work/workspace/gridss.full.20211206_114513.eede6af1f171.1.log
*****

[keiranmraine]

I suspect this is due to samtools being installed from system packages:

gridss/Dockerfile

Line 56 in 472bbc8

samtools \

gridss/Dockerfile

Line 104 in 472bbc8

samtools \

[keiranmraine]

The same appears to occur with a locally built image (method as per #548). I made the following (rapid) changes to the dockerfile and the samtools is now accepted:

diff --git a/Dockerfile b/Dockerfile
index debef817..6d614570 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -25,6 +25,9 @@ RUN mkdir /opt/gridss/
 ARG GRIDSS_VERSION
 COPY src/main/c /opt/gridss/src/main/c
 COPY src/test/resources /opt/gridss/src/test/resources
+RUN cd /opt/gridss/src/main/c/gridsstools && make clean || true
+RUN cd /opt/gridss/src/main/c/gridsstools/htslib && make clean || true
+COPY src/main/c/gridsstools/htslib /opt/gridss/src/main/c/gridsstools/htslib
 RUN cd /opt/gridss/src/main/c/gridsstools/htslib && \
        autoreconf -i && ./configure && make -j 8 && \
        cd .. && \
@@ -38,7 +41,7 @@ WORKDIR /opt/gridss/
 COPY pom.xml /opt/gridss/
 COPY repo /opt/gridss/repo
 #RUN mvn -Dmaven.artifact.threads=8 dependency:go-offline
-# run all stages so all dependencies are 
+# run all stages so all dependencies are
 RUN mvn -Dmaven.artifact.threads=8 verify && rm -rf target
 # Build GRIDSS jar
 ARG GRIDSS_VERSION
@@ -53,7 +56,6 @@ RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
                apt-utils \
                gawk \
                openjdk-11-jre-headless \
-               samtools \
                bwa \
                time \
                libssl1.1 \
@@ -64,10 +66,11 @@ RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
                liblzma5 \
                libdeflate0 \
                libomp5 \
+               curl make gcc zlib1g-dev libbz2-dev liblzma-dev libcurl4-gnutls-dev libncurses5-dev \
        && rm -rf /var/lib/apt/lists/*
 # Hack a fake Rscript so that insert size metrics dont break when creating the histogram
 RUN echo "!/bin/sh" > /usr/local/bin/Rscript && chmod +x /usr/local/bin/Rscript
-ENV PATH="/opt/gridss/:$PATH"
+ENV PATH="/opt/gridss/:/opt/samtools/bin:$PATH"
 # Install GRIDSS
 ARG GRIDSS_VERSION
 ENV GRIDSS_VERSION=${GRIDSS_VERSION}
@@ -82,6 +85,12 @@ COPY --from=gridss_builder_c /opt/gridss/gridsstools /opt/gridss/
 COPY --from=gridss_builder_java /opt/gridss/gridss-${GRIDSS_VERSION}-gridss-jar-with-dependencies.jar /opt/gridss/
 COPY scripts/gridss /opt/gridss/
 RUN chmod +x /opt/gridss/*
+RUN cd /tmp && \
+ curl -sSL https://github.com/samtools/samtools/releases/download/1.13/samtools-1.13.tar.bz2 | tar -xj && \
+ cd samtools* && \
+ ./configure --prefix=/opt/samtools && \
+ make -j4 all && make install && \
+ cd .. && rm -rf samtools*
 WORKDIR /data/
 
 
@@ -101,7 +110,6 @@ RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
                apt-utils \
                gawk \
                openjdk-11-jre-headless \
-               samtools \
                bwa \
                hmmer \
                bedtools \
@@ -155,7 +163,7 @@ RUN mkdir /opt/kraken2 && \
        rm -r kraken2-$KRAKEN_VERSION v*.tar.gz
 RUN sh -c "$(wget -q ftp://ftp.ncbi.nlm.nih.gov/entrez/entrezdirect/install-edirect.sh -O -)" && \
        mv $HOME/edirect /opt/edirect
-ENV PATH="/opt/gridss/:/opt/RepeatMasker:/opt/rmblast/:/opt/trf:/opt/kraken2:/opt/blast:/opt/edirect:$PATH"
+ENV PATH="/opt/gridss/:/opt/samtools/bin:/opt/RepeatMasker:/opt/rmblast/:/opt/trf:/opt/kraken2:/opt/blast:/opt/edirect:$PATH"
 # configure repeatmasker
 RUN cd /opt/RepeatMasker && \
        perl configure \
@@ -172,8 +180,9 @@ LABEL software.version="$GRIDSS_VERSION"
 LABEL about.summary="Genomic Rearrangement IDentification Software Suite"
 LABEL about.home="https://github.com/PapenfussLab/gridss"
 LABEL about.tags="Genomics"
-RUN mkdir /opt/gridss/ /data
+RUN mkdir /opt/gridss/ /data /opt/samtools/
 COPY --from=gridss_builder_c /opt/gridss/gridsstools /opt/gridss/
+COPY --from=gridss_minimal /opt/samtools /opt/samtools
 COPY --from=gridss_builder_java /opt/gridss/gridss-${GRIDSS_VERSION}-gridss-jar-with-dependencies.jar /opt/gridss/
 COPY scripts/gridss \
        scripts/gridss_annotate_vcf_kraken2 \

[keiranmraine]

The above edits allow completion:

Mon Dec  6 18:30:18 UTC 2021: Complete calling	work/result/XXXXXX.gridss.vcf.gz
Mon Dec  6 18:30:18 UTC 2021: Run complete with 57 warnings and 0 errors.

[afurches]

Hi, I'm still getting samtools v1.10 errors with this newest docker (see my comment here).
Thanks,
Anna

[afurches]

@d-cameron @keiranmraine Are there plans to release an updated docker in which the samtools version error is fixed? Newb here, I edited the dockerfile but don't know how to rebuild a custom container (I'm using singularity to implement docker gridss on HPC without sudo for extra layer of complexity). Any help appreciated!

[keiranmraine]

@afurches we use singularity in the same way (doesn't really seem to cause us any real issues). I personally have no control of releases, just a consumer like you.

[afurches]

@keiranmraine to implement your fix above, would I need to:
(1) rebuild a custom gridss docker using the edited Dockerfile per your example above,
(2) push it to my own repository on docker,
(3) use singularity to build the image using my custom gridss docker?

If so, would you mind sharing tips for how to accomplish (1) and (2)? (Sorry if that's asking too much -- I'm a field/wet lab biologist by training who is a bit lost in the woods here.)

[d-cameron]

Are there plans to release an updated docker in which the samtools version error is fixed?

I can do that. There's a few new features I'm implementing but I can split it up and push out a patch update (2.13.1) with the bug fixes before christmas and move the new features into a 2.14.0 to be released in the first week of Feb.

[afurches]

I can do that. There's a few new features I'm implementing but I can split it up and push out a patch update (2.13.1) with the bug fixes before christmas and move the new features into a 2.14.0 to be released in the first week of Feb.

@d-cameron thank you! After rerunning the reference and preprocessing steps, the invalid file pointer error messages have disappeared and the assembly step is running without issue.