New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error running BWA with Docker #103

Closed
jpdna opened this Issue Feb 7, 2018 · 5 comments

Comments

Projects
None yet
2 participants
@jpdna
Member

jpdna commented Feb 7, 2018

I'm trying to run cannoli bwa with the command:

../cannoli/bin/cannoli-submit bwa data/test1_small.bam out8 mysample -index ../bwa_index/hg19.fa -use_docker

Note, I aded below "#### bwaCommand" to print out that value.

I get the error that the docker command is badly formed and doesn't know about "--rm":

./cannoli/bin/cannoli-submit bwa data/test1_small.bam out8 mysample -index ../bwa_index/hg19.fa -use_docker
Using SPARK_SUBMIT=/Users/paschalj/test9/spark/spark2/spark-2.1.1-bin-hadoop2.7/bin/spark-submit
2018-02-07 10:35:06 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
#### This is bwaCommand: docker -v ../bwa_index:../bwa_index run --rm quay.io/biocontainers/bwa:0.7.17--pl5.22.0_0 mem -t 1 -R @RG\tID:mysample\tLB:mysample\tPL:ILLUMINA\tPU:0\tSM:mysample -p ../bwa_index/hg19.fa -
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Feb 7, 2018 10:35:20 AM INFO: org.apache.parquet.hadoop.codec.CodecConfig: Compression: GZIP
Feb 7, 2018 10:35:20 AM INFO: org.apache.parquet.hadoop.codec.CodecConfig: Compression: GZIP
Feb 7, 2018 10:35:20 AM INFO: org.apache.parquet.hadoop.ParquetOutputFormat: Parquet block size to 134217728
Feb 7, 2018 10:35:20 AM INFO: org.apache.parquet.hadoop.ParquetOutputFormat: Parquet page size to 1048576
Feb 7, 2018 10:35:20 AM INFO: org.apache.parquet.hadoop.ParquetOutputFormat: Parquet dictionary page size to 1048576
Feb 7, 2018 10:35:20 AM INFO: org.apache.parquet.hadoop.codec.CodecConfig: Compression: GZIP
...
...
Feb 7, 2018 10:35:23 AM INFO: org.apache.parquet.hadoop.ParquetOutputFormat: Dictionary is on
Feb 7, 2018 10:35:23 AM INFO: org.apache.parquet.hadoop.ParquetOutputFormat: Validation is off
Feb 7, 2018 10:35:23 AM INFO: org.apache.parquet.hadoop.ParquetOutputFormat: Writer version is: PARQUET_1_0
Feb 7, 2018 10:35:23 AM INFO: org.apache.parquet.hadoop.ParquetOutputFormat: Maximum row group padding size is 0 bytes
Feb 7, 2018 10:35:23 AM INFO: org.apache.parquet.hadoop.codec.CodecConfig: Compression: GZIP
Feb 7, 2018 10:35:23 AM INFO: org.apache.parquet.hadoop.ParquetOutputFormat: Parquet block size to 134217728
Feb 7, 2018 10:35:23 AM Iunknown flag: --rm
See 'docker --help'.

Usage:	docker COMMAND

A self-sufficient runtime for containers

Options:
      --config string      Location of client config files (default
                           "/Users/paschalj/.docker")
  -D, --debug              Enable debug mode
      --help               Print usage
  -H, --host list          Daemon socket(s) to connect to
  -l, --log-level string   Set the logging level
                           ("debug"|"info"|"warn"|"error"|"fatal")
                           (default "info")
      --tls                Use TLS; implied by --tlsverify
      --tlscacert string   Trust certs signed only by this CA (default
                           "/Users/paschalj/.docker/ca.pem")
      --tlscert string     Path to TLS certificate file (default
                           "/Users/paschalj/.docker/cert.pem")
...

Run 'docker COMMAND --help' for more information on a command.

Exception in thread "Thread-185" java.io.IOException: Broken pipe
	at java.io.FileOutputStream.writeBytes(Native Method)
	at java.io.FileOutputStream.write(FileOutputStream.java:326)
	at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
	at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
	at java.io.FilterOutputStream.write(FilterOutputStream.java:97)
	at org.bdgenomics.adam.rdd.fragment.InterleavedFASTQInFormatter$$anonfun$write$2.apply(InterleavedFASTQInFormatter.scala:91)
	at org.bdgenomics.adam.rdd.fragment.InterleavedFASTQInFormatter$$anonfun$write$2.apply(InterleavedFASTQInFormatter.scala:71)
	at scala.collection.Iterator$class.foreach(Iterator.scala:893)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
	at org.bdgenomics.adam.rdd.fragment.InterleavedFASTQInFormatter.write(InterleavedFASTQInFormatter.scala:71)
	at org.bdgenomics.adam.rdd.InFormatterRunner.run(InFormatter.scala:27)
	at java.lang.Thread.run(Thread.java:745)
2018-02-07 10:35:26 ERROR Utils:91 - Aborting task
java.lang.RuntimeException: Piped command List(docker, -v, ../bwa_index:../bwa_index, run, --rm, quay.io/biocontainers/bwa:0.7.17--pl5.22.0_0, mem, -t, 1, -R, @RG\tID:mysample\tLB:mysample\tPL:ILLUMINA\tPU:0\tSM:mysample, -p, ../bwa_index/hg19.fa, -) exited with error code 125.

which is the same error I get if I try to run manually

docker -v /Users/paschalj/test9/singularity/v1/run1/bwa_index:/Users/paschalj/test9/singularity/v1/run1/bwa_index -v /Users/paschalj/test9/singularity/v1/run1/data/:/Users/paschalj/test9/singularity/v1/run1/data/ --rm quay.io/biocontainers/bwa:0.7.17--pl5.22.0_0 mem -t 1 -R @RG\tID:mysample\tLB:mysample\tPL:ILLUMINA\tPU:0\tSM:mysample -p  /Users/paschalj/test9/singularity/v1/run1/bwa_index/hg19.fa data/test1_small.bam

There may be also an issue that I should be using abosulte paths, but I think that is downstream.
The version of docker I am using on a mac is: Version 17.06.0-ce-mac19 (18663)

@heuermh

This comment has been minimized.

Member

heuermh commented Feb 7, 2018

That's a fairly old version of Docker.

$ docker --version
Docker version 18.01.0-ce, build 03596f5

From the manual

--rm | Automatically remove the container when it exits

https://docs.docker.com/engine/reference/commandline/run/#options

@heuermh

This comment has been minimized.

Member

heuermh commented Feb 14, 2018

ok to close?

@jpdna

This comment has been minimized.

Member

jpdna commented Feb 14, 2018

This is not a problem for me any more, but I would note that I can't seem to find docker version 18+ on mac from the Docker site, do you have it on mac?

@heuermh

This comment has been minimized.

Member

heuermh commented Feb 14, 2018

@jpdna

This comment has been minimized.

Member

jpdna commented Feb 14, 2018

Ah ,cool, solved.

@jpdna jpdna closed this Feb 14, 2018

@heuermh heuermh added this to the 0.2.0 milestone Feb 14, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment