[E::bgzf_flush] File write failed (wrong size)

[stevehilo]

I'm running Arcs with Tigmint. It runs about 16 hours and crashes. I'm using Samtools-1.9. Longranger was used to create the interleaved barcoded file from a Chromium run. Taro.contigs.fa is assembled from Nanopore runs using Canu. I'm running on a single node with 20 cores and 128GB RAM.
arcs-make arcs-tigmint draft=Taro.contigs reads=barcoded
End part of the error file is below:
[M::mem_process_seqs] Processed 573478 reads in 662.421 CPU sec, 81.837 real sec
[M::process] 0 single-end sequences; 573478 paired-end sequences
[E::bgzf_flush] [E::bgzf_flush] File write failed (wrong size)[E::bgzf_flush] File write failed (wrong size)
[E::bgzf_flush] File write failed (wrong size)
[E::bgzf_flush] File write failed (wrong size)
[E::bgzf_flush] File write failed (wrong size)
File write failed (wrong size)
[E::bgzf_close] File write failed
[E::bgzf_close] File write failed
[E::bgzf_close] File write failed
[E::bgzf_flush] File write failed (wrong size)
[E::bgzf_flush] File write failed (wrong size)

[E::bgzf_close] File write failed
[E::bgzf_close] File write failed
[E::bgzf_close] File write failed
[E::bgzf_close] File write failed
[E::bgzf_close] File write failed
samtools sort: failed to create temporary file "/tmp/Taro.contigs.barcoded.sortbx.bam.FfJGIn.0224.bam": No space left on device
samtools sort: failed to create temporary file "/tmp/Taro.contigs.barcoded.sortbx.bam.FfJGIn.0225.bam": No space left on device
samtools sort: failed to create temporary file "/tmp/Taro.contigs.barcoded.sortbx.bam.FfJGIn.0226.bam": No space left on device
samtools sort: failed to create temporary file "/tmp/Taro.contigs.barcoded.sortbx.bam.FfJGIn.0227.bam": No space left on device
samtools sort: failed to create temporary file "/tmp/Taro.contigs.barcoded.sortbx.bam.FfJGIn.0228.bam": No space left on device
samtools sort: failed to create temporary file "/tmp/Taro.contigs.barcoded.sortbx.bam.FfJGIn.0229.bam": No space left on device
samtools sort: failed to create temporary file "/tmp/Taro.contigs.barcoded.sortbx.bam.FfJGIn.0230.bam": No space left on device
samtools sort: failed to create temporary file "/tmp/Taro.contigs.barcoded.sortbx.bam.FfJGIn.0231.bam": No space left on device
time user=478289.20s system=444.58s elapsed=59684.76s cpu=802% memory=4870 job=bwa mem -t8 -pC Taro.contigs.fa barcoded.fq.gz
time user=952.14s system=103.13s elapsed=59684.72s cpu=1% memory=10 job=samtools view -u -F4
time user=3901.95s system=245.10s elapsed=59684.72s cpu=6% memory=6619 job=
make[1]: *** [Taro.contigs.barcoded.sortbx.bam] Error 1
make: *** [Taro.contigs.tigmint.fa] Error 2

[lcoombe]

Hi @stevehilo,

I'm assuming this is the bwa mem alignment step from the tigmint step?

$(gtime) bwa mem -t$t -pC $(draft).fa $< | samtools view -u -F4 | samtools sort -@$t -tBX -T$$(mktemp -u -t $@.XXXXXX) -o $@

It looks like you are running out of room in /tmp, which is being used by samtools sort for the intermediate alignment files.
To get around this, you could try setting your $TMPDIR to another directory which has more disk space. Otherwise, you could remove the -T$$(mktemp -u -t $@.XXXXXX) part of that command in the tigmint makefile - then the intermediate files will be created in your current working directory.

BTW - You can use triple back ticks (https://help.github.com/articles/basic-writing-and-formatting-syntax/#quoting-code) for code/log blocks to make them a bit more readable.

Hope that helps!
Lauren

[stevehilo]

Thanks Lauren. I'm giving it a try by removing the -T$$(mktemp -u -t $@.XXXXXX) Steven Starnes University of Hawaii, Hilo

…

On Wed, Jan 30, 2019 at 11:26 AM lcoombe ***@***.***> wrote: Hi @stevehilo <https://github.com/stevehilo>, I'm assuming this is the bwa mem alignment step from the tigmint step? $(gtime) bwa mem -t$t -pC $(draft).fa $< | samtools view -u -F4 | samtools sort -@$t -tBX -T$$(mktemp -u -t ***@***.***) -o $@ It looks like you are running out of room in /tmp, which is being used by samtools sort for the intermediate alignment files. To get around this, you could try setting your $TMPDIR to another directory which has more disk space. Otherwise, you could remove the -T$$(mktemp -u -t ***@***.***) part of that command in the tigmint makefile - then the intermediate files will be created in your current working directory. BTW - You can use triple back ticks ( https://help.github.com/articles/basic-writing-and-formatting-syntax/#quoting-code) for code/log blocks to make them a bit more readable. Hope that helps! Lauren — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#52 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Ag5IkzbRjEg_nv6d5JIFd2Bgd-M2FvGLks5vIg4hgaJpZM4abEHf> .

-- Steven M. Starnes UHH, College of Agriculture Mobile: 808-333-0505

[lcoombe]

Sounds good, Steven!