multiBamSummary cannot find an existing file #1059
Description
Hello,
I am using deeptools for my ChIPseq data and was following the instructions provided on the wiki page. Unfortunately it does not seem to be able to read a file that does exist. I did see a similar issue (issue #663), however, it did not provide an answer to my problem and was regarding a different script. I am using multiBamSummary to compare four BAM files that correspond to the same chromatin marker. These files are in directories corresponding to their marker (so four BAM and BAM.bai files in each directory).
My script:
`#!/bin/bash/
SEI5079="/home/alejandro/data/tmp/chip_SEI5079"
SEI5080="/home/alejandro/data/tmp/chip_SEI5080"
out="/home/alejandro/data/deeptools"
for dir in $SEI5079/* ; do
dirname=echo $dir | awk -F/ '{print $NF}'
multiBamSummary bins -b $dir/* -o $out/multiBamSummary/SEI5079/$dirname --numberOfProcessors 8
plotCorrelation -in $out/multiBamSummary/SEI5079/$dirname -o $out/plotCorrelation/SEI5079/$dirname -p heatmap
--corMethod spearman
done
for dir in $SEI5080/* ; do
dirname=echo $dir | awk -F/ '{print $NF}'
multiBamSummary bins -b $dir/* -o $out/multiBamSummary/SEI5080/$dirname --numberOfProcessors 8
plotCorrelation -in $out/multiBamSummary/SEI5080/$dirname -o $out/plotCorrelation/SEI5080/$dirname -p heatmap
--corMethod spearman
done
`
I installed deeptools with conda. I first used the latest version 3.5.1 with Python 3.8.8. I mentioned this to my supervisor who uses version 3.1.3 and Python 2.7.4. He suggested me to use those versions instead in a new environment, but I get the same error as far as I can tell.
With deeptools version 3.5.1 and Python 3.8.8:
(sequence_manipulation) alejandro@wildtype2:[SCREEN]:~:bash run_deeptools.sh [E::hts_hopen] Failed to open file /home/alejandro/data/tmp/chip_SEI5079/SEI5079A10PA1/D4AR010_H2C2TBGXG_S10_L001_R1_001_trimmed.bam.bai [E::hts_open_format] Failed to open file "/home/alejandro/data/tmp/chip_SEI5079/SEI5079A10PA1/D4AR010_H2C2TBGXG_S10_L001_R1_001_trimmed.bam.bai" : Exec format error [E::hts_hopen] Failed to open file /home/alejandro/data/tmp/chip_SEI5079/SEI5079A10PA1/D4AR010_H2C2TBGXG_S10_L001_R1_001_trimmed.bam.bai [E::hts_open_format] Failed to open file "/home/alejandro/data/tmp/chip_SEI5079/SEI5079A10PA1/D4AR010_H2C2TBGXG_S10_L001_R1_001_trimmed.bam.bai" : Exec format error The file '/home/alejandro/data/tmp/chip_SEI5079/SEI5079A10PA1/D4AR010_H2C2TBGXG_S10_L001_R1_001_trimmed.bam.bai' does not exist Traceback (most recent call last): File "/home/alejandro/anaconda3/envs/sequence_manipulation/bin/plotCorrelation", line 12, in <module> main(args) File "/home/alejandro/anaconda3/envs/sequence_manipulation/lib/python3.8/site-packages/deeptools/plotCorrelation.py", line 210, in main corr = Correlation(args.corData, File "/home/alejandro/anaconda3/envs/sequence_manipulation/lib/python3.8/site-packages/deeptools/correlation.py", line 43, in __init__ self.load_matrix(matrix_file) File "/home/alejandro/anaconda3/envs/sequence_manipulation/lib/python3.8/site-packages/deeptools/correlation.py", line 85, in load_matrix _ma = np.load(matrix_file) File "/home/alejandro/anaconda3/envs/sequence_manipulation/lib/python3.8/site-packages/numpy/lib/npyio.py", line 417, in load fid = stack.enter_context(open(os_fspath(file), "rb")) FileNotFoundError: [Errno 2] No such file or directory: '/home/alejandro/data/deeptools/multiBamSummary/SEI5079/SEI5079A10PA1'
With deeptools version 3.1.3 and Python 2.7.4:
(deeptools) alejandro@wildtype2:[SCREEN]:~:ls -lah /home/alejandro/data/alignment/star/chip_SEI5079/SEI5079A15PA1/E6AR015_H2C2TBGXG_S15_L001_R1_001_trimmed.bam.bai -rw-r--r-- 1 alejandro binf 214K Apr 7 14:14 /home/alejandro/data/alignment/star/chip_SEI5079/SEI5079A15PA1/E6AR015_H2C2TBGXG_S15_L001_R1_001_trimmed.bam.bai (deeptools) alejandro@wildtype2:[SCREEN]:~:bash run_deeptools.sh [E::hts_hopen] Failed to open file /home/alejandro/data/tmp/chip_SEI5079/SEI5079A10PA1/D4AR010_H2C2TBGXG_S10_L001_R1_001_trimmed.bam.bai [E::hts_open_format] Failed to open file "/home/alejandro/data/tmp/chip_SEI5079/SEI5079A10PA1/D4AR010_H2C2TBGXG_S10_L001_R1_001_trimmed.bam.bai" : Exec format error [E::hts_hopen] Failed to open file /home/alejandro/data/tmp/chip_SEI5079/SEI5079A10PA1/D4AR010_H2C2TBGXG_S10_L001_R1_001_trimmed.bam.bai [E::hts_open_format] Failed to open file "/home/alejandro/data/tmp/chip_SEI5079/SEI5079A10PA1/D4AR010_H2C2TBGXG_S10_L001_R1_001_trimmed.bam.bai" : Exec format error The file '/home/alejandro/data/tmp/chip_SEI5079/SEI5079A10PA1/D4AR010_H2C2TBGXG_S10_L001_R1_001_trimmed.bam.bai' does not exist Traceback (most recent call last): File "/home/alejandro/anaconda3/envs/deeptools/bin/plotCorrelation", line 12, in <module> main(args) File "/home/alejandro/anaconda3/envs/deeptools/lib/python2.7/site-packages/deeptools/plotCorrelation.py", line 213, in main skip_zeros=args.skipZeros) File "/home/alejandro/anaconda3/envs/deeptools/lib/python2.7/site-packages/deeptools/correlation.py", line 41, in __init__ self.load_matrix(matrix_file) File "/home/alejandro/anaconda3/envs/deeptools/lib/python2.7/site-packages/deeptools/correlation.py", line 83, in load_matrix _ma = np.load(matrix_file) File "/home/alejandro/anaconda3/envs/deeptools/lib/python2.7/site-packages/numpy/lib/npyio.py", line 422, in load fid = open(os_fspath(file), "rb") IOError: [Errno 2] No such file or directory: '/home/alejandro/data/deeptools/multiBamSummary/SEI5079/SEI5079A10PA1'
I have checked the paths and they are not faulty:
alejandro@wildtype2:~:ls -lah /home/alejandro/data/tmp/chip_SEI5079/SEI5079A10PA1/D4AR010_H2C2TBGXG_S10_L001_R1_001_trimmed.bam.bai -rw-r--r-- 1 alejandro binf 110K Apr 21 13:19 /home/alejandro/data/tmp/chip_SEI5079/SEI5079A10PA1/D4AR010_H2C2TBGXG_S10_L001_R1_001_trimmed.bam.bai alejandro@wildtype2:~:
I generated the alignments with STAR, and the indexes were made using samtools. If there is any more info I need to provide, please let me know.