ValueError #7
Comments
|
Okay, I increased the threads and now I'm not getting allocate error anymore, but I get this: -bash-4.2$ alignqc analyze aln.bam -g ../Mus_musculus.GRCm38.cdna.all.fa -t ../UCSC_Main_on_Mouse__all_mrna.gtf.gz -o report.xhtml --portable_output report.portable.xhtml --threads 10 check for best set Get ready for alignment plot Killed and the only output that I see in my directory is Rplots.pdf |
|
Hi @rojinsafavi |
|
Thanks Jason! I will run it on a single thread and will report the result to you if I still get an error |
|
Hi Jason, I ran it again with 1 thread, and I got the same error (OSError: [Errno 12] Cannot allocate memory). I have to mention that I'm only testing only 15 fast5 files here ( just for testing purposes). -bash-4.2$ alignqc analyze trial-fast5/aln.bam -g Mus_musculus.GRCm38.cdna.all.fa -t UCSC_Main_on_Mouse__all_mrna.gtf.gz -o report.xhtml --portable_output report.portable.xhtml --output_folder alignqc-output --threads 1 Using Rscript version: check for best set Get ready for alignment plot 536 alignments, 2499 min context coverage 546 alignments, 2499 min context coverage Exception in thread Thread-4:ext coverage Exception in thread Thread-1:ext coverage Killedignments, 3072 min context coverage |
|
Thanks for the error details. Sampling the context errors can also be a little unfriendly when it comes to memory and that looks like where you are running into trouble. The easiest way to fix this it to sample less.
will reduce the depth of sampling that is done (I think default is 2500), but the plot it generates should be pretty representative (unless your error rates are super-low). |
|
Thanks Jason, I think the main reason that I was getting that error was because I was using toplevel reference genome. I was able to overcome that issue by using gencode.vM16.primary_assembly.annotation.gtf.gz and GRCm38.primary_assembly.genome.fa. But now I'm getting the same error as this issue : #6 Sorting in reference genePred since you asked for the GFT file, I'm gonna attach it here: ftp://ftp.sanger.ac.uk/pub/gencode/Gencode_mouse/release_M16/gencode.vM16.primary_assembly.annotation.gtf.gz |
|
Hi Jason, |
|
Hi @rojinsafavi Sorry for the delay. I'm a busy these days so if lose track of these I appreciate getting the reminder :) It looks like a problem streaming the data. Is your alignment file sorted by genomic position? If they are ... I have a second more complicated problem that this may be due to. If they are supported by position, do you know if the index of your chromosomes are in alphabetical order? I notice that different aligners have different behaviors when it comes to sorting and sometimes they sort chromosomes alphabetically ... sometimes they do other things. And i may be making the alphabetical assumption in the ordering-check. Something you can try is my sort tool thats in seqtools
If this is the cause I may rethink my order check because I don't want to require another sort before running. Thanks for your help in figuring this out. |
|
Thanks Jason! |
|
Thanks for posting the file. Problem with samtools sort ... well some would probably consider it a feature not a problem, is that the order of chromosomes in the file is defined by the samfile header and its not actually sorted by any criteria other than the samfile header. So my assumption of order based on alphabet doesn't work if the sam header was sorted otherwise. Since samtools is the gold standard for sam management I'd be better off changing my sorting behavior to fit its convention. In the meantime I'd suggest you use my sorting function I mentioned above. it uses samtools sort to do the sort, but before that, it sorts the header to be sorted. You can see the difference if you do a samtools sort and a seq-tools sort --bam, and then you inspect the outputs with |
|
Hi @rojinsafavi I did some testing this morning to try to narrow down the problem. What I expected to be an issued turned out not to be. I guess my sorts at the beginning of the run mitigate that problem, so I closed the issue I had opened on that. I used the test files you sent me. ... the mouse transcriptome and the aln.bam, and I was able to generate to run without error both on my mac os computer and from the docker. With those files in a
Next steps I would suggest is that you also try running from the Docker, and see if you still get the same error you reported before, with the test data you sent me. If not, I will probably need some test data that I can use to replicate the error, and then I can track down more whats going on. Also open to any suggestion if you think you have any idea what is the cause, but sometimes it can be hard to track down without being able to replicate. Thanks! |
|
Hi @rojinsafavi Just following up because I was looking through my issues for anything else I can help with and I saw another conversation i had where a streaming error occured with genepred |
|
Hi Jason,
and I was able to reproduce the result you made For some reasons the same thing won't works on our server, and It gave me the error that I sent you. But I was able to save the plots by providing a temp dict ( only for a small subsample) Our server does not let us use docker installation, I have to check with the admin to see if they can install it on our sever. But I did conda installation and again I got the same error ( but was able to save the plots in the temp dict for that small subsample) But, I was not able to even save the plots for all my data ( I have about 400,000 fast5 files), how many files do you usually use to get a good approximation? I think I might be using too many files! Kind regards, |
|
Hi Jason! So I was not able to run alignQC on our linux servers, but I managed to run it on my mac (I installed it through pip), it took a while but it worked with no error. Thanks alot for the help, and hope you have a great holiday! |
Hello,
I want to use ailgnQC to analyze some nanopore RNA data, but I keep getting this allocation error:
alignqc analyze aln.bam -g ../Mus_musculus.GRCm38.cdna.all.fa -t ../UCSC_Main_on_Mouse__all_mrna.gtf.gz -o report.xhtml --portable_output report.portable.xhtml --threads 10
Exception in thread Thread-4:ext coverage
Traceback (most recent call last):
File "/projects/nanopore-working/rojin/nanoraw-signalAlign-nanopolish/anaconda2/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/projects/nanopore-working/rojin/nanoraw-signalAlign-nanopolish/anaconda2/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/projects/nanopore-working/rojin/nanoraw-signalAlign-nanopolish/anaconda2/lib/python2.7/multiprocessing/pool.py", line 326, in _handle_workers
pool._maintain_pool()
File "/projects/nanopore-working/rojin/nanoraw-signalAlign-nanopolish/anaconda2/lib/python2.7/multiprocessing/pool.py", line 230, in _maintain_pool
self._repopulate_pool()
File "/projects/nanopore-working/rojin/nanoraw-signalAlign-nanopolish/anaconda2/lib/python2.7/multiprocessing/pool.py", line 223, in _repopulate_pool
w.start()
File "/projects/nanopore-working/rojin/nanoraw-signalAlign-nanopolish/anaconda2/lib/python2.7/multiprocessing/process.py", line 130, in start
self._popen = Popen(self)
File "/projects/nanopore-working/rojin/nanoraw-signalAlign-nanopolish/anaconda2/lib/python2.7/multiprocessing/forking.py", line 121, in init
self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory
Exception in thread Thread-1:
Traceback (most recent call last):
File "/projects/nanopore-working/rojin/nanoraw-signalAlign-nanopolish/anaconda2/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/projects/nanopore-working/rojin/nanoraw-signalAlign-nanopolish/anaconda2/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/projects/nanopore-working/rojin/nanoraw-signalAlign-nanopolish/anaconda2/lib/python2.7/multiprocessing/pool.py", line 326, in _handle_workers
pool._maintain_pool()
File "/projects/nanopore-working/rojin/nanoraw-signalAlign-nanopolish/anaconda2/lib/python2.7/multiprocessing/pool.py", line 230, in _maintain_pool
self._repopulate_pool()
File "/projects/nanopore-working/rojin/nanoraw-signalAlign-nanopolish/anaconda2/lib/python2.7/multiprocessing/pool.py", line 223, in _repopulate_pool
w.start()
File "/projects/nanopore-working/rojin/nanoraw-signalAlign-nanopolish/anaconda2/lib/python2.7/multiprocessing/process.py", line 130, in start
self._popen = Popen(self)
File "/projects/nanopore-working/rojin/nanoraw-signalAlign-nanopolish/anaconda2/lib/python2.7/multiprocessing/forking.py", line 121, in init
self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory
Killedignments, 3213 min context coverage
I would really appreciate if you can help me with that
The text was updated successfully, but these errors were encountered: