Support for an arbitrary number of CIGAR ops (was: convert2bed can't find sort-bed binary) #157
Comments
|
This error results from not having |
|
yes...I just got this to work on the built-from-source version. In the distributed package, I had added it to path but it wasn't working in that case. For the latter, I'd also run it from the bedops dir with all the executables in it, assuming that |
|
now that that's resolved, another thing happening: |
|
That's a different issue, perhaps a bug. If you can make your BAM file available somewhere, I can use that to figure out if an internal buffer needs resizing. |
|
(That is, accessible where I can download it and run it for local debugging.) |
|
Yep. Working on that. Seeing if I can make something accessible to through |
|
If you can split off a chromosome's worth of the original BAM, and converting that smaller file results in a core dump, that would perhaps be enough to throw onto a public share in a Dropbox folder or similar, unless you need to keep it protected. |
|
it was just the mapped trinity contigs mapped back to a ref genome with gmap, so it's only 23mb. you can find it here: |
|
Thanks for the report. I get a similar error. I'll take a look soon and follow up when I know more. |
|
It looks like I did not allocate enough memory to store large numbers of CIGAR operations. I bumped up the maximum number of operations: https://github.com/bedops/bedops/blob/master/applications/bed/conversion/src/convert2bed.h#L69 This patch is in the v2p4p20 branch, if you want a quick fix. I'll probably push out a master release in the next day or two. |
|
Hi Alex! I'm afraid I'm also having the latter issue, with the latest release bedops_2.4.20.v2 (albeit looking at some 3rd generation mapping data, so my reads and hence CIGARs are looong (over 1.5 million nt in raw read length). Is there any possibility for such monster CIGARs to be handled? Thanks in advance! |
|
Sorry about this. I will need to look at doing something a bit more dynamic with this, it seems, so I'll add this to the to-do pile. In the meantime, if you have a chromosome of your BAM file you can post somewhere for me to download, that would be very useful for testing. |
alexpreynolds
changed the title
|
Investigating https://bitbucket.org/genomeinformatics/simlord/ to simulate long sequencing reads. |
|
Added patch in commit 502726e that includes support for third-generation sequencer reads. Testing was done against simulated reads from SimLoRD, up to 100 kbases in length. If others have test inputs they wish to try, the |
|
hey @alexpreynolds I was testing code on your For nanopore: For pacbio: The installation works perfectly on illumina data (101 nt per read), though. |
|
Thanks for the report. I am using sample Nanopore BAM reads from here to help debug: https://github.com/nanopore-wgs-consortium/NA12878 I did a brief test on OS X 10.12.4 with I'll do some more testing and a patch will probably get put into 2.4.27. If you have PacBio reads on hand, that would be useful for further testing. |
|
Many thanks @alexpreynolds for the reply. Please e-mail me (a.echchiki@gmail.com) to let me know what kind of PacBio data do you need, and which format. I would be happy to help! Cheers |
|
Provisional fix in commit d45974c |
|
Commit 8f8a43c includes fixes for a new splice processing option and the option to reduce output conversion and sorting overhead. I'm going to close this up, but please feel free to follow up if there are further problems. |
|
Hi, I got the same error using bedops v.2.4.30 on Nanopore data I am trying to convert wig to bed format. |
|
The Nanopore data may require larger fields. If you have downloaded the binary version of BEDOPS, then: This will symlink in a build that can support far larger fields, and you can call convert2bed as normal. |
|
I followed your suggestion
but I got the same error |
|
Hi Maria, If possible, I'd like to test out what you're running locally. Would you be able to post your WIG file where I can download it? Regards, |
|
[myfile.wig.gz](https://github.com/bedops/bedops/files/1786354/myfile.wig.gz)
This is my input file.
Thanks for your help.
Maria Angela
|
|
Thanks, I'll investigate as soon as I can and follow up here. |
|
Thanks for the test input. There are chromosome names in your sample WIG file that are longer than what I allot to the buffer for storing chromosome names. The I resized this buffer in v2.4.31 (commit 97bfc2b) to the suite-wide constant and this resolves the segmentation fault for your input, at least from tests on a Mac OS X host. After some more tests, I will push out prebuilt binaries of this new version of BEDOPS in the next day or two, but you could build v2.4.31 from source, if you need new binaries sooner; cf. http://bedops.readthedocs.io/en/latest/content/installation.html#via-source-code Thanks for the report and please let me know if you have any questions. |
|
Thanks for your help. I changed this line |
|
I may need to do more testing, but is the 2.4.31 version you are compiling located at /home/madiroma/bin/bedops or in another directory?
I ask because running wig2bed may be calling the 2.4.30 version of the convert2bed binary, if that is what is found first in your environment's PATH variable, and it is the 2.4.30 version of convert2bed that is crashing.
If you have 2.4.30 and 2.4.31 binaries in separate folders, you might run something like the following:
```
$ /path/to/2.4.31/convert2bed --input=wig < myfile.wig > myfile.bed
```
Calling the new version of convert2bed directly will sidestep wig2bed potentially using an older convert2bed.
The other possibility is that there is still another problem, but when I do more testing on Linux I'll see if I can reproduce this issue on those hosts.
… On Mar 7, 2018, at 08:08, Maria Angela Diroma ***@***.***> wrote:
Thanks for your help. I changed this line
#define C2B_MAX_CHROMOSOME_LENGTH 32
to
#define C2B_MAX_CHROMOSOME_LENGTH TOKEN_CHR_MAX_LENGTH
within bedops/applications/bed/conversion/src/convert2bed.h
Now I got a new error
*** glibc detected *** convert2bed: free(): invalid next size (fast): 0x0000000001f4bdf0 ***
======= Backtrace: =========
[0x4289fa]
[0x42b7cc]
[0x40d40b]
[0x41d93b]
[0x400425]
======= Memory map: ========
00400000-004df000 r-xp 00000000 08:14 3899030 /home/madiroma/bin/bedops/bin/convert2bed
006df000-006e1000 rw-p 000df000 08:14 3899030 /home/madiroma/bin/bedops/bin/convert2bed
006e1000-006e8000 rw-p 00000000 00:00 0
01f45000-01f68000 rw-p 00000000 00:00 0 [heap]
7f9398000000-7f93984e6000 rw-p 00000000 00:00 0
7f93984e6000-7f939c000000 ---p 00000000 00:00 0
7f93a0000000-7f93a04e6000 rw-p 00000000 00:00 0
7f93a04e6000-7f93a4000000 ---p 00000000 00:00 0
7f93a4000000-7f93a44e6000 rw-p 00000000 00:00 0
7f93a44e6000-7f93a8000000 ---p 00000000 00:00 0
7f93a8000000-7f93a84e6000 rw-p 00000000 00:00 0
7f93a84e6000-7f93ac000000 ---p 00000000 00:00 0
7f93ac6bc000-7f93ac89e000 rw-p 00000000 00:00 0
7f93acd63000-7f93acd64000 ---p 00000000 00:00 0
7f93acd64000-7f93ad764000 rw-p 00000000 00:00 0
7f93ad764000-7f93ad765000 ---p 00000000 00:00 0
7f93ad765000-7f93ae165000 rw-p 00000000 00:00 0
7f93ae165000-7f93ae166000 ---p 00000000 00:00 0
7f93ae166000-7f93aeb66000 rw-p 00000000 00:00 0
7f93af566000-7f93af567000 rw-p 00000000 00:00 0
7ffe6c9f4000-7ffe6ca37000 rw-p 00000000 00:00 0 [stack]
7ffe6cbc6000-7ffe6cbc7000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
I followed your link 97bfc2b and then used git clone to download the package https://github.com/bedops/bedops/tree/97bfc2b22b74a1b7e7c1c96eed82b047e8fc1489, but the version installed is 2.4.30. |
|
The current version is 2.4.30. The new version (not yet released as a compiled package) is 2.4.31 and includes the fix described.
I was suggesting manually compiling and using 2.4.31 in case you needed a fixed binary sooner than when I will have a precompiled package ready.
I apologize for any difficulties and should have the new version out today or tomorrow.
… On Mar 7, 2018, at 09:22, Maria Angela Diroma ***@***.***> wrote:
I followed your link 97bfc2b and then used git clone to download the package https://github.com/bedops/bedops/tree/97bfc2b22b74a1b7e7c1c96eed82b047e8fc1489, but the version installed is 2.4.30.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
I understand. I compiled it using make and make install, but probably I
used a wrong link. If you could tell me where to download 2.4.31, I can try
again.
Thanks for your help.
Il 07/mar/2018 18:50, "BEDOPS" <notifications@github.com> ha scritto:
The current version is 2.4.30. The new version (not yet released as a
compiled package) is 2.4.31 and includes the fix described.
I was suggesting manually compiling and using 2.4.31 in case you needed a
fixed binary sooner than when I will have a precompiled package ready.
I apologize for any difficulties and should have the new version out today
or tomorrow.
On Mar 7, 2018, at 09:22, Maria Angela Diroma ***@***.***>
wrote:
I followed your link 97bfc2b and then used git clone to download the
package https://github.com/bedops/bedops/tree/97bfc2b22b74a1b7e7c1c96eed82b0
47e8fc1489, but the version installed is 2.4.30.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#157 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ANJGfp05Ox32D7-0mkLJ_W12ywO4-SY9ks5tcB3KgaJpZM4JQ_Fa>
.
|
|
You might try something like the following: The After I have published an "official" version, you won't need to do any of this; however, these steps may be useful for conversion work until that is available. |
|
Perfect! Thank you very much. Now it works. Best wishes, |
|
Thanks for the bug report and helping make this a better toolkit!
…On Wed, Mar 7, 2018 at 1:04 PM, Maria Angela Diroma < ***@***.***> wrote:
Perfect! Thank you very much. Now it works.
Best wishes,
Maria Angela
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#157 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AFPCbOcSuCk79b2wn8o676k1bGTtibJHks5tcEt2gaJpZM4JQ_Fa>
.
|
In trying to convert a bam file to a bed file....whether I add the bedops dir to path, point specifically to convert2bed, use the pre-built or build from source, I get the same error:
Error: Cannot find sort-bed binary required for sorting BED output, and then the usage help string.
cmd used is ~/software/bedops_distr/convert2bed --input=BAM < sorted_mappedonly_mus_dendtritic_stranded_Trinity.GmapToGenome.bam > test.bed
Also tried opening the bam file and piping to convert2bed...same error.
The text was updated successfully, but these errors were encountered: