Assertion `tstart > 0' failed. in overlap.cc

[emreerhan]

While running trans-ABySS v.1.4.10, my team member had an error where an assertion failed in abyss-overlap in ABySS 1.3.5. After checking the log file, I saw that the assertion that failed was assert(tstart > 0) in overlap.cc. I realized that in later versions of ABySS that this assertion was removed, so I simply commented out the assertion and ran trans-ABySS again. With this run, another assertion fails: assert(ep0.distance < ep.distance).

[sjackman]

Thanks for the bug report, Emre. Please include the exact command that you ran and the exact error message, copied and pasted from the log file.

[sjackman]

Are you able to reproduce this error using abyss-overlap from ABySS 2.0.1?
/gsc/btl/linuxbrew/Cellar/abyss/2.0.1-k96/bin/abyss-overlap

[emreerhan]

+ abyss-overlap -v -j20 -m96 merge/indel_bubbles/<removed>
Reading `merge/indel_bubbles/<removed>'...
Reading `merge/indel_bubbles/<removed>'...
Building the suffix array...
Building the Burrows-Wheeler transform...
Building the character occurrence table...
Read 4.34 GB in 809867 contigs.
Using 38.2 GB of memory and 8.79 B/bp.
Reading `merge/indel_bubbles/<removed>'...
abyss-overlap: overlap.cc:200: void addPrefixOverlaps(Graph&, const FastaIndex&, const FMIndex&, const ContigNode&, const Match&): Assertion `ep0.distance < ep.distance' failed.
/PATH_to_transabyss/trans-ABySS/v1.4.10/code/wrappers/abyss-rmdups-iterative: line 281: 23277 Aborted                 (core dumped) abyss-overlap -v -j$threads -m$k $out_file.incomplete > $out_file.incomplete.dot

real    122m3.661s
user    155m27.943s
sys     14m46.354s

So the exact command is: abyss-overlap -v -j20 -m96 merge/indel_bubbles/<removed> and the error is Assertion 'ep0.distance < ep.distance' failed.

[benvvalk]

Thanks for your help, @sjackman

[emreerhan]

I will try using ABySS 2.0.1. I'll run TA now, it should take about 2-3 hours.

[benvvalk]

This is the function in abyss-overlap that is causing the trouble. (@emreerhan You said the version of abyss-overlap you were running was 1.3.5, correct?)

Unfortunately, I don't understand what it's doing.

static void addPrefixOverlaps(Graph &g,
                const FastaIndex& faIndex, const FMIndex& fmIndex,
                const ContigNode& v, const Match& fmi)
{
        typedef edge_property<Graph>::type EP;
        typedef graph_traits<Graph>::edge_descriptor E;
        typedef graph_traits<Graph>::vertex_descriptor V;

        assert(v.sense());
        assert(fmi.qstart == 0);
        Distance ep(-fmi.qspan());
        assert(ep.distance < 0);
        for (unsigned i = fmi.l; i < fmi.u; ++i) {
                size_t toffset = fmIndex[i];
                FastaIndex::SeqPos seqPos = faIndex[toffset];
                const FAIRecord& tseq = seqPos.get<0>();
                size_t tstart = seqPos.get<1>();
                size_t tend = tstart + fmi.qspan();
                assert(tstart > 0);
                if (tend < tseq.size) {
                        // This match is due to an ambiguity code in the target sequence.

[emreerhan]

That is correct.

[sjackman]

It fails with ABySS 2.0.1 as well.
/projects/btl/sjackman/abyss-overlap/make-1.log

[emreerhan]

Assertion `tend <= tseq.size' failed.
<removed> line 281: 45922 Aborted

This is the error I get with ABySS 2.0.1

[sjackman]

I won't have time to troubleshoot this error this week as we're working on revising the ABySS 2.0 manuscript. Remind me to look at this error next week.

[sjackman]

In the mean time as a workaround, you could probably skip this step of Trans-ABySS pipeline.

[benvvalk]

I asked @kmnip (Trans-ABySS veteran) and he said he has seen this error on rare occasions but doesn't know the root cause (or have any workarounds).

@emreerhan, is this is a time-critical job?

Unfortunately, I'm in the same boat as Shaun. I will try to identify the problem sequence by adding some debugging statements to the code, when I can steal some time. Perhaps if it is only one sequence that is causing the problem we can manually remove it.

[emreerhan]

The deadline is TBC right now, so we do have some time.

Thanks for the help.

[emreerhan]

Hi Ben and Shaun. If you do not have time to debug this error this week, we will use our other tools to complete the analysis. If you do get around to fixing this however, it would be good for us to know for the future, in case it happens again.

[benvvalk]

@emreerhan

I will likely have time on Wed and will have a look then. Thanks for your patience.

[emreerhan]

Great, thanks @benvvalk!

[benvvalk]

Sorry @emreerhan, I need one more day. Will look tomorrow morning.

[emreerhan]

For the record, even after reassembling the genome, trans-abyss still gives the same error.

[calchoo]

Hi guys, we recently got this error in 3 more genome analysis runs. Do you guys have some time to look at this again?

[benvvalk]

I won't be able to spend a lot of time on it, but I can take another stab at least. To be honest, I'm not very optimistic because I'm not familiar with this particular code or with FM indices in general.

I would need an example command that crashes and access to the input files in order to debug it (like Emre provided before). You can send that info in a private e-mail if the data is sensitive (benv / sjackman at bcgsc).

Btw, I will be away Thurs-Fri this week.

[emreerhan]

@calchoo, try using pavfinder

[benvvalk]

@calchoo: I looked at this a bit yesterday but I will need more time. I will allocate some more time for it next week.

I was indeed able to able to reproduce the crash using the files you provided me by e-mail. Also, I did some background reading about FM indices which should help.

I'll let you know if I make any progress.

[sjackman]

Shot in the dark here, but here goes nothing. abyss-overlap may be asserting that every overlap is a strict overlap, whereas in this FASTA file one sequence is a prefix of another sequence. So not a dovetail overlap as expected, but contained within another sequence at the very start of it.

[benvvalk]

Thank you, @sjackman. I will try logging out the offending sequences and see if it something like that.

[benvvalk]

@calchoo I'm sorry, it does not look like I will be able to fix this.

I spent yesterday afternoon and all day today debugging it, and although I understand the code much better, the root cause of the problem is still not clear. I also got some help from @sjackman (the original author) for a couple of hours, but unfortunately we were not able to solve it.

I may try a few more things next week, but I do not plan to spend much more time on it, unless Inanc/Rene say otherwise.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.