Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

vg augment crashed #2502

Open
verne91 opened this issue Oct 10, 2019 · 9 comments
Open

vg augment crashed #2502

verne91 opened this issue Oct 10, 2019 · 9 comments

Comments

@verne91
Copy link

verne91 commented Oct 10, 2019

I am trying to augment a graph with the .gam file using the following command line.

vg augment -p -t 12 merged.vg x.gam -C -A x.aug.gam > x.aug.vg

The standard output is:

Reading input graph
vg: src/augment.cpp:327: std::unordered_map<long int, std::set<std::tuple<long int, bool, long unsigned int> > > vg::forwardize_breakpoints(const HandleGraph*, const std::unordered_map<long int, std::set<std::tuple<long int, bool, long unsigned int> > >&): Assertion `graph->has_node(node_id)' failed.
ERROR: Signal 6 occurred. VG has crashed. Run 'vg bugs --new' to report a bug.
Stack trace path: /tmp/vg_crash_PnjvsC/stacktrace.txt

The stacktrace.txt is:

Crash report for vg v1.19.0 "Tramutola"
Stack trace (most recent call last):
#13   Object "/home/cnsun/software/bin/vg", at 0x4d6a79, in _start
#12   Object "/home/cnsun/software/bin/vg", at 0x1ac8318, in __libc_start_main
#11   Object "/home/cnsun/software/bin/vg", at 0x40a7d2, in main
#10   Object "/home/cnsun/software/bin/vg", at 0x9a94e7, in vg::subcommand::Subcommand::operator()(int,
char**) const
#9    Object "/home/cnsun/software/bin/vg", at 0x96dc2b, in main_augment(int, char**)
#8    Object "/home/cnsun/software/bin/vg", at 0xa4538c, in vg::get_input_file(s
td::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void (std::istream&)>)
#7    Object "/home/cnsun/software/bin/vg", at 0x96a2f1, in std::_Function_handler<void (std::istream&), main_augment(int, char**)::{lambda(std::istream&)#7}>::_M_invoke(std::_Any_data const&, std::istream&)
#6    Object "/home/cnsun/software/bin/vg", at 0xa7d887, in vg::augment(handlegraph::MutablePathMutableHandleGraph*, std::istream&, std::vector<vg::Translation, std::allocator<vg::Translation> >*, std::ostream*, bool, bool, bool)
#5    Object "/home/cnsun/software/bin/vg", at 0xa7ca2b, in vg::augment_impl(handlegraph::MutablePathMutableHandleGraph*, std::function<void (std::function<void (vg::Alignment&)>, bool)>, std::vector<vg::Translation, std::allocator<vg::Translation> >*, std::ostream*, bool, bool, bool)
#4    Object "/home/cnsun/software/bin/vg", at 0xa78d44, in vg::forwardize_breakpoints(handlegraph::HandleGraph const*, std::unordered_map<long, std::set<std::tuple<long, bool, unsigned long>, std::less<std::tuple<long, bool, unsigned long> >, std::allocator<std::tuple<long, bool, unsigned long> > >, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, std::set<std::tuple<long, bool, unsigned long>, std::less<std::tuple<long, bool, unsigned long> >, std::allocator<std::tuple<long, bool, unsigned long> > > > > > const&)
#3    Object "/home/cnsun/software/bin/vg", at 0x1acc231, in __assert_fail
#2    Object "/home/cnsun/software/bin/vg", at 0x1acc1bb, in __assert_fail_base
#1    Object "/home/cnsun/software/bin/vg", at 0x1ad8be0, in abort
#0    Object "/home/cnsun/software/bin/vg", at 0x10ab3d7, in raise
@glennhickey
Copy link
Contributor

glennhickey commented Oct 10, 2019 via email

@glennhickey
Copy link
Contributor

To get a more verbose error (if a graph/gam mismatch is indeed the issue) you can run

vg validate -x merged.vg -a x.gam

@verne91
Copy link
Author

verne91 commented Oct 10, 2019

It's complaining about a node in the GAM not being in the graph. Was x.gam created by mapping to merged.vg?
…
On Thu, Oct 10, 2019 at 2:00 PM Chen Sun @.> wrote: I am trying to augment a graph with the .gam file using the following command line. vg augment -p -t 12 merged.vg x.gam -C -A x.aug.gam > x.aug.vg The standard output is: Reading input graph vg: src/augment.cpp:327: std::unordered_map<long int, std::set<std::tuple<long int, bool, long unsigned int> > > vg::forwardize_breakpoints(const HandleGraph, const std::unordered_map<long int, std::set<std::tuple<long int, bool, long unsigned int> > >&): Assertion `graph->has_node(node_id)' failed. ERROR: Signal 6 occurred. VG has crashed. Run 'vg bugs --new' to report a bug. Stack trace path: /tmp/vg_crash_PnjvsC/stacktrace.txt The stacktrace.txt is: Crash report for vg v1.19.0 "Tramutola" Stack trace (most recent call last): #13 Object "/home/cnsun/software/bin/vg", at 0x4d6a79, in _start #12 Object "/home/cnsun/software/bin/vg", at 0x1ac8318, in __libc_start_main #11 Object "/home/cnsun/software/bin/vg", at 0x40a7d2, in main #10 Object "/home/cnsun/software/bin/vg", at 0x9a94e7, in vg::subcommand::Subcommand::operator()(int, char) const #9 Object "/home/cnsun/software/bin/vg", at 0x96dc2b, in main_augment(int, char**) #8 Object "/home/cnsun/software/bin/vg", at 0xa4538c, in vg::get_input_file(s td::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::function<void (std::istream&)>) #7 Object "/home/cnsun/software/bin/vg", at 0x96a2f1, in std::_Function_handler<void (std::istream&), main_augment(int, char**)::{lambda(std::istream&)#7}>::_M_invoke(std::_Any_data const&, std::istream&) #6 Object "/home/cnsun/software/bin/vg", at 0xa7d887, in vg::augment(handlegraph::MutablePathMutableHandleGraph*, std::istream&, std::vector<vg::Translation, std::allocatorvg::Translation >, std::ostream, bool, bool, bool) #5 Object "/home/cnsun/software/bin/vg", at 0xa7ca2b, in vg::augment_impl(handlegraph::MutablePathMutableHandleGraph*, std::function<void (std::function<void (vg::Alignment&)>, bool)>, std::vector<vg::Translation, std::allocatorvg::Translation >, std::ostream, bool, bool, bool) #4 Object "/home/cnsun/software/bin/vg", at 0xa78d44, in vg::forwardize_breakpoints(handlegraph::HandleGraph const*, std::unordered_map<long, std::set<std::tuple<long, bool, unsigned long>, std::less<std::tuple<long, bool, unsigned long> >, std::allocator<std::tuple<long, bool, unsigned long> > >, std::hash, std::equal_to, std::allocator<std::pair<long const, std::set<std::tuple<long, bool, unsigned long>, std::less<std::tuple<long, bool, unsigned long> >, std::allocator<std::tuple<long, bool, unsigned long> > > > > > const&) #3 Object "/home/cnsun/software/bin/vg", at 0x1acc231, in __assert_fail #2 Object "/home/cnsun/software/bin/vg", at 0x1acc1bb, in __assert_fail_base #1 Object "/home/cnsun/software/bin/vg", at 0x1ad8be0, in abort #0 Object "/home/cnsun/software/bin/vg", at 0x10ab3d7, in raise — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2502?email_source=notifications&email_token=AAG373XCD2G3YDYQFVWVOYDQN5USZA5CNFSM4I7QGYL2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HRAE4UA>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAG373T7EVFQ7QDFW7NRX53QN5USZANCNFSM4I7QGYLQ .

That may be the reason. I generate the .vg file for each chromosome and merge them with vg concat. But the index files I used for mapping is from each chromosome.

@glennhickey
Copy link
Contributor

Perhaps. A few things

  • if you're mapping each chromosome individually, it's best to augment them individually. augment takes a lot of memory (though i'm working to fix this right now)
  • it's best to merge chromosomes using cat and not vg concat. This is a frequent mistake, so we probably need to make the documentation more clear.

@verne91
Copy link
Author

verne91 commented Oct 10, 2019

Thanks for your suggestions. I will try them to see whether they works.
Another question:
I built .xg and .gsca2 index from all the chromosomes and generate a single index file. Can I use this for mapping? Or I need to cat all the .vg file and build the index from this merged .vg file?

@glennhickey
Copy link
Contributor

Merging the input beforehand with cat won't change the output of vg index. We recommend passing in the original chromosome vg's in the wiki https://github.com/vgteam/vg/wiki/Working-with-a-whole-genome-variation-graph to make an index for whole-genome mapping.

@verne91
Copy link
Author

verne91 commented Oct 10, 2019

I am still a little confusing when I using vg augment. Therefore I have something to confirm with you.

I built the .vg for each chromosome and generated a .xg and a .gcsa2 index file for the whole genome as the wiki https://github.com/vgteam/vg/wiki/Working-with-a-whole-genome-variation-graph said. Then I mapped the sample using these two index files and generated a .gam file. I want to augment the whole genome graph with this .gam file. But I don't have one. So I need to use cat to concatenate the .vg files as a whole genome graph. So the question is can I use this whole genome graph and .gam file as the input for vg augment?

@glennhickey
Copy link
Contributor

Unless your genome is small or your gam is very low coverage, then the answer is probably no, unfortunately.  augment will use too much memory.  If you have a lot of memory and want to try anyway, I recommend running vg convert graph.vg -p > graph.pg and giving that as an input to augment in order to save some memory.

One work-around, which maybe you were trying to get at to begin with, is to work on chromosomes. 

vg augment chr1.pg whole-genome.gam 
vg augment chr2.pg whole-genome.gam etc.

But then you'll run into that assertion error unless your GAM is also split by chromosome (possible but slow).

I will patch augment to not fail on these cases so the above by-chromosome workflow would run.   Hopefully we'll have whole-genome augmentation running better in the next release.

@glennhickey glennhickey mentioned this issue Oct 10, 2019
Merged
@srachita
Copy link

I am trying to augment the graph produced using vcf file along with the reference genome (working on a cluster with RAM 700Gb) using following command:

  1. vg augment -t 100 minimap_sniffles_graph.vg sample.gam -p -A sample_aug.gam > sample_aug.vg
    Gives me the following error:
    Reading input graph
    terminate called after throwing an instance of 'std::bad_alloc'
    what(): std::bad_alloc
    ERROR: Signal 6 occurred. VG has crashed.

  2. I then converted my graph using :
    vg convert -t 7 minimap_sniffles_graph.vg -p > minimap_sniffles_graph.pg (worked fine)

  3. Then I re attempted augmentation :
    vg augment -t 100 minimap_sniffles_graph.pg sample.gam -p -A sample_aug.gam > sample_aug.vg
    Gives me the following error:
    Reading input graph
    terminate called after throwing an instance of 'std::bad_alloc'
    what(): std::bad_alloc
    ERROR: Signal 6 occurred. VG has crashed.

  4. Then I re attempted augmentation with only 1 thread:
    vg augment -t 1 minimap_sniffles_graph.pg sample.gam -p -A sample_aug.gam > sample_aug.vg
    Gives the following error:
    Reading input graph
    terminate called after throwing an instance of 'std::runtime_error'
    what(): io::MessageEmitter::emit_group: I/O error writing protobuf
    ERROR: Signal 6 occurred. VG has crashed.

Could you please suggest what is going wrong? I have generated the graph using vg and the gams using vg as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@glennhickey @verne91 @srachita