xg indexing doesn't work on whole-genome graph

[glennhickey]

Since the new xg got merged in, I can no longer index whole-genome graphs. Here is one on courtyard that used to work fine: /public/home/hickey/dev/hgsvc/HGSVC-jan5/HGSVC.vg

Now vg index HGSVC.vg -x HGSVC.xg segfaults after 12 minutes

#8    Object "", at 0xffffffffffffffff, in 
#7    Object "/data1/vg/bin/vg", at 0x55aabb6d3429, in _start
#6    Object "/lib/x86_64-linux-gnu/libc-2.27.so", at 0x7f7ec5501b96, in __libc_start_main
      Source "../csu/libc-start.c", line 310, in __libc_start_main [0x7f7ec5501b96]
#5    Object "/data1/vg/bin/vg", at 0x55aabb60a8c2, in main
      Source "src/main.cpp", line 73, in main [0x55aabb60a8c2]
#4    Object "/data1/vg/bin/vg", at 0x55aabbb6e2e7, in vg::subcommand::Subcommand::operator()(int, char**) const
    | Source "src/subcommand/subcommand.cpp", line 72, in operator()
      Source "/usr/include/c++/7/bits/std_function.h", line 706, in operator() [0x55aabbb6e2e7]
        703:     {
        704:       if (_M_empty())
        705: 	__throw_bad_function_call();
      > 706:       return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...);
        707:     }
        708: 
        709: #if __cpp_rtti
#3    Object "/data1/vg/bin/vg", at 0x55aabbad79d6, in main_index(int, char**)
      Source "src/subcommand/index_main.cpp", line 544, in main_index [0x55aabbad79d6]
#2    Object "/data1/vg/bin/vg", at 0x55aabbf26806, in vg::VGset::to_xg(xg::XG&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)> const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, vg::Path, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, vg::Path> > >*)
      Source "src/vg_set.cpp", line 182, in __stack_chk_fail [0x55aabbf26806]
#1    Object "/data1/vg/bin/vg", at 0x55aabc7b650f, in xg::XG::from_enumerators(std::function<void (std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, long const&)> const&)> const&, std::function<void (std::function<void (long const&, bool const&, long const&, bool const&)> const&)> const&, std::function<void (std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, long const&, bool const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&)> const&)> const&, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)
      Source "deps/xg/src/xg.cpp", line 812, in from_enumerators [0x55aabc7b650f]
#0    Object "/data1/vg/bin/vg", at 0x55aabc7eb50e, in mmmulti::map<unsigned long, unsigned long>::index(unsigned long)
      Source "/data1/vg/include/mmmultimap.hpp", line 406, in index [0x55aabc7eb50e]
        403:                 last = curr;
        404:             }
        405:             // the last key in the sort is our max key
      > 406:             max_key = nth_key(n_records-1);
        407:             key_bv[n_records] = 1; // sentinel
        408:             // build the compressed bitvector
        409:             sdsl::util::assign(key_cbv, sdsl::sd_vector<>(key_bv));

[ekg]

I'm wondering if this is caused by underflow. I'll take a look.

…

On Wed, Sep 11, 2019, 23:17 Glenn Hickey ***@***.***> wrote: Since the new xg got merged in, I can no longer index whole-genome graphs. Here is one on courtyard that used to work fine: /public/home/hickey/dev/hgsvc/HGSVC-jan5/HGSVC.vg Now vg index HGSVC.vg -x HGSVC.xg segfaults after 12 minutes #8 Object "", at 0xffffffffffffffff, in #7 Object "/data1/vg/bin/vg", at 0x55aabb6d3429, in _start #6 Object "/lib/x86_64-linux-gnu/libc-2.27.so", at 0x7f7ec5501b96, in __libc_start_main Source "../csu/libc-start.c", line 310, in __libc_start_main [0x7f7ec5501b96] #5 Object "/data1/vg/bin/vg", at 0x55aabb60a8c2, in main Source "src/main.cpp", line 73, in main [0x55aabb60a8c2] #4 Object "/data1/vg/bin/vg", at 0x55aabbb6e2e7, in vg::subcommand::Subcommand::operator()(int, char**) const | Source "src/subcommand/subcommand.cpp", line 72, in operator() Source "/usr/include/c++/7/bits/std_function.h", line 706, in operator() [0x55aabbb6e2e7] 703: { 704: if (_M_empty()) 705: __throw_bad_function_call(); > 706: return _M_invoker(_M_functor, std::forward<_ArgTypes>(__args)...); 707: } 708: 709: #if __cpp_rtti #3 Object "/data1/vg/bin/vg", at 0x55aabbad79d6, in main_index(int, char**) Source "src/subcommand/index_main.cpp", line 544, in main_index [0x55aabbad79d6] #2 Object "/data1/vg/bin/vg", at 0x55aabbf26806, in vg::VGset::to_xg(xg::XG&, std::function<bool (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)> const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, vg::Path, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, vg::Path> > >*) Source "src/vg_set.cpp", line 182, in __stack_chk_fail [0x55aabbf26806] #1 Object "/data1/vg/bin/vg", at 0x55aabc7b650f, in xg::XG::from_enumerators(std::function<void (std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, long const&)> const&)> const&, std::function<void (std::function<void (long const&, bool const&, long const&, bool const&)> const&)> const&, std::function<void (std::function<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, long const&, bool const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool const&, bool const&)> const&)> const&, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) Source "deps/xg/src/xg.cpp", line 812, in from_enumerators [0x55aabc7b650f] #0 Object "/data1/vg/bin/vg", at 0x55aabc7eb50e, in mmmulti::map<unsigned long, unsigned long>::index(unsigned long) Source "/data1/vg/include/mmmultimap.hpp", line 406, in index [0x55aabc7eb50e] 403: last = curr; 404: } 405: // the last key in the sort is our max key > 406: max_key = nth_key(n_records-1); 407: key_bv[n_records] = 1; // sentinel 408: // build the compressed bitvector 409: sdsl::util::assign(key_cbv, sdsl::sd_vector<>(key_bv)); — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2447?email_source=notifications&email_token=AABDQEKXOHMMC37OAXATRE3QJFU6HA5CNFSM4IVZOGNKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HK23I3Q>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABDQEMC6IEDKVRVO7WY3S3QJFU6HANCNFSM4IVZOGNA> .

[ekg]

@glennhickey I'm unable to reproduce this problem. Do you think it might be a transient issue? If so, I can try to run in a loop until it happens.

-> % time vg index -x HGSVC.xg HGSVC.vg
vg index -x HGSVC.xg HGSVC.vg  1253.65s user 855.02s system 204% cpu 17:13.33 total

The index seems fine.

[glennhickey]

Weird. It happens to me every time. Do you mind sticking the xg somewhere on courtyard so I can get my hands on it? I'll verify my setup. I'm on a fresh aws instance, and perhaps the default tmpdir is pointing somewhere without enough space. Thanks!

…

On Thu, Sep 12, 2019 at 10:46 AM Erik Garrison ***@***.***> wrote: @glennhickey <https://github.com/glennhickey> I'm unable to reproduce this problem. Do you think it might be a transient issue? If so, I can try to run in a loop until it happens. -> % time vg index -x HGSVC.xg HGSVC.vg vg index -x HGSVC.xg HGSVC.vg 1253.65s user 855.02s system 204% cpu 17:13.33 total The index seems fine. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2447?email_source=notifications&email_token=AAG373VRKK763X7CLTSSZOLQJJI3PA5CNFSM4IVZOGNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6SERDA#issuecomment-530860172>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAG373ULQZTRAZRWQ5JZSITQJJI3PANCNFSM4IVZOGNA> .

[ekg]

Sure, it'll show up in /public/home/ergarris/graphs/human/HGSVC when the copy's done (maybe 45 minutes from now).

I think you can set TMPDIR or something to where you want to work.

I can't compile vg on courtyard, so I'm not working there. I guess docker is a solution, but it's also a new set of problems. Should have this sorted soon.

[ekg]

That's TMPDIR in your shell environment.

[ekg]

Did you build this vg on exactly the same system? It seems our static builds are less platform-independent than they used to be. I'm seeing lots of weird errors when using even v1.16.0 in the CPANG19 course environment.

[glennhickey]

I'm building and running on the same system (aws node with ubuntu 18.04). I've got it going again with the TMPDIR explicitly set. I'm pretty sure this is the issue, but will know for sure soon. If this is the case, I think we're going to want to be able to give a better error message as, from experience with gcsa indexing, it will be a common ocurrence.

…

On Thu, Sep 12, 2019 at 10:55 AM Erik Garrison ***@***.***> wrote: Did you build this vg on exactly the same system? It seems our static builds are less platform-independent than they used to be. I'm seeing lots of weird errors when using even v1.16.0 in the CPANG19 course environment. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2447?email_source=notifications&email_token=AAG373URKVX2FK7ZHH3TUV3QJJJ5VA5CNFSM4IVZOGNKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6SFSEQ#issuecomment-530864402>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAG373QKZPYQ7BFQ5CBWNELQJJJ5VANCNFSM4IVZOGNA> .

[glennhickey]

My bad. It was indeed the TMPDIR. With it set, it ran through fine for me too. What's extra nice is that including the 145000 alt paths vg index -L only cost an extra 10s of runtime and 3G of RAM.