crash with trsp on Ubuntu 13.04/Postgres 9.2

[szilvesztererdos]

I experience random crashes on my VPS with Ubuntu 13.04, Postgres 9.2 and Pgrouting 2.0.0rc1 built from source. The crash occurs with pgr_trsp function and not with pgr_dijkstra, the postgres log says:

terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
server process (PID 28752) was terminated by signal 6: Aborted

Maybe it is worth mentioning that the crash occurs if I issue multiple queries at the same time, so server load is high (10-15 multiple processes). Btw. it is a server with 16 CPU cores, 20 GB RAM and SSD, postgres memory config settings are on default.

I tried to debug it with gdb and core file enabled and here is the top of the stack trace (I think this is the relevant part, but I could send all the 184 rows):
#0 0x00007f7a6c2e7037 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007f7a6c2ea698 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007f7a6651ce8d in __gnu_cxx::__verbose_terminate_handler() ()

from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3 0x00007f7a6651af76 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4 0x00007f7a6651afa3 in std::terminate() ()

from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5 0x00007f7a6651b1de in __cxa_throw ()

from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6 0x00007f7a6651b6cd in operator new(unsigned long) ()

from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#7 0x00007f7a66096bf0 in allocate (__n=8388608, this=)

at /usr/include/c++/4.7/ext/new_allocator.h:94

#8 _M_allocate (__n=, this=)

at /usr/include/c++/4.7/bits/stl_vector.h:169

#9 std::vector<GraphEdgeInfo, std::allocator >::_M_insert_aux (

this=this@entry=0x7fff4e793710, __position=..., __x=...)
at /usr/include/c++/4.7/bits/vector.tcc:346

#10 0x00007f7a660907b3 in push_back (__x=..., this=0x7fff4e793710)

at /usr/include/c++/4.7/bits/stl_vector.h:893

#11 GraphDefinition::addEdge (this=0x7fff4e793710, edgeIn=...)

at /home/sly/pgrouting-2.0.0-rc1/src/trsp/src/GraphDefinition.cpp:629

[woodbri]

Thanks, it looks like I forgot to add a try-catch wrapper on the c++ code. I just added that so it will catch these errors and exit gracefully without a server crash. As to why it is getting an std::bad_alloc this is another issue but for whatever reason the C++ code can not allocate memory and therefore can not continue. I'm closing the crash issue, if you can find out more about why it is have memory issues, please open a new issue for that.

[szilvesztererdos]

Sorry for commenting on a closed issue but I think this is not a good workaround because with the new code I am experiencing crash as well, although I have not got the std::bad_alloc exception just "cannot allocate memory for thread-local data: ABORT" (I am pretty sure that the above thing caused this error because I ran the same queries and I changed nothing besides the new pgrouting code).

Btw I have looked into this thing (I am not a programmer just curious and heard about one thing or two), and found the following things:

• according to the stack trace, the function call m_vecEdgeVector.push_back(newEdge) causes the problem (/src/trsp/src/GraphDefinition.cpp:629)
• from GraphDefinition.h I have realized that m_vecEdgeVector is a GraphEdgeVector (row 107) and GraphEdgeVector is a std::vector (row 75)
• according to the c++ reference, "If a reallocation happens, the storage is allocated using the container's allocator, which may throw exceptions on failure (for the default allocator, bad_alloc is thrown if the allocation request does not succeed)." (http://www.cplusplus.com/reference/vector/vector/push_back/)
• according to a forum conversation on cprogramming.com, this happens when the server runs out of memory, which is happen to be our case as well
• in the above link, forum posters advise several workarounds (do not keep that big array in memory, use vector's reserve method in order to avoid allocating extra memory, reducing the size of the GraphEdgeInfo structure by changing doubles to floats)

Maybe these information help you to solve this issue.

[woodbri]

My fix was only intended to prevent the server crash. I will reopen this and take another look at it given your additional comments. Thanks.

[woodbri]

OK, I just committed changes that I think will fix this issue. Can you check them out and try again. Also if it is convenient to run valgrind on this test that would be a big help. Post back you results, please.

[szilvesztererdos]

I ran a couple of tests with the newest code and could not reproduce the issue (yet... ).
Since the server has not crashed, maybe it has no point running valgrind - although I have tried it and the postgres server crashed with segfault... So I am not attaching valgrind report :)

I really hope that this fix solved the issue, I post back if I experience any problem.
Thanks for helping!

[woodbri]

ok, closing this. If you find a new problem open a new ticket. Thanks for reporting the issue.