Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segmentation fault (core dumped) #1081

Closed
escurzo opened this issue Jun 13, 2014 · 18 comments
Closed

Segmentation fault (core dumped) #1081

escurzo opened this issue Jun 13, 2014 · 18 comments

Comments

@escurzo
Copy link

escurzo commented Jun 13, 2014

Hi,

I have this error

oper@osrm:~/Project-OSRM$ ./osrm-datastore ../data/planet-latest.osrm
[info] load names from: "../data/planet-latest.osrm.names"
[info] size: 5190126
[info] allocating shared memory of 26923646936 bytes
[warn] could not delete DATA_1
[warn] could not delete LAYOUT_1
[info] all data loaded
oper@osrm:~/Project-OSRM$ ./osrm-routed --sharedmemory=yes
[info] starting up engines, v0.4.1, compiled at Jun 13 2014, 16:35:01
[info] loaded plugin: table
[info] loaded plugin: hello
[info] loaded plugin: locate
[info] loaded plugin: nearest
[info] loaded plugin: timestamp
[info] loaded plugin: viaroute
[info] http 1.1 compression handled by zlib version 1.2.8
[info] running and waiting for requests
[info] 13-06-2014 18:31:26 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?loc=41.407366,2.217661&loc=41.398475,2.200613
Segmentation fault (core dumped)
oper@osrm:~/Project-OSRM$ ./osrm-routed --sharedmemory=yes
[info] starting up engines, v0.4.1, compiled at Jun 13 2014, 16:35:01
[info] loaded plugin: table
[info] loaded plugin: hello
[info] loaded plugin: locate
[info] loaded plugin: nearest
[info] loaded plugin: timestamp
[info] loaded plugin: viaroute
[info] http 1.1 compression handled by zlib version 1.2.8
[info] running and waiting for requests
[info] 13-06-2014 18:35:52 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?z=18&output=json&jsonp=OSRM.JSONP.callbacks.route&checksum=-1246200715&loc=24.812646,46.503126&loc=24.737156,46.757469&instructions=false
[info] 13-06-2014 18:35:53 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /favicon.ico
[info] 13-06-2014 18:36:12 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?z=18&output=json&loc=24.812646,46.503126&loc=24.737156,46.757469&instructions=false
[info] routing took 1ms
[info] descriptor took 0ms
[info] 13-06-2014 18:37:18 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?z=18&output=json&loc=41.398475,2.200613&loc=41.480449,2.316999&instructions=false
[info] routing took 1ms
[info] descriptor took 0ms
[info] 13-06-2014 18:39:58 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?z=14&output=json&loc=41.488070,2.067661&loc=41.475789,2.089376&instructions=false
Segmentation fault (core dumped)

Same url https://router.project-osrm.org/viaroute?z=14&output=json&loc=41.488070,2.067661&loc=41.475789,2.089376&instructions=false works fine.

Some machine parameters
top - 18:44:27 up 21 min, 2 users, load average: 0.14, 0.09, 0.14
Tasks: 100 total, 1 running, 99 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.0 us, 0.0 sy, 0.0 ni,100.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem: 29355984 total, 29198168 used, 157816 free, 4856 buffers
KiB Swap: 16774140 total, 0 used, 16774140 free. 28801940 cached Mem

I use ubuntu 14.04

Thanks

@alex85k
Copy link
Contributor

alex85k commented Jun 13, 2014

Are you using data files and osrm-datastore from the same release as osrm-routed version?

If the errors are repeating, you could try to build debug version (cmake .. -DCMake_BUILD_TYPE=Debug; make -j2)
and run it via gdb:

gdb ./osrm-routed
run --sharedmemory=yes

When it breaks with segfault, the details will be printed (if no, send backtrace command to gdb).

@escurzo
Copy link
Author

escurzo commented Jun 13, 2014

Yes diferent machine but same osrm version

El viernes, 13 de junio de 2014, alex85k notifications@github.com
escribió:

Are you data files from the same release as osrm-routed version?

If the errors are repeating, you could try to build debug version (cmake
.. -DCMake_BUILD_TYPE=Debug)
and run it via gdb:

gdb ./osrm-routed
run --sharedmemory=yes

When it breaks with segfault, the details will be printed (if no, send
backtrace command to gdb).


Reply to this email directly or view it on GitHub
#1081 (comment)
.

Josep Ferrer
CTO

IGGEOS
EU: +34 93 176 02 15 / +34 91 119 63 01
US: +1 (415) 376 4112
www.iggeos.com

@DennisOSRM
Copy link
Collaborator

@escurzo do I get this right, that some queries work fine and others fail?

@escurzo
Copy link
Author

escurzo commented Jun 16, 2014

@DennisOSRM all queries have invalid results but only some queries prodices Segmentation fault

This is the backtrace

oper@osrm:~/Project-OSRM$ ./osrm-routed --sharedmemory=yes
[info] starting up engines, v0.4.1, compiled at Jun 13 2014, 16:35:01
[warn] exception: no leaf index file given. Is any data loaded into shared memory?
oper@osrm:~/Project-OSRM$ ./osrm-
osrm-datastore  osrm-extract    osrm-prepare    osrm-routed     
oper@osrm:~/Project-OSRM$ ./osrm-datastore ../data/planet-latest.osrm
[info] load names from: "../data/planet-latest.osrm.names"
[info] size: 5190126
[info] allocating shared memory of 26923646936 bytes
[warn] could not delete DATA_1
[warn] could not delete LAYOUT_1
[info] all data loaded
oper@osrm:~/Project-OSRM$ gdb ./osrm-routed
GNU gdb (Ubuntu 7.7-0ubuntu3.1) 7.7
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./osrm-routed...(no debugging symbols found)...done.
(gdb) run --sharedmemory=yes
Starting program: /home/oper/Project-OSRM/osrm-routed --sharedmemory=yes
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[info] starting up engines, v0.4.1, compiled at Jun 13 2014, 16:35:01
[info] loaded plugin: table
[info] loaded plugin: hello
[info] loaded plugin: locate
[info] loaded plugin: nearest
[info] loaded plugin: timestamp
[info] loaded plugin: viaroute
[info] http 1.1 compression handled by zlib version 1.2.8
[New Thread 0x7ff9ae8b8700 (LWP 1092)]
[info] running and waiting for requests
[New Thread 0x7ff9ae0b7700 (LWP 1093)]
[New Thread 0x7ff9ad8b6700 (LWP 1094)]
[New Thread 0x7ff9ad0b5700 (LWP 1095)]
[New Thread 0x7ff9ac8b4700 (LWP 1096)]
[New Thread 0x7ff9a7fff700 (LWP 1097)]
[info] 16-06-2014 15:14:15 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?z=14&output=json&loc=41.488070,2.067661&loc=41.475789,2.089376&instructions=false

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ff9ae0b7700 (LWP 1093)]
0x000000000045f5a8 in BasicRoutingInterface<BaseDataFacade<QueryEdge::EdgeData> >::UnpackEdge(unsigned int, unsigned int, std::vector<unsigned int, std::allocator<unsigned int> >&) const ()
(gdb) backtrace
#0  0x000000000045f5a8 in BasicRoutingInterface<BaseDataFacade<QueryEdge::EdgeData> >::UnpackEdge(unsigned int, unsigned int, std::vector<unsigned int, std::allocator<unsigned int> >&) const ()
#1  0x000000000046b62a in AlternativeRouting<BaseDataFacade<QueryEdge::EdgeData> >::operator()(PhantomNodes const&, RawRouteData&) ()
#2  0x00000000004794c6 in ViaRoutePlugin<BaseDataFacade<QueryEdge::EdgeData> >::HandleRequest(RouteParameters const&, http::Reply&) ()
#3  0x000000000047b0d0 in OSRM_impl::RunQuery(RouteParameters&, http::Reply&) ()
#4  0x000000000047c443 in RequestHandler::handle_request(http::Request const&, http::Reply&) ()
#5  0x000000000047d354 in http::Connection::handle_read(boost::system::error_code const&, unsigned long) ()
#6  0x000000000048ae97 in boost::asio::detail::completion_handler<boost::asio::detail::rewrapped_handler<boost::asio::detail::binder2<boost::asio::detail::wrapped_handler<boost::asio::io_service::strand, boost::_bi::bind_t<void, boost::_mfi::mf2<void, http::Connection, boost::system::error_code const&, unsigned long>, boost::_bi::list3<boost::_bi::value<std::shared_ptr<http::Connection> >, boost::arg<1> (*)(), boost::arg<2> (*)()> >, boost::asio::detail::is_continuation_if_running>, boost::system::error_code, unsigned long>, boost::_bi::bind_t<void, boost::_mfi::mf2<void, http::Connection, boost::system::error_code const&, unsigned long>, boost::_bi::list3<boost::_bi::value<std::shared_ptr<http::Connection> >, boost::arg<1> (*)(), boost::arg<2> (*)()> > > >::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) ()
#7  0x000000000048bbc8 in boost::asio::detail::reactive_socket_recv_op<boost::asio::mutable_buffers_1, boost::asio::detail::wrapped_handler<boost::asio::io_service::strand, boost::_bi::bind_t<void, boost::_mfi::mf2<void, http::Connection, boost::system::error_code const&, unsigned long>, boost::_bi::list3<boost::_bi::value<std::shared_ptr<http::Connection> >, boost::arg<1> (*)(), boost::arg<2> (*)()> >, boost::asio::detail::is_continuation_if_running> >::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) ()
#8  0x000000000048cf34 in boost::asio::detail::epoll_reactor::descriptor_state::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) ()
#9  0x000000000048c6e7 in boost::asio::io_service::run() ()
#10 0x00007ffff63c4bf0 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#11 0x00007ffff6a40182 in start_thread (arg=0x7ff9ae0b7700) at pthread_create.c:312
#12 0x00007ffff7b0f30d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
(gdb) 

@DennisOSRM
Copy link
Collaborator

looks like the data is broken.

@escurzo
Copy link
Author

escurzo commented Jun 16, 2014

Looking at the log it seems that the generation of data worked well!

Someone I can pass the planet data in OSRM format for to do some tests please? My OSRM version is v0.4.1, compiled at Jun 13 2014, 16:35:01

[info] generating edge-expanded edges
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
[info] Generated 506417744 edge based nodes
[info] Node-based graph contains 209269532 edges
[info] Edge-expanded graph ...
[info] contains 421095960 edges
[info] skips 141460 turns, defined by 260742 restrictions
[info] skips 179319580 U turns
[info] skips 203348 turns over barriers
[info] Timing statistics for edge-expanded graph:
[info] Geometry compression: 698.107s
[info] Renumbering edges: 2.65s
[info] Generating nodes: 233.761s
[info] Generating edges: 428.706s
[info] building r-tree ...
[info] constructing r-tree of 506417744 edge elements build on-top of 479684747 coordinates
[info] finished r-tree construction in 848.53 seconds
[info] using hardware based CRC32 computation
[info] CRC32: 354183759
[info] writing node map ...
[info] initializing contractor
merged 637746 edges out of 842191920
contractor finished initalization
Contractor is using 8 threads
initializing elimination PQ ...ok
preprocessing 209269532 nodes .... 10% . 20% . 30% . 40% . 50% . 60% . [flush 136928945 nodes] 70% . 80% . 90% . 100%
[info] Contraction took 13012.3 sec
[info] Getting edges of minimized graph
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
[info] Serializing compacted graph of 1170001723 edges
[info] Building node array
[info] Building edge array
[info] Preprocessing : 16454.9 seconds
[info] Expansion : 219072 nodes/sec and 95573.5 edges/sec
[info] Contraction: 16082.4 nodes/sec and 89915.1 edges/sec
[info] finished preprocessing

@DennisOSRM
Copy link
Collaborator

What are the file sizes of your data files.

@escurzo
Copy link
Author

escurzo commented Jun 16, 2014

-rw-rw-r-- 1 oper oper 22368465228 Jun 12 19:26 planet-latest.osrm
-rw-rw-r-- 1 oper oper 5053151524 Jun 12 20:01 planet-latest.osrm.edges
-rw-rw-r-- 1 oper oper 22284377948 Jun 12 20:17 planet-latest.osrm.fileIndex
-rw-rw-r-- 1 oper oper 4122249420 Jun 12 20:02 planet-latest.osrm.geometry
-rw-rw-r-- 1 oper oper 14877098976 Jun 13 00:01 planet-latest.osrm.hsgr
-rw-rw-r-- 1 oper oper 105656405 Jun 12 19:26 planet-latest.osrm.names
-rw-rw-r-- 1 oper oper 5756216968 Jun 12 20:18 planet-latest.osrm.nodes
-rw-rw-r-- 1 oper oper 138662680 Jun 12 20:17 planet-latest.osrm.ramIndex
-rw-rw-r-- 1 oper oper 4185920 Jun 12 13:34 planet-latest.osrm.restrictions

Are correct?

@alex85k
Copy link
Contributor

alex85k commented Jun 16, 2014

Did you try to run osrm-routed directly from file, not with datastore?

@escurzo
Copy link
Author

escurzo commented Jun 17, 2014

oper@osrm:~/Project-OSRM$ ./osrm-routed ../data/planet-latest.osrm
[info] starting up engines, v0.4.1, compiled at Jun 13 2014, 16:35:01
[info] HSGR file: "../data/planet-latest.osrm.hsgr"
[info] loading graph data
[info] loading graph from ../data/planet-latest.osrm.hsgr
[info] number_of_nodes: 209269533, number_of_edges: 1170001723
[info] loaded 209269534 nodes and 1170001723 edges
[info] Data checksum is 354183759
[info] loading egde information
[info] loading geometries
[info] loading r-tree
[info] loading timestamp
[info] loading street names
[info] loaded plugin: table
[info] loaded plugin: hello
[info] loaded plugin: locate
[info] loaded plugin: nearest
[info] loaded plugin: timestamp
[info] loaded plugin: viaroute
[info] http 1.1 compression handled by zlib version 1.2.8
[info] running and waiting for requests

[info] 17-06-2014 10:45:25 192.168.2.4 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?z=14&output=json&loc=41.488070,2.067661&loc=41.475789,2.089376&instructions=false
Segmentation fault (core dumped)

In my /var/log/syslog
Jun 17 10:35:26 osrm kernel: [ 3.277827] IPv6: ADDRCONF(NETDEV_UP): eth2: link is not ready
Jun 17 10:35:26 osrm kernel: [ 3.277831] IPv6: ADDRCONF(NETDEV_UP): eth3: link is not ready
Jun 17 10:35:26 osrm kernel: [ 3.340855] EXT4-fs (xvda1): re-mounted. Opts: errors=remount-ro
Jun 17 10:35:26 osrm kernel: [ 3.472886] lp: driver loaded but no devices found
Jun 17 10:35:26 osrm kernel: [ 3.570777] type=1400 audit(1402994125.810:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/sbin/dhclient" pid=384 comm="apparmor_parser"
Jun 17 10:35:26 osrm kernel: [ 3.570785] type=1400 audit(1402994125.810:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=384 comm="apparmor_parser"
Jun 17 10:35:26 osrm kernel: [ 3.570790] type=1400 audit(1402994125.810:4): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=384 comm="apparmor_parser"
Jun 17 10:35:26 osrm kernel: [ 3.571323] type=1400 audit(1402994125.810:5): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=384 comm="apparmor_parser"
Jun 17 10:35:26 osrm kernel: [ 3.571330] type=1400 audit(1402994125.810:6): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=384 comm="apparmor_parser"
Jun 17 10:35:26 osrm kernel: [ 3.571586] type=1400 audit(1402994125.810:7): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=384 comm="apparmor_parser"
Jun 17 10:35:26 osrm kernel: [ 3.574628] type=1400 audit(1402994125.814:8): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/sbin/dhclient" pid=386 comm="apparmor_parser"
Jun 17 10:35:26 osrm kernel: [ 3.574637] type=1400 audit(1402994125.814:9): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=386 comm="apparmor_parser"
Jun 17 10:35:26 osrm kernel: [ 3.574642] type=1400 audit(1402994125.814:10): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=386 comm="apparmor_parser"
Jun 17 10:35:26 osrm rsyslogd-2039: Could no open output pipe '/dev/xconsole': No such file or directory [try http://www.rsyslog.com/e/2039 ]
Jun 17 10:35:26 osrm acpid: cannot open input layer
Jun 17 10:35:26 osrm cron[748]: (CRON) INFO (pidfile fd = 3)
Jun 17 10:35:26 osrm cron[815]: (CRON) STARTUP (fork ok)
Jun 17 10:35:26 osrm acpid: starting up with netlink and the input layer
Jun 17 10:35:26 osrm cron[815]: (CRON) INFO (Running @reboot jobs)
Jun 17 10:35:26 osrm acpid: 1 rule loaded
Jun 17 10:35:26 osrm acpid: waiting for events: event logging is off
Jun 17 10:35:32 osrm ntpdate[603]: step time server 91.189.94.4 offset 0.019535 sec
Jun 17 10:35:48 osrm ntpdate[986]: adjust time server 91.189.94.4 offset 0.000734 sec
Jun 17 10:37:21 osrm kernel: [ 119.645321] Adding 314571772k swap on /dev/xvdb1. Priority:-2 extents:1 across:314571772k SSFS
Jun 17 10:45:26 osrm kernel: [ 603.808610] show_signal_msg: 48 callbacks suppressed
Jun 17 10:45:26 osrm kernel: [ 603.808617] osrm-routed[1072]: segfault at 7f3f196d400b ip 000000000045f5a8 sp 00007f3049e4d020 error 4 in osrm-routed[400000+af000]

@DennisOSRM
Copy link
Collaborator

File sizes look ok, but running and preprocessing on different machines may have issues such as you are experiencing. Especially if you use a 32bit architecture somewhere.

@escurzo
Copy link
Author

escurzo commented Jun 17, 2014

I was tested OSRM in the same machine that I have processes files and the process crash

ubuntu@ip-172-31-27-7:~/Project-OSRM$ ./osrm-routed /data/planet-latest.osrm
[warn] ./osrm-routed could not be locked to RAM
[info] starting up engines, v0.4.1, compiled at Jun 17 2014, 10:16:10
[info] HSGR file: "/data/planet-latest.osrm.hsgr"
[info] loading graph data
[info] loading graph from /data/planet-latest.osrm.hsgr
[info] number_of_nodes: 209269533, number_of_edges: 1170001723
T[info] loaded 209269534 nodes and 1170001723 edges
[info] Data checksum is 354183759
[info] loading egde information
[info] loading geometries
[info] loading r-tree
[info] loading timestamp
[info] loading street names
[info] loaded plugin: table
[info] loaded plugin: hello
[info] loaded plugin: locate
[info] loaded plugin: nearest
[info] loaded plugin: timestamp
[info] loaded plugin: viaroute
[info] http 1.1 compression handled by zlib version 1.2.8
[info] running and waiting for requests
[info] 17-06-2014 10:38:22 80.28.209.138 - Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36 /viaroute?z=14&output=json&loc=41.488070,2.067661&loc=41.475789,2.089376&instructions=false
Segmentation fault (core dumped)

the system log shows
Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 7.431832] ISOFS: Unable to identify CD-ROM format.
Jun 17 10:11:53 ip-172-31-27-7 rsyslogd-2039: Could no open output pipe '/dev/xconsole': No such file or directory [try http://www.rsyslog.com/e/2039 ]
Jun 17 10:11:53 ip-172-31-27-7 dbus[1193]: [system] AppArmor D-Bus mediation is enabled
Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.100084] audit_printk_skb: 9 callbacks suppressed
Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.100088] type=1400 audit(1402999913.825:14): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/sbin/dhclient" pid=1239 comm="apparmor_parser"
Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.100096] type=1400 audit(1402999913.825:15): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=1239 comm="apparmor_parser"
Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.100102] type=1400 audit(1402999913.825:16): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=1239 comm="apparmor_parser"
Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.100569] type=1400 audit(1402999913.825:17): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=1239 comm="apparmor_parser"
Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.100575] type=1400 audit(1402999913.825:18): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=1239 comm="apparmor_parser"
Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.100812] type=1400 audit(1402999913.825:19): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=1239 comm="apparmor_parser"
Jun 17 10:11:53 ip-172-31-27-7 kernel: [ 8.105861] type=1400 audit(1402999913.829:20): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/sbin/tcpdump" pid=1241 comm="apparmor_parser"
Jun 17 10:11:54 ip-172-31-27-7 pollinate[1306]: system was previously seeded at [2014-06-11 14:34:42.806643000 +0000]
Jun 17 10:11:54 ip-172-31-27-7 pollinate[1309]: To re-seed this system again, use the -r|--reseed option
Jun 17 10:11:54 ip-172-31-27-7 pollinate[1378]: system was previously seeded at [2014-06-11 14:34:42.806643000 +0000]
Jun 17 10:11:54 ip-172-31-27-7 acpid: starting up with netlink and the input layer
Jun 17 10:11:54 ip-172-31-27-7 pollinate[1381]: To re-seed this system again, use the -r|--reseed option
Jun 17 10:11:54 ip-172-31-27-7 cron[1271]: (CRON) INFO (pidfile fd = 3)
Jun 17 10:11:54 ip-172-31-27-7 acpid: 1 rule loaded
Jun 17 10:11:54 ip-172-31-27-7 acpid: waiting for events: event logging is off
Jun 17 10:11:54 ip-172-31-27-7 cron[1389]: (CRON) STARTUP (fork ok)
Jun 17 10:11:54 ip-172-31-27-7 cron[1389]: (CRON) INFO (Running @reboot jobs)
Jun 17 10:11:59 ip-172-31-27-7 ntpdate[963]: step time server 91.189.94.4 offset 0.874286 sec
Jun 17 10:12:06 ip-172-31-27-7 ntpdate[1499]: adjust time server 91.189.94.4 offset -0.000206 sec
Jun 17 10:12:58 ip-172-31-27-7 kernel: [ 71.871860] EXT4-fs (xvdf): mounted filesystem with ordered data mode. Opts: (null)
Jun 17 10:17:01 ip-172-31-27-7 CRON[2391]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Jun 17 10:38:09 ip-172-31-27-7 dhclient: DHCPREQUEST of 172.31.27.7 on eth0 to 172.31.16.1 port 67 (xid=0x69de4aec)
Jun 17 10:38:09 ip-172-31-27-7 dhclient: DHCPACK of 172.31.27.7 from 172.31.16.1
Jun 17 10:38:09 ip-172-31-27-7 dhclient: bound to 172.31.27.7 -- renewal in 1710 seconds.
Jun 17 10:38:22 ip-172-31-27-7 kernel: [ 1595.811003] osrm-routed[2607]: segfault at 7f3af4ae100b ip 00000000004616c8 sp 00007f2c1e7f9fb0 error 4 in osrm-routed[400000+af000]

There is an amazon i2.4xlarge instance with ubuntu 14.04

@escurzo
Copy link
Author

escurzo commented Jun 17, 2014

Revising logs during procesing files I can see
[STXXL-ERRMSG] External memory block allocation error: 2097152 bytes requested, 0 bytes free. Trying to extend the external memory space...

@DennisOSRM
Copy link
Collaborator

rebuild with debug configuration. osrm-routed will then inspect the data files very thoroughly and report if they are broken (which I suspect). If they are broken then you need to check configuration of swap, /tmp as well as the stxxl files.

@emiltin
Copy link
Contributor

emiltin commented Jun 17, 2014

inspection of input would be a nice option in the release version, ie:

osrm-routed planet.osrm --inspect

@DennisOSRM
Copy link
Collaborator

Perhaps we could have this as a separate tool.

@emiltin
Copy link
Contributor

emiltin commented Jul 21, 2014

would probably make sense

@DennisOSRM
Copy link
Collaborator

This is now in develop branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants