Segmentation fault while importing diff with tiles expire option

[jendrusk]

In some cases (not always) osm2pgsql is rising segmentation fault error.
4808 Segmentation fault (core dumped)
Full command is:

/usr/local/bin/osm2pgsql --number-processes 1 -v -C 2000 -G -K -j -x -s -S ./default.style --append -d osm2pgsql --bbox 13.50,48.50,24.50,55.50 --expire-tiles 5-17 --expire-output /home/osm/scripts/osm/expire_list_tmp current.osc

Version of osm2pgsql:

$ osm2pgsql -v
osm2pgsql version 0.91.0-dev (64 bit id space)

Osm2pgsql failed due to ERROR: Usage error. For further information see:
        osm2pgsql -h|--help

System:

$ uname -a
Linux mapa 4.4.0-47-generic #68-Ubuntu SMP Wed Oct 26 19:39:52 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

The same command without --expire-tiles and --expire-output options working fine

Attached two diff files (made by osmosis) causing error
current.zip

[pnorman]

This works for me:

./osm2pgsql --number-processes 1 -v -C 2000 -G -K -j -x -s -S ../default.style --append \
-d gis --bbox 13.50,48.50,24.50,55.50 \
--expire-tiles 5-17 --expire-output tmp_out -r xml current.osc_old

I had to first load some data into the DB so did that with

./osm2pgsql -v -C 2000 -G -K -j -x -s -S ../default.style ../tests/liechtenstein-2013-08-03.osm.pbf

[lonvia]

Can you provide the backtrace for the segmentation fault?

[jendrusk]

@lonvia Sorry - forget most important think... Log for 'current.osc_old1'
Script is launched by cron, in logfile I've got this:

Sat Nov 19 00:20:01 CET 2016
No current.osc file found, downloading new one...
Nov 19, 2016 12:20:01 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.44.1
Nov 19, 2016 12:20:01 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
Nov 19, 2016 12:20:01 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.
Nov 19, 2016 12:20:01 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline executing, waiting for completion.
Nov 19, 2016 12:20:02 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline complete.
Nov 19, 2016 12:20:02 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Total execution time: 1036 milliseconds.
osm2pgsql version 0.91.0-dev (64 bit id space)

Using built-in tag processing pipeline
Using projection SRS 3857 (Spherical Mercator)
Setting up table: planet_osm_point
Setting up table: planet_osm_line
Setting up table: planet_osm_polygon
Setting up table: planet_osm_roads
Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=2000MB, maxblocks=32000*65536, allocation method=11
Mid: pgsql, scale=100 cache=2000
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels

Reading in file: current.osc
Applying Bounding box: 13.500000,48.500000 to 24.500000,55.500000
Using XML parser.

Processing: Node(0k 0.0k/s) Way(1k 0.50k/s) Relation(0 0.00/s)
Processing: Node(0k 0.0k/s) Way(1k 0.62k/s) Relation(10 10.00/s) parse time: 2s
Node stats: total(0), max(0) in 0s
Way stats: total(1232), max(454391292) in 2s
Relation stats: total(12), max(6721047) in 0s
Committing transaction for planet_osm_point
Committing transaction for planet_osm_line
Committing transaction for planet_osm_polygon
Committing transaction for planet_osm_roads
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels
Using built-in tag processing pipeline

Going over pending ways...
905 ways are pending

Using 1 helper-processes

Finished processing 905 ways in 1 s

905 Pending ways took 1s at a rate of 905.00/s
Committing transaction for planet_osm_point
Committing transaction for planet_osm_line
Committing transaction for planet_osm_polygon
Committing transaction for planet_osm_roads

Going over pending relations...
23 relations are pending

Using 1 helper-processes

Finished processing 23 relations in 0 s

Committing transaction for planet_osm_point
WARNING: there is no transaction in progress
Committing transaction for planet_osm_line
WARNING: there is no transaction in progress
Committing transaction for planet_osm_polygon
WARNING: there is no transaction in progress
Committing transaction for planet_osm_roads
WARNING: there is no transaction in progress
Completed planet_osm_point
Completed planet_osm_line
Completed planet_osm_roads
Completed planet_osm_polygon

Log ends here, and via mail from cron I'm getting this:

/home/osm/scripts/osm/replication_update.sh: line 43: 20672 Segmentation fault (core dumped) /usr/local/bin/osm2pgsql --number-processes 1 -v -C 2000 -G -K -j -x -s -S ./default.style --append -d osm2pgsql --bbox 13.50,48.50,24.50,55.50 --expire-tiles 5-17 --expire-output /home/osm/scripts/osm/expire_list_tmp current.osc >> $logpath$logfile 2>&1

Of course database is not empty - running few months without problems. All started when I've installed mod_tile and edited script so it returns expire tiles list. Here is map made from this data by geoserver.

Attached full script and style file.
default.zip

[SomeoneElseOSM]

First thought - what has changed (osm2pgsql version, mod_tile tile expiry script, machine environment or data)?

I did notice a few problems with tile expiry (running out of memory) when I was looking at increasing zoom levels supported by rendering - see https://wiki.openstreetmap.org/wiki/User:SomeoneElse/mod_tile_28 and the links from there - but I'd be surprised if zoom level 17 was enough to cause the problem.

[lonvia]

@jendrusk That's quite helpful, so it segfaults while cleaning up. I'd still like to have a look at the backtrace: your log says 'Segmentation fault (core dumped)', so somewhere there must be a file called 'core'. Linux normally writes it into the current working directory, I'm not sure where that would be on your system, most likely in /home/osm or /home/osm/scripts/osm. Once you have found the file, please install gdb. Then run gdb /usr/local/bin/osm2pgsql <your core file>. You'll get a prompt, where you should type bt. The output of that command is the backtrace I'm interested in.

[jendrusk]

@lonvia - as radiotelegraphists say 'SRI IM LID'

osm@mapa:~$ gdb /usr/local/bin/osm2pgsql /home/osm/scripts/osm_test/core
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/local/bin/osm2pgsql...done.
[New LWP 1661]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/local/bin/osm2pgsql --number-processes 1 -v -C 2000 -G -K -j -x -s -S ./de'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00000000004cc92f in tile::output_and_destroy (this=0x0, output=0x7ffc23eb1d80, x=x@entry=0, y=y@entry=0, this_zoom=this_zoom@entry=0)
    at /root/download/osm2pgsql/osm2pgsql/expire-tiles.cpp:118
118     /root/download/osm2pgsql/osm2pgsql/expire-tiles.cpp: Permission denied.
(gdb) bt
#0  0x00000000004cc92f in tile::output_and_destroy (this=0x0, output=0x7ffc23eb1d80, x=x@entry=0, y=y@entry=0, this_zoom=this_zoom@entry=0)
    at /root/download/osm2pgsql/osm2pgsql/expire-tiles.cpp:118
#1  0x00000000004ccb07 in expire_tiles::output_and_destroy (this=0x1ae60e0, output=<optimized out>) at /root/download/osm2pgsql/osm2pgsql/expire-tiles.cpp:160
#2  0x00000000004ccbb5 in expire_tiles::output_and_destroy (this=this@entry=0x1ae60e0, filename=<optimized out>, minzoom=<optimized out>)
    at /root/download/osm2pgsql/osm2pgsql/expire-tiles.cpp:169
#3  0x00000000004b938f in output_pgsql_t::stop (this=0x1ae5ce0) at /root/download/osm2pgsql/osm2pgsql/output-pgsql.cpp:357
#4  0x000000000046d469 in std::_Mem_fn_base<void (output_t::*)(), true>::operator()<, void>(output_t*) const (__object=<optimized out>, this=<optimized out>)
    at /usr/include/c++/5/functional:600
#5  std::_Bind_simple<std::_Mem_fn<void (output_t::*)()> (output_t*)>::_M_invoke<0ul>(std::_Index_tuple<0ul>) (this=<optimized out>) at /usr/include/c++/5/functional:1531
#6  std::_Bind_simple<std::_Mem_fn<void (output_t::*)()> (output_t*)>::operator()() (this=<optimized out>) at /usr/include/c++/5/functional:1520
#7  std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<void>, std::__future_base::_Result_base::_Deleter>, std::_Bind_simple<std::_Mem_fn<void (output_t::*)()> (output_t*)>, void>::operator()() const (this=0x7ffc23eb1f40) at /usr/include/c++/5/future:1342
#8  std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> (), std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result<void>, std::__future_base::_Result_base::_Deleter>, std::_Bind_simple<std::_Mem_fn<void (output_t::*)()> (output_t*)>, void> >::_M_invoke(std::_Any_data const&) (
    __functor=...) at /usr/include/c++/5/functional:1857
#9  0x000000000046dd29 in std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>::operator()() const (this=<optimized out>)
    at /usr/include/c++/5/functional:2267
#10 std::__future_base::_State_baseV2::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*) (this=0x1ca8d70,
    __f=<optimized out>, __did_set=0x7ffc23eb1f0f) at /usr/include/c++/5/future:527
#11 0x00007f301880fae9 in __pthread_once_slow (once_control=0x1ca8d88, init_routine=0x45d150 <__once_proxy@plt>) at pthread_once.c:116
#12 0x000000000046e634 in __gthread_once (__func=<optimized out>, __once=0x1ca8d88) at /usr/include/x86_64-linux-gnu/c++/5/bits/gthr-default.h:699
#13 std::call_once<void (std::__future_base::_State_baseV2::*)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*>(std::once_flag&, void (std::__future_base::_State_baseV2::*&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*&&, bool*&&) (__f=<optimized out>, __once=...)
    at /usr/include/c++/5/mutex:738
#14 std::__future_base::_State_baseV2::_M_set_result(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>, bool) (
    __ignore_failure=true, __res=..., this=0x1ca8d70) at /usr/include/c++/5/future:387
#15 std::__future_base::_Deferred_state<std::_Bind_simple<std::_Mem_fn<void (output_t::*)()> (output_t*)>, void>::_M_complete_async() (this=0x1ca8d70) at /usr/include/c++/5/future:1606
#16 0x000000000046c1f2 in std::__future_base::_State_baseV2::wait (this=0x1ca8d70) at /usr/include/c++/5/future:319
#17 std::__basic_future<void>::_M_get_result (this=0x1b9c120) at /usr/include/c++/5/future:681
#18 std::future<void>::get (this=0x1b9c120) at /usr/include/c++/5/future:846
#19 osmdata_t::stop (this=this@entry=0x7ffc23eb21e0) at /root/download/osm2pgsql/osm2pgsql/osmdata.cpp:415
#20 0x000000000045e934 in main (argc=<optimized out>, argv=<optimized out>) at /root/download/osm2pgsql/osm2pgsql/osm2pgsql.cpp:99
(gdb)

Is it OK?

[lonvia]

@jendrusk Thanks, that's exactly it. Turns out that this particular segfault has been fixed a while ago in c427abc. Update your osm2pgsql to the latest version from master and it should be okay.

[jendrusk]

Thanks Sarah, I'll do it stright away :)