Pointer being freed was not allocated during import

[jecor]

The issue arises under MacOS X (10.12), with osm2pgsql 0.92.0 and libgeos 3.6.1.

Command line:

osm2pgsql -c -G -C 8000 -s -d auvergne auvergne-latest.osm.pbf -H /tmp

(auvergne-lastest.osm.pbf can be fetched in Europe/France via geofabrik.de).

Here is the debug trace:

lldb -- osm2pgsql -c -G -C 8000 -s -d auvergne auvergne-latest.osm.pbf -H /tmp
(lldb) target create "osm2pgsql"
Current executable set to 'osm2pgsql' (x86_64).
(lldb) settings set -- target.run-args "-c" "-G" "-C" "8000" "-s" "-d" "auvergne" "auvergne-latest.osm.pbf" "-H" "/tmp"
(lldb) b malloc_error_break
Breakpoint 1: where = libsystem_malloc.dylib`malloc_error_break, address = 0x0000000000012123
(lldb) r
Process 960 launched: '/usr/local/bin/osm2pgsql' (x86_64)
osm2pgsql version 0.92.0 (64 bit id space)

Using built-in tag processing pipeline
Using projection SRS 3857 (Spherical Mercator)
Setting up table: planet_osm_point
Setting up table: planet_osm_line
Setting up table: planet_osm_polygon
Setting up table: planet_osm_roads
Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=8000MB, maxblocks=128000*65536, allocation method=11
Mid: pgsql, scale=100 cache=8000
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels

Reading in file: auvergne-latest.osm.pbf
Using PBF parser.
Processing: Node(12680k 275.7k/s) Way(0k 0.00k/s) Relation(0 0.00/s)osm2pgsql(960,0x7fffdd0663c0) malloc: *** error for object 0x7fff5fbfee08: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Process 960 stopped

• thread Fix for postgis svn 2.0 #1: tid = 0xdc9b, 0x00007fffd43ab123 libsystem_malloc.dylibmalloc_error_break, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1 frame #0: 0x00007fffd43ab123 libsystem_malloc.dylibmalloc_error_break
  libsystem_malloc.dylib`malloc_error_break:
  -> 0x7fffd43ab123 <+0>: pushq %rbp
  0x7fffd43ab124 <+1>: movq %rsp, %rbp
  0x7fffd43ab127 <+4>: nop
  0x7fffd43ab128 <+5>: nopl (%rax)
  (lldb) bt
• thread Fix for postgis svn 2.0 #1: tid = 0xdc9b, 0x00007fffd43ab123 libsystem_malloc.dylib`malloc_error_break, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
  • frame #0: 0x00007fffd43ab123 libsystem_malloc.dylibmalloc_error_break frame #1: 0x00007fffd439cfc6 libsystem_malloc.dylibfree + 409
    frame Bad SQL statement when importing with tablespaces #2: 0x000000010024fffd libgeos-3.6.1.dylibgeos::geom::Geometry::~Geometry() + 29 frame #3: 0x0000000100257641 libgeos-3.6.1.dylibgeos::geom::LineString::~LineString() + 31
    frame build error on mountain lion #4: 0x0000000100006cfa osm2pgsqlgeometry_builder::get_wkb_split(std::__1::vector<osmNode, std::__1::allocator<osmNode> > const&, int, double) const [inlined] std::__1::default_delete<geos::geom::Geometry>::operator()(geos::geom::Geometry*) const + 10 at memory:2525 [opt] frame #5: 0x0000000100006cf0 osm2pgsqlgeometry_builder::get_wkb_split(std::__1::vector<osmNode, std::__1::allocator > const&, int, double) const [inlined] std::__1::unique_ptr<geos::geom::Geometry, std::__1::default_deletegeos::geom::Geometry >::reset(__p=) + 15 at memory:2724 [opt]
    frame recent osm2pgsql does not import some polygons (dependency issue?) #6: 0x0000000100006ce1 osm2pgsqlgeometry_builder::get_wkb_split(std::__1::vector<osmNode, std::__1::allocator<osmNode> > const&, int, double) const [inlined] std::__1::unique_ptr<geos::geom::Geometry, std::__1::default_delete<geos::geom::Geometry> >::~unique_ptr() at memory:2692 [opt] frame #7: 0x0000000100006ce1 osm2pgsqlgeometry_builder::get_wkb_split(std::__1::vector<osmNode, std::__1::allocator > const&, int, double) const [inlined] std::__1::unique_ptr<geos::geom::Geometry, std::__1::default_deletegeos::geom::Geometry >::~unique_ptr() at memory:2692 [opt]
    frame Error undefined reference to `error_message' #8: 0x0000000100006ce1 osm2pgsqlgeometry_builder::get_wkb_split(this=<unavailable>, nodes=<unavailable>, polygon=<unavailable>, split_at=100000) const + 1489 at geometry-builder.cpp:328 [opt] frame #9: 0x000000010002c7bd osm2pgsqloutput_pgsql_t::pgsql_out_way(this=0x0000000100509758, id=3595351, outtags=0x00007fff5fbfeff0, nodes=size=148, polygon=, roads=1) + 125 at output-pgsql.cpp:97 [opt]
    frame make planet_osm_roads table optional and/or configurable #10: 0x000000010002ed62 osm2pgsqloutput_pgsql_t::way_add(this=0x0000000100509758, id=3595351, nds=size=148, tags=<unavailable>) + 274 at output-pgsql.cpp:387 [opt] frame #11: 0x000000010001f816 osm2pgsqlosmdata_t::way_add(this=, id=, nodes=, tags=) + 86 at osmdata.cpp:50 [opt]
    frame Add support for CZ/SK addresses #12: 0x0000000100040f89 osm2pgsqlparse_osmium_t::way(this=<unavailable>, way=<unavailable>) + 249 at parse-osmium.cpp:168 [opt] frame #13: 0x00000001000404cb osm2pgsqlvoid osmium::apply<osmium::io::Reader, parse_osmium_t>(osmium::io::Reader&, parse_osmium_t&) [inlined] void osmium::detail::apply_item_recurse<parse_osmium_t, osmium::memory::Item>(handler=) + 187 at visitor.hpp:67 [opt]
    frame No import any nodes where id>type integer  #14: 0x0000000100040488 osm2pgsqlvoid osmium::apply<osmium::io::Reader, parse_osmium_t>(osmium::io::Reader&, parse_osmium_t&) [inlined] void osmium::apply<osmium::io::InputIterator<osmium::io::Reader, osmium::memory::Item>, parse_osmium_t>(osmium::io::InputIterator<osmium::io::Reader, osmium::memory::Item>, osmium::io::InputIterator<osmium::io::Reader, osmium::memory::Item>, parse_osmium_t&) + 55 at visitor.hpp:236 [opt] frame #15: 0x0000000100040451 osm2pgsqlvoid osmium::apply<osmium::io::Reader, parse_osmium_t>(c=, handlers=) + 65 at visitor.hpp:243 [opt]
    frame duplicate key error in slim mode #16: 0x000000010003ff16 osm2pgsqlparse_osmium_t::stream_file(this=0x00007fff5fbff6a8, filename="auvergne-latest.osm.pbf", fmt="auto") + 262 at parse-osmium.cpp:123 [opt] frame #17: 0x0000000100003e7e osm2pgsqlmain(argc=, argv=) + 542 at osm2pgsql.cpp:88 [opt]
    frame PBF reader uses long int instead of osmid_t for osm ids #18: 0x00007fffd4219255 libdyld.dylibstart + 1 frame #19: 0x00007fffd4219255 libdyld.dylibstart + 1

[lonvia]

Geos 3.6 is known to break osm2pgsql code, see #634. You need libgeos 3.5 or less.

[jecor]

Thanks for your quick response Sarah, and sorry for the disturbance (went through the existing issues quite lightly).

Working fine with geos 3.5 (unfortunately no longer available through standard MacPorts package manager; had to built it from source).

Maybe this could be documented in README.md?

Thanks again,

Jérôme

[lonvia]

I'm surprised that it even compiles for you. #684 will remove the geos dependency on master, so that is taken care of. But I suspect we need a test in CMakeLists for the release.

Closing here, #634 is still open for further comments.