recent osm2pgsql does not import some polygons (dependency issue?)

[kingcu]

On a Debian Squeeze box, the master branch of osm2pgsql imports everything as expected from a planet.osm file. However, the same repository on an ubuntu 12.04 install misses some polygon features like forests and the rest of a river after a damn, some islands etc.

I tested osm2pgsql on a debian box, pointing at the database on my new ubuntu machine. So, it's not the postgres server, but probably one of the libraries that osm2pgsql depends on for parsing. Same problem happens on the PBF or XML file.

Where do I start to get this figured out? I have a new shiny new dedicated tile server and need to migrate off the old server. Due to dependency hell I have not been able to get the exact versions of libgeos and postgis running on the new ubuntu server.

[springmeyer]

My guess would be differing geos versions. But the only way to track this down for sure would be to have a reduced/very small testcase that could be shared so that others could run it and dig into the issue.

[apmon]

No idea how that can happen, but one potential place to start looking for the difference are the "isValid" checks in build_geometry.cpp and the "--exclude-invalid-polygon" command line option.

osm2pgsql either just ignores invalid polygons, dropping them from the database, or tries to fix them by using the buffer(0) call. Perhaps these checks behave differently from version to version of geos?

[apmon]

Closing this as outdated. Things have changed since then in the polygon processing code, so it would need to be reconfirmed that this is an issue with current versions of osm2pgsql.

[kingcu]

I can confirm this is still an issue. Installing osm2pgsql on ubuntu 12.04 using apt yields a binary that can successfully import a regional dump containing the willamette river (osm_id of 215160). Compiling from source on ubuntu 12.04 (separate machine) yields a binary that, with the same region dump, will not import a polygon of the river.

From apt, osm2pgsql version is 0.81.0. I checked out the 0.81.0 tag in git, which yielded the same problem.

Finally, removing the apt installed libgeos and gdal, and installing libgeos-3.3.5 and gdal-1.9.2 from source, recompiling osm2pgsql results in correct imports of my regional extract.

[pnorman]

Finally, removing the apt installed libgeos and gdal, and installing libgeos-3.3.5 and gdal-1.9.2 from source, recompiling osm2pgsql results in correct imports of my regional extract.

Sounds like a geos problem then, not an osm2pgsql problem. Do you have a way or relation ID?

Edit: added relation

[kingcu]

Thanks for the response, yes it definitely seems that it's likely a geos problem, I just wanted to ensure that the issue was documented/noted for other people to find.

Relation ID is 215160: http://www.openstreetmap.org/relation/215160

This is not the only issue, a considerable amount of inland rivers are not present on the import.

[pnorman]

Are we certain this is a geos problem rather than geos handling invalid geometries differently on different versions?

Can you post the file you're using?

[kingcu]

Here you go:
https://dl.dropboxusercontent.com/u/195961/planet_oregon_extract.pbf

On Tue, May 6, 2014 at 9:38 AM, Paul Norman notifications@github.comwrote:

Are we certain this is a geos problem rather than geos handling invalid
geometries differently on different versions?

Can you post the file you're using?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/6#issuecomment-42326503
.

[pnorman]

Where was this from? osmconvert and osmosis produce an invalid OSM file when told to convert it to XML, with invalid versions, timestamps, changesets, and no authors.

https://wiki.openstreetmap.org/wiki/Osmconvert#Dispose_of_Author_Information may be relevant.

Usually you will not encounter any problems when deleting the author information from .osm or .o5m files, however it is not encouraged to do this with .pbf files because most programs will not cope with this change of format

[kingcu]

That was from an online tool to extract a region from the entire dump.
Here's another, different region extract from mapzen that has the same
issue. mapzen states their planet import was last updated 3/28/2014.

https://s3.amazonaws.com/metro-extracts.mapzen.com/portland.osm.pbf

On Tue, May 6, 2014 at 9:54 AM, Paul Norman notifications@github.comwrote:

Where was this from? osmconvert and osmosis produce an invalid OSM file
when told to convert it to XML, with invalid versions, timestamps,
changesets, and no authors.

https://wiki.openstreetmap.org/wiki/Osmconvert#Dispose_of_Author_Informationmay be relevant.

Usually you will not encounter any problems when deleting the author
information from .osm or .o5m files, however it is not encouraged to do
this with .pbf files because most programs will not cope with this change
of format

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/6#issuecomment-42328553
.