Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
OpenStreetMap data to PostgreSQL converter
C++ Python Lua Other

Merge pull request #333 from pnorman/multi_tag_processing

Fix a bug where modified tags weren't being copied from Lua
latest commit 9dc967972e
@pnorman pnorman authored
Failed to load latest commit information.
docs Add documentation on how to migrate between versions of osm2pgsql
geos-fallback/geos/noding Add a few header files for people compiling under Ubuntu. geos-dev
m4 Update ax_lua.m4
protobuf * improved configure setup (including automake and libtool)
tests Fix a bug where modified tags weren't being copied from Lua
.gitignore Always resize the node_cache in geometry-processor
.travis.yml Clean up whitespace errors
900913.sql Add postgis definition for the 900913 spherical mercator projection w…
AUTHORS Add a few more authors to the AUTHORS file who seem to have done a lo…
COPYING * improved configure setup (including automake and libtool)
ChangeLog Revert "Remove the old SVN Changelog"
Makefile.am new types for keyval and for tag, member and node lists
NEWS started a NEWS file
README Trim down README and move more detailed information into docs directory
README.md Fix missing backslash in prerequisites
TODO remove outdated gazetteer V1 code,
UTF8sanitizer.cpp Clean up whitespace errors
autogen.sh * improved configure setup (including automake and libtool)
configure.ac reenable Wall and fix resulting warnings
cygpkg.sh default.style was missing from cygwin package.
default.style Add recommended types to columns for extra attributes
empty.style Add abandoned: tags to phstore list
expire-tiles.cpp reenable Wall and fix resulting warnings
expire-tiles.hpp new types for keyval and for tag, member and node lists
geometry-builder.cpp reenable Wall and fix resulting warnings
geometry-builder.hpp new types for keyval and for tag, member and node lists
geometry-processor.cpp reenable Wall and fix resulting warnings
geometry-processor.hpp new types for keyval and for tag, member and node lists
id-tracker.cpp remove pending tracking from the ram middle processing as now the mid…
id-tracker.hpp new types for keyval and for tag, member and node lists
input.cpp Clean up whitespace errors
input.hpp Renamed C files to C++ in preparation for destaticification.
install-postgis-osm-db.sh Fix syntax error in install script
install-postgis-osm-user.sh [osm2pgsql] Update debian package scripts to handle postgis-9.1
middle-pgsql.cpp reenable Wall and fix resulting warnings
middle-pgsql.hpp new types for keyval and for tag, member and node lists
middle-ram.cpp reenable Wall and fix resulting warnings
middle-ram.hpp new types for keyval and for tag, member and node lists
middle.cpp new types for keyval and for tag, member and node lists
middle.hpp new types for keyval and for tag, member and node lists
multi.lua Rework sample multi config
multi.style.json Rework sample multi config
node-persistent-cache-reader.cpp new types for keyval and for tag, member and node lists
node-persistent-cache.cpp reenable Wall and fix resulting warnings
node-persistent-cache.hpp new types for keyval and for tag, member and node lists
node-ram-cache.cpp new types for keyval and for tag, member and node lists
node-ram-cache.hpp new types for keyval and for tag, member and node lists
options.cpp Clean up help spacing
options.hpp new types for keyval and for tag, member and node lists
osm2pgsql.cpp remove text-tree and make keyval a real C++ class
osm2pgsql.spec.in * improved configure setup (including automake and libtool)
osmdata.cpp new types for keyval and for tag, member and node lists
osmdata.hpp fix more warnings where clang complains
osmtypes.hpp correctly overwrite way_area for each polygon part
output-gazetteer.cpp new types for keyval and for tag, member and node lists
output-gazetteer.hpp fix more warnings where clang complains
output-multi.cpp Fix a bug where modified tags weren't being copied from Lua
output-multi.hpp new types for keyval and for tag, member and node lists
output-null.cpp new types for keyval and for tag, member and node lists
output-null.hpp new types for keyval and for tag, member and node lists
output-pgsql.cpp second attempt to fix broken way_area assignment
output-pgsql.hpp new types for keyval and for tag, member and node lists
output.cpp refactored a bunch of native pointers to smart ones. tests still fail…
output.hpp new types for keyval and for tag, member and node lists
parse-o5m.cpp reenable Wall and fix resulting warnings
parse-o5m.hpp Clean up whitespace errors
parse-pbf.cpp reenable Wall and fix resulting warnings
parse-pbf.hpp new types for keyval and for tag, member and node lists
parse-xml2.cpp new types for keyval and for tag, member and node lists
parse-xml2.hpp new types for keyval and for tag, member and node lists
parse.cpp new types for keyval and for tag, member and node lists
parse.hpp new types for keyval and for tag, member and node lists
pgsql-id-tracker.cpp Clean up whitespace errors
pgsql-id-tracker.hpp Clean up whitespace errors
pgsql.cpp rewrite tag handling of gazetteer output as class
pgsql.hpp rewrite tag handling of gazetteer output as class
processor-line.cpp new types for keyval and for tag, member and node lists
processor-line.hpp new types for keyval and for tag, member and node lists
processor-point.cpp change the interface in build_geometry so that it can be better used …
processor-point.hpp change the interface in build_geometry so that it can be better used …
processor-polygon.cpp new types for keyval and for tag, member and node lists
processor-polygon.hpp new types for keyval and for tag, member and node lists
rb.cpp Clean up whitespace errors
rb.hpp Renamed C files to C++ in preparation for destaticification.
reprojection.cpp Clean up whitespace errors
reprojection.hpp Clean up whitespace errors
sample.multi.json fix tab in sample multi config
sanitizer.hpp Renamed C files to C++ in preparation for destaticification.
sprompt.cpp Clean up whitespace errors
sprompt.hpp Renamed C files to C++ in preparation for destaticification.
style.lua Fix style.lua error with non numerical layer values
table.cpp reenable Wall and fix resulting warnings
table.hpp new types for keyval and for tag, member and node lists
taginfo.cpp remove unnecessary memory-leaking strdup
taginfo.hpp Removed a lot of static, global state variables from `output_pgsql_t`.
taginfo_impl.hpp Add more missing includes for std::pair
tagtransform.cpp reenable Wall and fix resulting warnings
tagtransform.hpp new types for keyval and for tag, member and node lists
util.cpp replace printf with boost::format
util.hpp replace printf with boost::format
wildcmp.cpp Clean up whitespace errors
wildcmp.hpp Wildcard matching should operate on const char arrays.
win_fsync.h fix headers and missing functions for MSVC/MinGW

README.md

osm2pgsql

osm2pgsql is a tool for loading OpenStreetMap data into a PostgreSQL / PostGIS database suitable for applications like rendering into a map, geocoding with Nominatim, or general analysis.

Features

  • Converts OSM files to a PostgreSQL DB
  • Conversion of tags to columns is configurable in the style file
  • Able to read .gz, .bz2, .pbf and .o5m files directly
  • Can apply diffs to keep the database up to date
  • Support the choice of output projection
  • Configurable table names
  • Gazetteer back-end for Nominatim
  • Support for hstore field type to store the complete set of tags in one database field if desired

Installing

The latest source code is available in the OSM git repository on github and can be downloaded as follows:

$ git clone git://github.com/openstreetmap/osm2pgsql.git

Building

Osm2pgsql uses the GNU Build System to configure and build itself and requires

It also requires access to a database server running PostgreSQL and PostGIS.

Make sure you have installed the development packages for the libraries mentioned in the requirements section and a C and C++ compiler.

To install on a Debian or Ubuntu system, first install the prerequisites:

sudo apt-get install autoconf automake libtool make g++ libboost-dev \
  libboost-system-dev libboost-filesystem-dev libboost-thread-dev libxml2-dev \
  libgeos-dev libgeos++-dev libpq-dev libbz2-dev libproj-dev \
  protobuf-c-compiler libprotobuf-c0-dev lua5.2 liblua5.2-dev

To install on a Fedora system, use

sudo yum install gcc-c++ boost-devel libxml2-devel geos-devel \
  postgresql-devel bzip2-devel proj-devel protobuf-compiler

To install on a FreeBSD system, use

pkg install devel/git devel/autoconf devel/automake devel/gmake devel/libtool \
  textproc/libxml2 graphics/geos graphics/proj databases/postgresql94-client \
  devel/boost-libs devel/protobuf-c lang/lua52 devel/pkgconf

Then you should be able to bootstrap the build system:

./autogen.sh

And then run the standard GNU build install:

./configure && make && make install

Please see ./configure --help for more options on how to control the build process.

On FreeBSD instead bootstrap and then run

LUA_LIB=`pkg-config --libs lua-5.2` ./configure && gmake && gmake install

Usage

Osm2pgsql has one program, the executable itself, which has 43 command line options.

Before loading into a database, the database must be created and the PostGIS and optionally hstore extensions must be loaded. A full guide to PostgreSQL setup is beyond the scope of this readme, but with reasonably recent versions of PostgreSQL and PostGIS this can be done with

createdb gis
psql -d gis -c 'CREATE EXTENSION postgis; CREATE EXTENSION hstore;'

A basic invocation to load the data into the database gis for rendering would be

osm2pgsql --create --database gis data.osm.pbf

This will load the data from data.osm.pbf into the planet_osm_point, planet_osm_line, planet_osm_roads, and planet_osm_polygon tables.

When importing a large amount of data such as the complete planet, a typical command line would be

osm2pgsql -c -d gis --slim -C <cache size> \
  --flat-nodes <flat nodes> planet-latest.osm.pbf

where

  • <cache size> is 24000 on machines with 32GiB or more RAM or about 75% of memory in MiB on machines with less
  • <flat nodes> is a location where a 24GiB file can be saved.

The databases from either of these commands can be used immediately by Mapnik for rendering maps with standard tools like renderd/mod_tile, TileMill, Nik4, among others. It can also be used for spatial analysis or shapefile exports.

Additional documentation is available on writing command lines.

Alternate backends

In addition to the standard pgsql backend designed for rendering there is also the gazetteer database for geocoding, principally with Nominatim, and the null backend for testing. For flexibility a new multi backend is also avialable which allows the configuration of custom postgres tables instead of those provided in the pgsql backend.

Any questions should be directed at the osm dev list http://wiki.openstreetmap.org/index.php/Mailing_lists

Testing

The code also comes with a suite of tests which can be run by executing make check.

Some of these tests depend on being able to set up a database and run osm2pgsql against it. You need to ensure that PostgreSQL is running and that your user is a superuser of that system. To do that, run:

sudo -u postgres createuser -s $USER
sudo mkdir -p /tmp/psql-tablespace
sudo chown postgres.postgres /tmp/psql-tablespace
psql -c "CREATE TABLESPACE tablespacetest LOCATION '/tmp/psql-tablespace'" postgres

Once this is all set up, all the tests should run (no SKIPs), and pass (no FAILs). If you encounter a failure, you can find more information by looking in the test-suite.log. If you find something which seems to be a bug, please check to see if it is a known issue at https://github.com/openstreetmap/osm2pgsql/issues and, if it's not already known, report it there.

Contributing

We welcome contributions to osm2pgsql. If you would like to report an issue, please use the issue tracker on GitHub.

General queries can be sent to the tile-serving@ or dev@ mailing lists.

Something went wrong with that request. Please try again.