OpenStreetMap data to PostgreSQL converter
C++ Python CMake Lua Other
Latest commit a352ee1 Jul 18, 2016 @pnorman pnorman committed on GitHub Merge pull request #608 from WillieMaddox/patch-1
typo in man page
Failed to load latest commit information.
cmake remove some unused system checks and defines Jun 1, 2016
contrib/libosmium update to libosmium release 2.6.1 Feb 27, 2016
docs typo in man page Jul 18, 2016
tests make tile expiry a proper C++ class Jun 4, 2016
.gitignore Clean up gitignore, now that all build files are in a build dir Oct 25, 2015
.travis.yml also remove boost-thread for travis and readme Nov 8, 2015
AUTHORS Hoffmann has two n's Jul 7, 2015
CMakeLists.txt remove some unused system checks and defines Jun 1, 2016
CONTRIBUTING.md remove whitespace before EOL Nov 2, 2015
COPYING * improved configure setup (including automake and libtool) Nov 6, 2010
ChangeLog Revert "Remove the old SVN Changelog" Oct 3, 2013
README Trim down README and move more detailed information into docs directory Sep 3, 2014
README.md Add information on distro options for osm2pgsql Jul 17, 2016
appveyor.yml make appveyor more verbose Nov 9, 2015
default.style Remove entirely unused POI tag Apr 3, 2016
empty.style remove whitespace before EOL Nov 2, 2015
expire-tiles.cpp make tile expiry a proper C++ class Jun 4, 2016
expire-tiles.hpp make tile expiry a proper C++ class Jun 4, 2016
geometry-builder.cpp fix error message Feb 27, 2016
geometry-builder.hpp rework projection code Jan 17, 2016
geometry-processor.cpp rework projection code Jan 17, 2016
geometry-processor.hpp rename wkt_t types and move them to stack where possible Dec 3, 2015
id-tracker.cpp remove shared_ptr for id_trackers Mar 30, 2016
id-tracker.hpp remove shared_ptr for id_trackers Mar 30, 2016
install-postgis-osm-db.sh remove whitespace before EOL Nov 2, 2015
install-postgis-osm-user.sh remove whitespace before EOL Nov 2, 2015
middle-pgsql.cpp avoid using alloca() to allocate a fixed size buffer Jun 2, 2016
middle-pgsql.hpp middle-pgsql: use std::async for parallel indexing Nov 8, 2015
middle-ram.cpp Re-order headers in modified files Oct 20, 2015
middle-ram.hpp constexpr was too complex for MSVC Aug 17, 2015
middle.cpp Replace boost::shared_ptr with std::shared_ptr Jul 18, 2015
middle.hpp Improve Doxygen syntax Jul 21, 2015
multi.lua remove whitespace before EOL Nov 2, 2015
multi.style.json Fix builing typo in example multi style Apr 29, 2015
node-persistent-cache.cpp fix some warnings about int-size_t mismatch Oct 29, 2015
node-persistent-cache.hpp Replace boost::shared_ptr with std::shared_ptr Jul 18, 2015
node-ram-cache.cpp Windows: Fix msvc build Dec 3, 2015
node-ram-cache.hpp Use long ints for tracking cache hits Feb 27, 2016
options.cpp make tile expiry a proper C++ class Jun 4, 2016
options.hpp make tile expiry a proper C++ class Jun 4, 2016
osm2pgsql.cpp hasn't been SVN in a while Feb 28, 2016
osmdata.cpp make expire a simple member instead of using shared_ptr Mar 30, 2016
osmdata.hpp Replace boost::shared_ptr with std::shared_ptr Jul 18, 2015
osmtypes.hpp Add a taglist function for adding/replacing an existing tag Nov 1, 2015
output-gazetteer.cpp Display an error message from the right connection Apr 28, 2016
output-gazetteer.hpp remove unused static NAME members from output plugins Mar 2, 2016
output-multi.cpp make tile expiry a proper C++ class Jun 4, 2016
output-multi.hpp remove shared_ptr for id_trackers Mar 30, 2016
output-null.cpp Replace boost::shared_ptr with std::shared_ptr Jul 18, 2015
output-null.hpp Replace boost::shared_ptr with std::shared_ptr Jul 18, 2015
output-pgsql.cpp make tile expiry a proper C++ class Jun 4, 2016
output-pgsql.hpp remove shared_ptr for id_trackers Mar 30, 2016
output.cpp make expire a simple member instead of using shared_ptr Mar 30, 2016
output.hpp make expire a simple member instead of using shared_ptr Mar 30, 2016
parse-osmium.cpp rework projection code Jan 17, 2016
parse-osmium.hpp rework projection code Jan 17, 2016
pgsql.cpp replace C casts with C++ casts Jun 1, 2016
pgsql.hpp explicit type conversions to reduce MSVC warnings Oct 29, 2015
processor-line.cpp Process relations for line tables Jun 16, 2016
processor-line.hpp Process relations for line tables Jun 17, 2016
processor-point.cpp rename wkt_t types and move them to stack where possible Dec 3, 2015
processor-point.hpp rename wkt_t types and move them to stack where possible Dec 3, 2015
processor-polygon.cpp rename wkt_t types and move them to stack where possible Dec 3, 2015
processor-polygon.hpp rename wkt_t types and move them to stack where possible Dec 3, 2015
reprojection.cpp fix projection info for generic projection Apr 20, 2016
reprojection.hpp Switch from 900913 to 3857 Feb 27, 2016
sample.multi.json fix tab in sample multi config Dec 6, 2014
sprompt.cpp entirely avoid dynamic allocation for a single DWORD, put it on the s… Jun 2, 2016
sprompt.hpp Renamed C files to C++ in preparation for destaticification. May 12, 2014
style.lua Sort generic_keys and polygon_keys Jun 19, 2016
table.cpp Use C collation for ST_GeoHash ordering May 17, 2016
table.hpp rename wkt_t types and move them to stack where possible Dec 3, 2015
taginfo.cpp replace tagflags array with simpler std::map Feb 28, 2016
taginfo.hpp Removed a lot of static, global state variables from `output_pgsql_t`. May 14, 2014
taginfo_impl.hpp replace tagflags array with simpler std::map Feb 28, 2016
tagtransform.cpp rework wildcard code Feb 28, 2016
tagtransform.hpp Use foreach/auto loops in tagtransform Jul 19, 2015
util.cpp replace printf with boost::format Jan 16, 2015
util.hpp remove scale everywhere but in ramNode May 30, 2015
wildcmp.cpp rework wildcard code Feb 28, 2016
wildcmp.hpp rework wildcard code Feb 28, 2016
win_fsync.h fix headers and missing functions for MSVC/MinGW Oct 2, 2014

README.md

osm2pgsql

osm2pgsql is a tool for loading OpenStreetMap data into a PostgreSQL / PostGIS database suitable for applications like rendering into a map, geocoding with Nominatim, or general analysis.

Features

  • Converts OSM files to a PostgreSQL DB
  • Conversion of tags to columns is configurable in the style file
  • Able to read .gz, .bz2, .pbf and .o5m files directly
  • Can apply diffs to keep the database up to date
  • Support the choice of output projection
  • Configurable table names
  • Gazetteer back-end for Nominatim
  • Support for hstore field type to store the complete set of tags in one database field if desired

Installing

Most Linux distributions include osm2pgsql. It is also available on macOS with Homebrew. Unoffical builds for Windows are built by AppVeyor but you need to find the right build artifacts.

The latest source code is available in the OSM git repository on github and can be downloaded as follows:

$ git clone git://github.com/openstreetmap/osm2pgsql.git

Building

Osm2pgsql uses the cross-platform CMake build system to configure and build itself and requires

Required libraries are

It also requires access to a database server running PostgreSQL 9.1+ and PostGIS 2.0+.

Make sure you have installed the development packages for the libraries mentioned in the requirements section and a C++ compiler which supports C++11. Both GCC 4.8 and Clang 3.4 meet this requirement.

First install the dependencies.

On a Debian or Ubuntu system, this can be done with:

sudo apt-get install make cmake g++ libboost-dev libboost-system-dev \
  libboost-filesystem-dev libexpat1-dev zlib1g-dev \
  libbz2-dev libpq-dev libgeos-dev libgeos++-dev libproj-dev lua5.2 \
  liblua5.2-dev

On a Fedora system, use

sudo yum install cmake gcc-c++ boost-devel expat-devel zlib-devel bzip2-devel \
  postgresql-devel geos-devel proj-devel proj-epsg lua-devel

On RedHat / CentOS first run sudo yum install epel-release then install dependencies like on Fedora.

On a FreeBSD system, use

pkg install devel/cmake devel/boost-libs textproc/expat2 \
  databases/postgresql94-client graphics/geos graphics/proj lang/lua52

Once dependencies are installed, use CMake to build the Makefiles in a separate folder

mkdir build && cd build
cmake ..

If some installed dependencies are not found by CMake, more options may need to be set. Typically, setting CMAKE_PREFIX_PATH to a list of appropriate paths is sufficient.

When the Makefiles have been successfully built, compile with

make

The compiled files can be installed with

sudo make install

By default, the Release build with debug info is created and no tests are compiled. You can change that behavior by using additional options like following:

cmake .. -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTS=ON

Usage

Osm2pgsql has one program, the executable itself, which has 44 command line options.

Before loading into a database, the database must be created and the PostGIS and optional hstore extensions must be loaded. A full guide to PostgreSQL setup is beyond the scope of this readme, but with reasonably recent versions of PostgreSQL and PostGIS this can be done with

createdb gis
psql -d gis -c 'CREATE EXTENSION postgis; CREATE EXTENSION hstore;'

A basic invocation to load the data into the database gis for rendering would be

osm2pgsql --create --database gis data.osm.pbf

This will load the data from data.osm.pbf into the planet_osm_point, planet_osm_line, planet_osm_roads, and planet_osm_polygon tables.

When importing a large amount of data such as the complete planet, a typical command line would be

osm2pgsql -c -d gis --slim -C <cache size> \
  --flat-nodes <flat nodes> planet-latest.osm.pbf

where

  • <cache size> is about 75% of memory in MiB, to a maximum of about 30000. Additional RAM will not be used.
  • <flat nodes> is a location where a 24GiB file can be saved.

The databases from either of these commands can be used immediately by Mapnik for rendering maps with standard tools like renderd/mod_tile, TileMill, Nik4, among others. It can also be used for spatial analysis or shapefile exports.

Additional documentation is available on writing command lines.

Alternate backends

In addition to the standard pgsql backend designed for rendering there is also the gazetteer database for geocoding, principally with Nominatim, and the null backend for testing. For flexibility a new multi backend is also available which allows the configuration of custom PostgreSQL tables instead of those provided in the pgsql backend.

Contributing

We welcome contributions to osm2pgsql. If you would like to report an issue, please use the issue tracker on GitHub.

More information can be found in CONTRIBUTING.md.

General queries can be sent to the tile-serving@ or dev@ mailing lists.