diff --git a/README b/README deleted file mode 100644 index 7b8923600..000000000 --- a/README +++ /dev/null @@ -1,308 +0,0 @@ -osm2pgsql -========= -Converts OSM planet.osm data to a PostgreSQL / PostGIS database suitable -for specific applications like rendering into map tiles by Mapnik -or geocoding with Nominatim. - -osm2pgsql currently supports two different database schemas -1) A database schema that is optimized for ease of rendering -by Mapnik. -2) A database schema that is optimized for geocoding with Nominatim, -emphasizing the spatially hierarchical organizations of objects. - -Both schemas were specifically optimized for the purpose they were -intended for and they may therefore be less suitable for other -general purpose processing. Nevertheless, the rendering schema -might be useful for other purposes as well, and has been used -for a variety of additionally purposes. - - -For a broader view of the whole map rendering tool chain see -http://wiki.openstreetmap.org/index.php/Mapnik -http://wiki.openstreetmap.org/index.php/Osm2pgsql -http://wiki.openstreetmap.org/index.php/Slippy_Map - -You may find that the wiki pages are more up to date than this -readme and may include answers to issues not mentioned here. - -Any questions should be directed at the osm dev list -http://wiki.openstreetmap.org/index.php/Mailing_lists - -Features -======== -- Converts OSM files to a PostgreSQL DB -- Conversion of tags to columns is configurable in the style file -- Able to read .gz, .bz2, .pbf and .o5m files directly -- Can apply diffs to keep the database up to date -- Support the choice of output projection -- Configurable table names -- Gazetteer back-end for Nominatim - http://wiki.openstreetmap.org/wiki/Nominatim -- Support for hstore field type to store the complete set of tags in one database - field if desired - -Source code -=========== -The latest source code is available in the OSM git repository on github -and can be downloaded as follows: - -$ git clone git://github.com/openstreetmap/osm2pgsql.git - -Build requirements -================== -The code is written in C and C++ and relies on the libraries -below: -- libxml2 http://xmlsoft.org/ -- geos http://geos.refractions.net/ -- proj http://www.remotesensing.org/proj/ -- bzip2 http://www.bzip.org/ -- zlib http://www.zlib.net/ -- PostgreSQL http://www.postgresql.org/ -- PostGIS http://postgis.refractions.net/ - -To make use of the database generated by this tool you will -probably also want to install: -- Mapnik from http://mapnik.org/ - -Building -======== -Make sure you have installed the development packages for the -libraries mentioned in the requirements section and a C and C++ -compiler. - -e.g. on Fedora: -# yum install geos-devel proj-devel postgresql-devel libxml2-devel bzip2-devel gcc-c++ - -on Debian: -# aptitude install libxml2-dev libgeos-dev libgeos++-dev libpq-dev libbz2-dev libproj-dev protobuf-c-compiler libprotobuf-c0-dev autoconf automake libtool make g++ libbz2-dev - -To use lua tag transforms you need a lua interpreter and development packages installed on your system. These are normally lua5.2 and liblua5.2-dev - -On most Unix-like systems the program can be compiled by -running './autogen.sh && ./configure && make'. - -Operation -========= -You must create a PostgreSQL user and a database with the -PostGIS functions enabled. This requires access as the -database administrator, normally the 'postgres' user. - -PostgreSQL 9.1 and PostGIS 2.0 or later are strongly suggested -for databases in production. It is generally best to run the -latest released versions if possible. PostgreSQL 8.4 and PostGIS 1.5 -will work but are substantially slower. Additionally, PostGIS 2.0 -contains enhancements that increase reliability as well as add new -features that style sheet authors can use. - -The default name for this database is 'gis' but this may -be changed by using the osm2pgsql --database option. - -If the matches the unix user id running the import -and rendering then this allows the PostgreSQL 'ident sameuser' -authentication to be used which avoids the need to enter a -password when accessing the database. This is setup by default -on many Unix installs but does not work on Windows (due to the -lack of unix sockets). - -Some example commands are given below but you may find -this wiki page has more up to date information: -http://wiki.openstreetmap.org/wiki/Mapnik/PostGIS - -$ sudo -u postgres createuser -$ sudo -u postgres createdb -E UTF8 -O -$ sudo -u postgres createlang plpgsql - -Adding the PostGIS extensions. -$ sudo -u postgres psql -d -c "CREATE EXTENSION postgis;" - -On older versions of PostGIS you will have to run .sql scripts. Note the location of the -files may vary. - -$ sudo -u postgres psql -d -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql -$ sudo -u postgres psql -d -f /usr/share/postgresql/8.4/contrib/postgis-1.5/spatial_ref_sys.sql - -Next we need to give the access to update the postgis -meta-data tables - -$ sudo -u postgres psql -d -c "ALTER TABLE geometry_columns OWNER TO " -$ sudo -u postgres psql -d -c "ALTER TABLE spatial_ref_sys OWNER TO " - -The 900913 is not normally included with PostGIS. To add it you -should run: - -$ sudo psql -u postgres psql -d -f 900913.sql - -If you want to use hstore support then you will also need to enable the PostgreSQL -hstore-new extension. - -$ sudo -u postgres psql -d -c "CREATE EXTENSION hstore;" - -On PostgreSQL versions before 9.1 you need to install the hstore-new extension instead - -$ sudo -u postgres psql -d -f /usr/share/postgresql/8.4/contrib/hstore-new.sql - -Now you can run osm2pgsql to import the OSM data. -This will perform the following actions: - -1) osm2pgsql connects to database and creates the following 4 tables -when used with the default output back-end (pgsql): - - planet_osm_point - - planet_osm_line - - planet_osm_roads - - planet_osm_polygon -The default prefix "planet_osm" can be changed with the --prefix option. - -If you are using --slim mode, it will create the following additional 3 tables: - - planet_osm_nodes - - planet_osm_ways - - planet_osm_rels - -2) Runs a parser on the input file (typically planet-latest.osm.pbf) - and processes the nodes, ways and relations. - -3) If a node has a tag declared in the style file then it is - added to planet_osm_point. If it has no such tag then - the position is noted, but not added to the database. - -4) Ways are read in converted into WKT geometries by using the - positions of the nodes read in earlier. If the tags on the way - are listed in the style file then the way will be written into - the line or roads tables. - -5) If the way has one or more tags marked as 'polygon' and - forms a closed ring then it will be added to the planet_osm_polygon - table. - -6) The relations are parsed. Osm2pgsql has special handling for a - limited number of types: multipolygon, route, boundary - The code will build the appropriate geometries by referencing the - members and outputting these into the database. - -7) Indexes are added to speed up the queries by Mapnik. - -Tuning PostgreSQL -================= - -For an efficient operation of PostgreSQL you will need to tune the config -parameters of PostgreSQL from its default values. These are set in the -config file at /etc/postgresql/9.1/main/postgresql.conf - -The values you need to set will depend on the hardware you have available, -but you will likely need to increase the values for the following parameters: - -- shared_buffers -- checkpoint_segments -- work_mem -- maintenance_work_mem -- effective_cache_size - - -A quick note on projections -=========================== - -Depending on the command-line switches you can select which projection you -want the database in. You have three choices: - -4326: The standard lat/long coordinates -900913: The spherical Mercator projection, used by TileCache, Google Earth etc. -3395: The legacy (broken) WGS84 Mercator projection - -Depending on what you're using one or the other is appropriate. The default -Mapnik style (osm.xml) assumes that the data is stored in 900913 and this -is the default for osm2pgsql. - -Combining the -v and -h switches will tell about the exact definitions of -the projections. - -In case you want to use some completely different projection there is the -E -option. It will initialize the projection as +init=epsg:. This allows -you to use any projection recognized by proj4, which is useful if you want -to make a map in a different projection. These projections are usually -defined in /usr/share/proj/epsg. - -Database Access Examples -======================== -If you wish to access the data from the database then the -queries below should give you some hints. Note that these -examples all use the 'latlong' projection which is not the -default. - -$ psql gis -gis=> \d - List of relations - Schema | Name | Type | Owner ---------+--------------------+-------+---------- -... - public | planet_osm_line | table | jburgess - public | planet_osm_point | table | jburgess - public | planet_osm_polygon | table | jburgess - public | planet_osm_roads | table | jburgess -... - -gis=> \d planet_osm_line - Table "public.planet_osm_line" - Column | Type | Modifiers ------------+----------+----------- - osm_id | integer | - name | text | - place | text | - landuse | text | -... [ lots of stuff deleted ] ... - way | geometry | not null - z_order | integer | default 0 - - -Each of the tables contains a subset of the planet.osm file representing -a particular geometry type -- Point contains nodes which have interesting tags - e.g. place=city, name=London - -- Line contains ways with interesting tags - e.g. highway=motorway, ref=M25 - -- Polygon contains ways which form an enclosed area - e.g. landuse=reservoir - -The DB columns are used as follows: -- osm_id = the planet.osm ID of the node(point) or way(line,polygon) -- name, place, landuse, ... = the value of the given key, if present on -the node/way. If the tag is not present, the value is NULL. Only a -subset of all possible tags are stored in the DB. Only ones rendered in -the osm.xml are actually interesting to mapnik. -- way = PostGIS geometry describing the physical layout of the object. - -Querying specific data requires knowlege of SQL and the OSM key/value -system, e.g. - -gis=> select osm_id,astext(way),name from planet_osm_point where amenity='cinema' limit 5; - osm_id | astext | name -----------+-------------------------------------------+-------------------- - 26236284 | POINT(-79.7160836579093 43.6802306464618) | - 26206699 | POINT(51.4051989797638 35.7066045032235) | Cinema Felestin - 26206700 | POINT(51.3994885141459 35.7058460359352) | Cinema Asr-e Jadid - 20979630 | POINT(151.225781789807 -33.8943079539886) | Paris Cinema - 20979684 | POINT(151.226855394904 -33.8946830511095) | Hoyts -(5 rows) - -Mapnik renders the data in each table by applying the rules in the -osm.xml file. - - -> How could I get e.g. all highways in a given bounding box? - -The 'way' column contains the geo info and is the one which you need to -use in your WHERE clause. e.g. - -gis=> select osm_id,highway,name from planet_osm_line where highway is not null and way && GeomFromText('POLYGON((0 52, 0.1 52, 0.1 52.1, 0 52.1, 0 52))',4326); - -osm_id | highway | name ----------+--------------+------------------ - 4273848 | unclassified | - 3977133 | trunk | to Royston (tbc) - 4004841 | trunk | - 4019198 | trunk | - 4019199 | trunk | - 4238966 | unclassified | - -See the Postgis docs for details, e.g. -http://postgis.net/docs/manual-2.0/reference.html diff --git a/README b/README new file mode 120000 index 000000000..42061c01a --- /dev/null +++ b/README @@ -0,0 +1 @@ +README.md \ No newline at end of file diff --git a/README.md b/README.md new file mode 100644 index 000000000..fb1d97226 --- /dev/null +++ b/README.md @@ -0,0 +1,136 @@ +# osm2pgsql # + +osm2pgsql is a tool for loading OpenStreetMap data into a PostgreSQL / PostGIS +database suitable for applications like rendering into a map, geocoding with +Nominatim, or general analysis. + +## Features ## + +* Converts OSM files to a PostgreSQL DB +* Conversion of tags to columns is configurable in the style file +* Able to read .gz, .bz2, .pbf and .o5m files directly +* Can apply diffs to keep the database up to date +* Support the choice of output projection +* Configurable table names +* Gazetteer back-end for [Nominatim](http://wiki.openstreetmap.org/wiki/Nominatim) +* Support for hstore field type to store the complete set of tags in one database + field if desired + +## Installing ## + +The latest source code is available in the OSM git repository on github +and can be downloaded as follows: + +```sh +$ git clone git://github.com/openstreetmap/osm2pgsql.git +``` + +## Building ## + +Osm2pgsql uses the [GNU Build System](http://www.gnu.org/software/automake/manual/html_node/GNU-Build-System.html) +to configure and build itself and requires + +* [libxml2](http://xmlsoft.org/) +* [geos](http://geos.osgeo.org/) +* [proj](http://proj.osgeo.org/) +* [bzip2](http://www.bzip.org/) +* [zlib](http://www.zlib.net/) +* [Protocol Buffers](https://developers.google.com/protocol-buffers/) +* [PostgreSQL](http://www.postgresql.org/) client libraries +* [Lua](http://www.lua.org/) (Optional, used for [Lua tag transforms](docs/lua.md)) + +It also requires access to a database server running +[PostgreSQL](http://www.postgresql.org/) and [PostGIS](http://www.postgis.net/). + +Make sure you have installed the development packages for the +libraries mentioned in the requirements section and a C and C++ +compiler. + +To install on a Debian or Ubuntu system, first install the prerequisites: + +```sh +sudo apt-get install autoconf automake libtool make g++ libxml2-dev libgeos-dev + libgeos++-dev libpq-dev libbz2-dev libproj-dev protobuf-c-compiler + libprotobuf-c0-dev libbz2-dev lua5.2 liblua5.2-dev +``` + +To install on a Fedora system, use + +```sh +sudo yum install gcc-c++ libxml2-devel geos-develpostgresql-devel bzip2-devel + proj-devel protobuf-compiler +``` + +Then you should be able to bootstrap the build system: + + ./autogen.sh + +And then run the standard GNU build install: + + ./configure && make && make install + +Please see `./configure --help` for more options on how to control the build +process. + +## Usage ## + +Osm2pgsql has one program, the executable itself, which has **43** command line +options. + +Before loading into a database, the database must be created and the PostGIS +and optionally hstore extensions must be loaded. A full guide to PostgreSQL +setup is beyond the scope of this readme, but with reasonably recent versions +of PostgreSQL and PostGIS this can be done with + +```sh +createdb gis +psql -d gis -c 'CREATE EXTENSION postgis; CREATE EXTENSION hstore;' +``` + +A basic invocation to load the data into the database ``gis`` for rendering would be + +```sh +osm2pgsql --create --database gis data.osm.pbf +``` + +This will load the data from ``data.osm.pbf`` into the ``planet_osm_point``, +``planet_osm_line``, ``planet_osm_roads``, and ``planet_osm_polygon`` tables. + +When importing a large amount of data such as the complete planet, a typical +command line would be + +```sh + osm2pgsql -c -d gis --slim -C \ + --flat-nodes planet-latest.osm.pbf +``` +where +* ```` is 24000 on machines with 32GiB or more RAM + or about 75% of memory in MiB on machines with less +* ```` is a location where a 24GiB file can be saved. + +The databases from either of these commands can be used immediately by +[Mapnik](http://mapnik.org/) for rendering maps with standard tools like +[renderd/mod_tile](https://github.com/openstreetmap/mod_tile), +[TileMill](https://www.mapbox.com/tilemill/), [Nik4](https://github.com/Zverik/Nik4), +among others. It can also be used for [spatial analysis](docs/analysis.md) or +[shapefile exports](docs/export.md). + +[Additional documentation is available on writing command lines](docs/usage.md). + +## Alternate backends ## + +In addition to the standard [pgsql](docs/pgsql.md) backend designed for +rendering there is also the [gazetteer](docs/gazetteer.md) database for +geocoding, principally with [Nominatim](http://www.nominatim.org/), and the +null backend for testing. + +Any questions should be directed at the osm dev list +http://wiki.openstreetmap.org/index.php/Mailing_lists + +## Contributing ## + +We welcome contributions to osm2pgsql. If you would like to report an issue, +please use the [issue tracker on GitHub](https://github.com/openstreetmap/osm2pgsql/issues). + +General queries can be sent to the tile-serving@ or dev@ +[mailing lists](http://wiki.openstreetmap.org/wiki/Mailing_lists). diff --git a/docs/analysis.md b/docs/analysis.md new file mode 100644 index 000000000..aca10590a --- /dev/null +++ b/docs/analysis.md @@ -0,0 +1,81 @@ +# Geospatial analysis with osm2pgsql # + +An osm2pgsql database and PostGIS is well-suited for geospatial analysis using +OpenStreetMap data where topology is not a consideration. + +PostGIS provides an [extensive number of geometry functions](http://postgis.net/docs/manual-2.1/reference.html) +and a full description of how to perform analysis with them is beyond the +scope of a readme, but a simple example of finding the total road lengths by +classification for a municipality should help. + +To start with, we'll download the data for the region as an [extract from Geofabrik](http://download.geofabrik.de/) and import it with osm2pgsql. + + osm2pgsql --database gis --number-processes 4 --multi-geometry british-columbia-latest.osm.pbf + +``--multi-geometry`` (``-G``) is necessary for most analysis as it prevents +MULTIPOLYGONs from being split into multiple POLYGONs, a step that is +normally used to [increase rendering speed](http://paulnorman.ca/blog/2014/03/osm2pgsql-multipolygons) +but increases the complexity of analysis SQL. + +Loading should take about 10 minutes, depending on computer speed. Once this +is done we'll open a PostgreSQL terminal with ``psql -d gis``, although a GUI +like pgadmin or any standard tool could be used instead. + +To start, we'll create a partial index to speed up highway queries. + +```sql +CREATE INDEX planet_osm_line_highways_index ON planet_osm_line USING GiST (way) WHERE (highway IS NOT NULL); +``` + +We'll first find the ID of the polygon we want + +```sql +gis=# SELECT osm_id FROM planet_osm_polygon +WHERE boundary='administrative' AND admin_level='8' AND name='New Westminster'; + osm_id +---------- + -1377803 +``` + +The negative sign tells us that the geometry is from a relation, and checking +on [the OpenStreetMap site](https://www.openstreetmap.org/relation/1377803) +confirms which it is. + +We want to find all the roads in the city and get the length of the portion in +the city, sorted by road classification. Roads are in the ``planet_osm_line`` +table, not the ``planet_osm_roads`` table which is only has a subset of data +for low-zoom rendering. + +```sql +gis=# SELECT + round(SUM( + ST_Length(ST_Transform( + ST_Intersection(way, (SELECT way FROM planet_osm_polygon WHERE osm_id=-1377803)) + ,4326)::geography) + )) AS "distance (meters)", highway AS "highway type" + FROM planet_osm_line + WHERE highway IS NOT NULL + AND ST_Intersects(way, (SELECT way FROM planet_osm_polygon WHERE osm_id=-1377803)) + GROUP BY highway + ORDER BY "distance (meters)" DESC + LIMIT 10; + distance (meters) | highway type +-------------------+--------------- + 138122 | residential + 79519 | service + 51890 | footway + 25610 | tertiary + 23434 | secondary + 14900 | cycleway + 6468 | primary + 5217 | motorway + 4389 | motorway_link + 3728 | track +``` + +The ``ST_Transform(...,4326)::geography`` is necessary because the data was +imported in Mercator. This step could have been avoided by importing in a local +projection like a suitable UTM projection. + +More complicated analysises can be completed, but this simple example shows how +to use the tables and put conditions on the columns. \ No newline at end of file diff --git a/docs/export.md b/docs/export.md new file mode 100644 index 000000000..ae38dedbd --- /dev/null +++ b/docs/export.md @@ -0,0 +1,12 @@ +# Exporting with osm2pgsql # + +Osm2pgsql can be used in combination with [ogr2ogr](http://www.gdal.org/ogr2ogr.html) and a [PostgreSQL data source](http://www.gdal.org/drv_pg.html). + +An example command to export to GeoJSON would be + + ogr2ogr -f "GeoJSON" roads.geojson -t_srs EPSG:4326 \ + PG:"dbname=gis" -s_srs EPSG:900913 \ + -sql "SELECT name,highway,oneway,toll,way FROM planet_osm_line WHERE highway IS NOT NULL" + +Care should be taken if exporting to shapefiles, as characters may be present +which cannot be represented in ISO-8859-1, the standard encoding for shapefiles. diff --git a/docs/gazetteer.md b/docs/gazetteer.md new file mode 100644 index 000000000..b64844204 --- /dev/null +++ b/docs/gazetteer.md @@ -0,0 +1,7 @@ +# Gazetteer Backend # + +The gazetteer backend is designed for use with +[Nominatim](http://wiki.openstreetmap.org/wiki/Nominatim) +and will not generally be used outside that context. + +The tables are designed for a hiarchy of places. diff --git a/README_lua.md b/docs/lua.md similarity index 100% rename from README_lua.md rename to docs/lua.md diff --git a/docs/pgsql.md b/docs/pgsql.md new file mode 100644 index 000000000..2ba1a5b5e --- /dev/null +++ b/docs/pgsql.md @@ -0,0 +1,54 @@ +# Pgsql Backend # + +The pgsql backend is designed for rendering OpenStreetMap data, principally +with Mapnik, but is also useful for [analysis](docs/analysis.md) and +[exporting](docs/exporting.md) to other formats. + +## Database Layout ## +It connects to a PostgreSQL database and stores the data in four tables + +* ``planet_osm_point`` +* ``planet_osm_line`` +* ``planet_osm_roads`` +* ``planet_osm_polygon`` + +planet_osm_roads contains the data from other tables, but has tags selected +for low-zoom rendering. It does not only contain roads. + +The default prefix ``planet_osm`` can be changed with the ``--prefix`` option. + +If you are using ``--slim`` mode, it will create the following additional 3 +tables which are used by the pgsql middle layer, not the backend: + +* ``planet_osm_nodes`` +* ``planet_osm_ways`` +* ``planet_osm_rels`` + +With the ``--flat-nodes`` option, the ``planet_osm_nodes`` information is +instead stored in a binary file. + +## Importing ## + +1. Runs a parser on the input file and processes the nodes, ways and relations. + +2. If a node has a tag declared in the style file then it is added to + ``planet_osm_point``. Regardless of tags, its position is stored by the + middle layer. + +3. If there are tags on a way in the style file as linear but without polygon + tags, they are written into the lines and, depending on tags, roads tables. + + They are also stored by the middle layer. + +4. Ways without tags or with polygon tags are stored as "pending" in the + middle layer. + +5. Relations are parsed. In this stage, "new-style" multipolygon and boundary + relations are turned into polygons. Route relations are turned into + linestrings. + +6. "Pending" ways are processed, and they are either added as just the way, or + if a member of a multipolygon relation, they processed as multipolygons. + +7. Indexes are built. This may take substantial time, particularly for the + middle layer indexes created in non-slim mode. diff --git a/docs/usage.md b/docs/usage.md new file mode 100644 index 000000000..1e978c282 --- /dev/null +++ b/docs/usage.md @@ -0,0 +1,146 @@ +# Command-line usage # + +Osm2pgsql has one program, the executable itself, which has **43** command line +options. A full list of options can be obtained with ``osm2pgsql -h -v``. This +document provides an overview of options, and more importantly, why you might +use them. + +## Overall options + +* ``--append`` or ``--create`` specify if osm2pgsql is conducting a new import + or adding to an existing one. ``--slim`` is required with ``--append``. + +* ``--input-reader`` specifies the parser if the filetype can't be + automatically detected for some reason. The ``primitive`` parser is **not** + suggested. + +* ``--output`` specifies if the output backend is the default + [pgsql](pgsql.md), the [gazetteer](gazetteer.md) output used by Nominatim, or + null, which emits no output. + +## Performance + +Performance is heavily influenced by other options, but there are some options +that only impact performance. + +* ``--cache`` specifies how much memory to allocate for caching information. In + ``--slim`` mode, this is just node positions while in non-slim it has to + store information about ways and relations too. The maximum RAM it is useful + to set this to in slim mode is 8 bytes * number of nodes / efficiency, where + efficiency ranges from 50% on small extracts to 80% for a planet. + +* ``--number-processes`` sets the number of processes to use. This should + typically be set to the number of CPU threads, but gains in speed are minimal + past 8 threads. + +* ``--disable-parallel-indexing`` disables the clustering and indexing of all + tables in parallel. This reduces disk and ram requirements during the import, + but causes the last stages to take significantly longer. + +* ``--cache-strategy`` sets the cache strategy to use. The defaults are fine + here, and optimizied uses less RAM than the other options. + +## Database options ## + +osm2pgsql supports standard options for how to connect to PostgreSQL. If left +unset, it will attempt to connect to the ``gis`` database using a unix socket. +Most usage only requires setting ``--database``. + +``--tablespace`` options allow the location of main and slim tables and indexes +to be set to different tablespaces independently, typically on machines with +multiple drive arrays where one is not large enough for all of the database. + +``--flat-nodes`` specifies that instead of a table in PostgreSQL, a binary +file is used as a database of node locations. This should only be used on full +planet imports or very large extracts (e.g. Europe) but in those situations +offers significant space savings and speed increases, particularly on +mechanical drives. The file takes approximately 8 bytes * maximum node ID, or +about 23 GiB, regardless of the size of the extract. + +``--unlogged`` specifies to use unlogged tables which are dropped from the +database if the database server ever crashes, but are faster to import. + +``--prefix`` specifies the prefix for tables + +## Middle-layer options ## + +* ``--slim`` causes the middle layer to store node and way information in + database rather than in memory. It is required for updates and for large + extracts or the entire planet which will not fit in RAM. + +* ``--drop`` discards the slim tables when they are no longer needed in the + import, significantly reducing disk requirements and saving the time of + building slim table indexes. A ``--slim --drop`` + +## Output columns options ## + +### Column options + +* ``--extra-attributes`` creates psudo-tags with OSM meta-data like user, + last edited, and changeset. These also need to be added to the style file. + +* ``--style`` specifies the location of the style file. This defines what + columns are created, what tags denote areas, and what tags can be ignored. + The [default.style](../default.style) contains more documentation on this + file. + +* ``--tag-transform-script`` sets a [Lua tag transform](lua.md) to use in + place of the built-in C tag transform. + +### Hstore + +Hstore is a [PostgreSQL data type](http://www.postgresql.org/docs/9.3/static/hstore.html) +that allows storing arbitrary key-value pairs. It needs to be installed on +the database with ``CREATE EXTENSION hstore;`` + +osm2pgsql has five hstore options + +* ``--hstore`` or ``-k`` adds any tags not already in a conventional column to + a hstore column. With the standard stylesheet this would result in tags like + highway appearing in a conventional column while tags not in the style like + ``name:en`` or ``lanes:forward`` would appear only in the hstore column. + +* ``--hstore-all`` or ``-j`` adds all tags to a hstore column, even if they're + already stored in a conventional column. With the standard stylesheet this + would result in tags like highway appearing in conventional column and the + hstore column while tags not in the style like ``name:en`` or + ``lanes:forward`` would appear only in the hstore column. + +* ``--hstore-column`` or ``-z``, which adds an additional column for tags + starting with a specified string, e.g. ``--hstore-column 'name:'`` produces + a hstore column that contains all ``name:xx`` tags + +* ``--hstore-match-only`` modifies the above options and prevents objects from + being added if they only have tags in the hstore column and no conventional + tags. + +* ``--hstore-add-index`` adds a GIN index to the hstore columns. This can + speed up arbitrary queries, but for most purposes partial indexes will be + faster. + +Either ``--hstore`` or ``--hstore-all`` when combined with ``--hstore-match-only`` +should give the same rows as no hstore, just with the additional hstore column. + +Hstore is used to give more flexability to use additional tags without +reimporting the database, at the cost of a +[less speed and more space.](http://paulnorman.ca/blog/2014/03/osm2pgsql-and-hstore/) + +## Projection options + +* ``--latlong``, ``--merc``, or ``--proj`` are used to specify the projection + used for importing. The default, ``--merc`` is typically used for rendering, + while ``--latlong`` can offer advantages for analysis. Most stylesheets + assume ``--merc`` has been used. + +## Output data options + +* ``--multi-geometry`` skips an optimization for rendering where PostGIS + MULTIPOLYGONs are split into multiple POLYGONs. ``--multi-geometry`` can be + used to [avoid some labeling issues at the cost of speed](http://paulnorman.ca/blog/2014/03/osm2pgsql-multipolygons/). + It is also typically required for [analysis](analysis.md). + +* ``--keep-coastlines`` disables a hard-coded rule that would otherwise + discard ``natural=coastline`` ways. + +* ``--exclude-invalid-polygon`` prevents osm2pgsql from attempting to form + valid polygons from invalid ones and just rejects the invalid ones.