forked from osm2pgsql-dev/osm2pgsql
-
Notifications
You must be signed in to change notification settings - Fork 0
/
README
301 lines (230 loc) · 10.6 KB
/
README
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
osm2pgsql
=========
Converts OSM planet.osm data to a PostgreSQL / PostGIS database suitable
for specific applications like rendering into map tiles by Mapnik
or geocoding with Nominatim.
osm2pgsql currently supports two different database schemas
1) A database schema that is optimized for ease of rendering
by Mapnik.
2) A database schema that is optimized for geocoding with Nominatim,
emphasizing the spatially hierarchical organizations of objects.
Both schemas were specifically optimized for the purpose they were
intended for and they may therefore be less suitable for other
general purpose processing. Nevertheless, the rendering schema
might be useful for other purposes as well, and has been used
for a variety of additionally purposes.
For a broader view of the whole map rendering tool chain see
http://wiki.openstreetmap.org/index.php/Mapnik
http://wiki.openstreetmap.org/index.php/Osm2pgsql
http://wiki.openstreetmap.org/index.php/Slippy_Map
You may find that the wiki pages are more up to date than this
readme and may include answers to issues not mentioned here.
Any questions should be directed at the osm dev list
http://wiki.openstreetmap.org/index.php/Mailing_lists
Features
========
- Converts OSM files to a PostgreSQL DB
- Conversion of tags to columns is configurable in the style file
- Able to read .gz, .bz2, .pbf and .o5m files directly
- Can apply diffs to keep the database up to data
- Support the choice of output projection
- Configurable table names
- Gazetteer back-end for Nominatim
http://wiki.openstreetmap.org/wiki/Nominatim
- Support for hstore field type to store the complete set of tags in one database
field if desired
Source code
===========
The latest source code is available in the OSM git repository on github
and can be downloaded as follows:
$ git clone git://github.com/openstreetmap/osm2pgsql.git
Build requirements
==================
The code is written in C and C++ and relies on the libraries
below:
- libxml2 http://xmlsoft.org/
- geos http://geos.refractions.net/
- proj http://www.remotesensing.org/proj/
- bzip2 http://www.bzip.org/
- zlib http://www.zlib.net/
- PostgreSQL http://www.postgresql.org/
- PostGIS http://postgis.refractions.net/
To make use of the database generated by this tool you will
probably also want to install:
- Mapnik from http://mapnik.org/
Building
========
Make sure you have installed the development packages for the
libraries mentioned in the requirements section and a C and C++
compiler.
e.g. on Fedora:
# yum install geos-devel proj-devel postgresql-devel libxml2-devel bzip2-devel gcc-c++
on Debian:
# aptitude install libxml2-dev libgeos-dev libgeos++-dev libpq-dev libbz2-dev libproj-dev protobuf-c-compiler libprotobuf-c0-dev autoconf automake libtool make g++
On most Unix-like systems the program can be compiled by
running './autogen.sh && ./configure && make'.
Operation
=========
You must create a PostgreSQL user and a database with the
PostGIS functions enabled. This requires access as the
database administrator, normally the 'postgres' user.
The default name for this database is 'gis' but this may
be changed by using the osm2pgsql --database option.
If the <username> matches the unix user id running the import
and rendering then this allows the PostgreSQL 'ident sameuser'
authentication to be used which avoids the need to enter a
password when accessing the database. This is setup by default
on many Unix installs but does not work on Windows (due to the
lack of unix sockets).
Some example commands are given below but you may find
this wiki page has more up to data information:
http://wiki.openstreetmap.org/wiki/Mapnik/PostGIS
$ sudo -u postgres createuser <username>
$ sudo -u postgres createdb -E UTF8 -O <username> <dbname>
$ sudo -u postgres createlang plpgsql <dbname>
Adding the PostGIS extensions. Note the location of the
files may vary.
$ sudo -u postgres psql <dbname> < /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql
$ sudo -u postgres psql <dbname> < /usr/share/postgresql/8.4/contrib/postgis-1.5/spatial_ref_sys.sql
Next we need to give the <username> access to update the postgis
meta-data tables
$ sudo -u postgres psql -d <dbname> -c "ALTER TABLE geometry_columns OWNER TO <username>"
$ sudo -u postgres psql -d <dbname> -c "ALTER TABLE spatial_ref_sys OWNER TO <username>"
The 900913 is not normally included with PostGIS. To add it you
should run:
$ sudo psql -u postgres psql -d <dbname> -f 900913.sql
If you want to use hstore support then you will also need to enable the PostgreSQL
hstore-new extension.
$ sudo -u postgres psql <dbname> < /usr/share/postgresql/8.4/contrib/hstore.sql
On PostgreSQL 9.1 and above, you can install it by running "CREATE EXTENSION hstore;"
in your database.
Now you can run osm2pgsql to import the OSM data.
This will perform the following actions:
1) Osm2pgsql connects to database and creates the following 4 tables
when used with the default output back-end (pgsql):
- planet_osm_point
- planet_osm_line
- planet_osm_roads
- planet_osm_polygon
The prefix "planet_osm" can be changed with the --prefix option,
the above is the default.
If you are using --slim mode, it will create the following additional 3 tables:
- planet_osm_nodes
- planet_osm_ways
- planet_osm_rels
2) Runs an XML parser on the input file (typically planet.osm)
and processes the nodes, ways and relations.
3) If a node has a tag declared in the style file then it is
added to planet_osm_point. If it has no such tag then
the position is noted, but not added to the database.
4) Ways are read in converted into WKT geometries by using the
positions of the nodes read in earlier. If the tags on the way
are listed in the style file then the way will be written into
the line or roads tables.
5) If the way has one or more tags marked as 'polygon' and
forms a closed ring then it will be added to the planet_osm_polygon
table.
6) The relations are parsed. Osm2pgsql has special handling for a
limited number of types: multipolygon, route, boundary
The code will build the appropriate geometries by referencing the
members and outputting these into the database.
7) Indexes are added to speed up the queries by Mapnik.
Tuning PostgreSQL
=================
For an efficient operation of PostgreSQL you will need to tune the config
parameters of PostgreSQL from its default values. These are set in the
config file at /etc/postgresql/8.4/main/postgresql.conf
The values you need to set will depend on the hardware you have available,
but you will likely need to increase the values for the following parameters:
- shared_buffers
- checkpoint_segments
- work_mem
- maintenance_work_mem
- effective_cache_size
A quick note on projections
===========================
Depending on the command-line switches you can select which projection you
want the database in. You have three choices:
4326: The standard lat/long coordinates
900913: The spherical Mercator projection, used by TileCache, Google Earth etc.
3395: The legacy (broken) WGS84 Mercator projection
Depending on what you're using one or the other is appropriate. The default
Mapnik style (osm.xml) assumes that the data is stored in 900913 and this
is the default for osm2pgsql.
Combining the -v and -h switches will tell about the exact definitions of
the projections.
In case you want to use some completely different projection there is the -E
option. It will initialize the projection as +init=epsg:<num>. This allows
you to use any projection recognized by proj4, which is useful if you want
to make a map in a different projection. These projections are usually
defined in /usr/share/proj/epsg.
Database Access Examples
========================
If you wish to access the data from the database then the
queries below should give you some hints. Note that these
examples all use the 'latlong' projection which is not the
default.
$ psql gis
gis=> \d
List of relations
Schema | Name | Type | Owner
--------+--------------------+-------+----------
...
public | planet_osm_line | table | jburgess
public | planet_osm_point | table | jburgess
public | planet_osm_polygon | table | jburgess
public | planet_osm_roads | table | jburgess
...
gis=> \d planet_osm_line
Table "public.planet_osm_line"
Column | Type | Modifiers
-----------+----------+-----------
osm_id | integer |
name | text |
place | text |
landuse | text |
... [ lots of stuff deleted ] ...
way | geometry | not null
z_order | integer | default 0
Each of the tables contains a subset of the planet.osm file representing
a particular geometry type
- Point contains nodes which have interesting tags
e.g. place=city, name=London
- Line contains ways with interesting tags
e.g. highway=motorway, ref=M25
- Polygon contains ways which form an enclosed area
e.g. landuse=reservoir
The DB columns are used as follows:
- osm_id = the planet.osm ID of the node(point) or way(line,polygon)
- name, place, landuse, ... = the value of the given key, if present on
the node/way. If the tag is not present, the value is NULL. Only a
subset of all possible tags are stored in the DB. Only ones rendered in
the osm.xml are actually interesting to mapnik.
- way = PostGIS geometry describing the physical layout of the object.
Querying specific data requires knowlege of SQL and the OSM key/value
system, e.g.
gis=> select osm_id,astext(way),name from planet_osm_point where amenity='cinema' limit 5;
osm_id | astext | name
----------+-------------------------------------------+--------------------
26236284 | POINT(-79.7160836579093 43.6802306464618) |
26206699 | POINT(51.4051989797638 35.7066045032235) | Cinema Felestin
26206700 | POINT(51.3994885141459 35.7058460359352) | Cinema Asr-e Jadid
20979630 | POINT(151.225781789807 -33.8943079539886) | Paris Cinema
20979684 | POINT(151.226855394904 -33.8946830511095) | Hoyts
(5 rows)
Mapnik renders the data in each table by applying the rules in the
osm.xml file.
> How could I get e.g. all highways in a given bounding box?
The 'way' column contains the geo info and is the one which you need to
use in your WHERE clause. e.g.
gis=> select osm_id,highway,name from planet_osm_line where highway is not null and way && GeomFromText('POLYGON((0 52, 0.1 52, 0.1 52.1, 0 52.1, 0 52))',4326);
osm_id | highway | name
---------+--------------+------------------
4273848 | unclassified |
3977133 | trunk | to Royston (tbc)
4004841 | trunk |
4019198 | trunk |
4019199 | trunk |
4238966 | unclassified |
See the Postgis docs for details, e.g.
http://postgis.refractions.net/docs/ch04.html