Tool to dump all GPS traces collected by/for the OpenStreetMap project.
Python
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.gitignore
README.md
gpx_dump.py
gpx_tables.sql

README.md

planet-gpx-dump

Tools to dump all GPS traces collected by/for the OpenStreetMap project.

gpx_dump.py

A python tool designed to query the OpenStreetMap API database (the one behind the railsport) and write out a collection of GPX files based on the user-uploaded traces. In it's current form, this script will only write out public, identifiable, and trackable traces. Due to privacy concerns and the technical implications around anonymizing data marked as private, the tool will not export traces marked as private.

This repo also contains gpx_tables.sql, a representative dump of the actual GPX-related table schema running on the OSM database server as of the end of March 2013. It also inserts a few dummy traces to test with at the end.

Running

The python application is fairly self-documenting. Run python gpx_dump.py --help to get a help message like this:

usage: gpx_dump.py [-h] [--host HOST] [--port PORT] [--user USER]
                   [--password PASSWORD] --database DATABASE
                   [--privacy {public,identifiable,trackable}]
                   [--output OUTPUT] [--disable-tags] [--disable-uid-mapping]

Dumps GPX files from the OSM railsport database schema.

optional arguments:
  -h, --help            show this help message and exit
  --host HOST           postgres server host
  --port PORT           postgres server port
  --user USER           postgres user name
  --password PASSWORD   postgres user password
  --database DATABASE   postgres database name
  --privacy {public,identifiable,trackable}
                        select which privacy levels to write out
  --output OUTPUT       output directory to fill with resulting GPX files
  --disable-tags        disable dumping tags to the metadata for each gpx file

There are database connection options and output options. Use --host, --port, --user, --password, and --database to specify how to connect to the PostgreSQL database. The output options specify which privacy levels to output and whether or not to disable tags.

Output Format

The gpx_dump.py tool creates a directory for each of the privacy levels and a file named metadata.xml in the directory specified by the --output option. The GPX files created from the export process will end up in these directories according to their privacy setting. The individual files will be named using their database ID padded by several zeroes. The GPX files will be GPX 1.0 compliant XML documents.

The metadata XML file will contain one <gpxFile> element for each GPX file created. It has several attributes:

Name Description
id The database ID for the gpx file.
timestamp The timestamp for when the trace was created in the database.
points The number of points in the GPX file.
lat The latitude for the GPX file. By default, this is the latitude for the first point in the file but can be changed by the user.
lon The longitude for the GPX file. By default, this is the longtiude for the first point in the file but can be changed by the user.
visibility The visibility of the GPX file as specified by the user. One of private, identifiable, public, trackable.
uid The OSM user ID for the user that uploaded the trace. Note that this will only appear for identifiable and public traces.
user The OSM display name for the user that uploaded the trace. Note that this will only appear for identifiable and public traces and if "user id mapping" was enabled at run time.
filename The file name of the GPX trace (relative to the metadata.xml file).

Within each <gpxFile> element can exist:

  • at most one <description> tag containing the description of the GPX trace as specified by the user.
  • at most one <tags> element containing zero or more <tag> elements containing the tags for the GPX trace as specified by the user.

All times (in the GPX and metadata.xml files) are in UTC.