🔃 Extract, Transform & Load to OpenStreetMap
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
etl2osm
examples
models
tests
.coveragerc
.gitignore
.pylintrc
.travis.yml
MANIFEST.in
Makefile
README.md
pytest.ini
requirements-dev.txt
requirements.in
requirements.txt
setup.cfg
setup.py

README.md

ETL2OSM

Extract, Transform and Load to OpenStreetMap

Build Status Coverage Status

Features

  • Extracts & Loads data from multiple formats:

    • Shapefile
    • GeoJSON
    • OSM
    • KML (Coming Soon)
  • Replace direction fields (NE > Northeast, SW > Southwest)

  • Replace street suffix fields (AVE > Avenue, ST > Street)

  • Change & remove attribute names

  • Convert text fields into proper titlecase (ottawa ONTARIO > Ottawa Ontario)

  • Transform data into WGS84 (EPSG:4326)

API Overview

Step by Step doing a typical Extract Transform Load processing.

>>> import etl2osm
>>> data = etl2osm.extract("infile.shp")
>>> data.transform("config.json")
>>> data.save("outfile.osm")

Doing the entire process in a single line

>>> import etl2osm
>>> etl2osm.process("infile.shp", "config.json", "outfile.osm")

Command Line Interface

Reading the a file, the standard output will be in a GeoJSON format.

$ etl2osm "infile.shp" --config "config.json" --outfile "outfile.osm"

See Examples for more information.

Making a Config.json

Whenever you want to perform a transformation, include a config file in a JSON format.

Road Data

{
    "conform": {
        "addr:street": [
            {"function": "direction", "field":"DIRECTION"},
            {"function": "title", "field": "ST_NAME"},
            {"function": "suffix", "field": "ST_EXT"}
        ],
        "maxspeed": {"function": "mph", "field": "Speed_Limi"}
    }
}

Address Data

{
    "conform": {
        "addr:number": {"int": "True", "field": "NUMBER_"},
        "addr:street": [
            {"function": "direction", "field": "PREDIR"},
            {"function": "title", "field": "ST_NAME"},
            {"function": "suffix", "field": "STSUFFIX"}
        ],
        "addr:postcode": {"int": "True", "field": "ZIP_CODES"},
        "addr:unit": "UNIT"
    }
}

Installation

GDAL & Fiona

ETL2OSM depends on GDAL, Fiona & LXML; to install those packages using Windows. Download the appropriate library from Unofficial Windows Binaries for Python Extension Packages matching your Python version (Python27/34) & 64/32bit.

$ cd packages
$ pip install lxml‑3.4.4‑cp27‑none‑win32.whl
$ pip install Fiona-1.6.2-cp27-none-win32.whl
$ pip install GDAL-1.11.3-cp27-none-win32.whl

PyPi Install

Install ETL2OSM from PyPi's online packages.

$ pip install etl2osm

GitHub Install

Install the latest version directly from Github.

$ git clone https://github.com/osmottawa/etl2osm/
$ cd etl2osm
$ pip install .