Skip to content

Pushes a set of OpenStreetMap planet files to an ElasticSearch Geo Tile Cluster

License

Notifications You must be signed in to change notification settings

erikschlegel/planet2elastic

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

25 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Planet 2 Elastic

Join the chat at https://gitter.im/erikschlegel/planet2elastic

Pushes a set of OpenStreetMap planet files to an Tile Based Elastic Index. You can grab planet files at GeoFabrik. This will parse all binary pbf files and index all nodes and ways within their respective tile quadkey location. The end result is an elastic document that looks like the following. Document index convention follows {QuadKey}-{OsmId}

{
        "_index": "places",
        "_type": "place",
        "_id": "1202033221320231-100149001",
        "_score": 1,
        "_source": {
          "osmId": "100149001",
          "type": "way",
          "name": "FC Berwangen",
          "coordinates": {
            "type": "multilinestring",
            "coordinates": [
              [
                [
                  8.98524,
                  49.186888
                ],
                [
                  8.98611,
                  49.186606
                ],
                [
                  8.985465,
                  49.185754
                ],
                [
                  8.984595,
                  49.186035
                ],
                [
                  8.98524,
                  49.186888
                ]
              ]
            ]
          },
          "location": {
            "lat": 49.186434,
            "lon": 8.98533
          },
          "quadKey": "1202033221320231",
          "id": "1202033221320231-100149001"
        }
      }

##Setup Please complete the following steps before you kickoff the elastic indexing. This assumes you have some general familiarity with elasticsearch. Please refer to this Gist if you have an Azure subscription and in need of a cloud-based elastic cluster.

###Environment Settings Setup the following environment variables to specify the destination elastic cluster. This assumes your elastic cluster has basic authentication enabled, via Shield.

  • ES_HOST: my-elasticbeast.azure.cloud.com
  • ES_PORT: 9200
  • ES_AUTH_USER: my-username
  • ES_AUTH_PWD: mypassword

Provide the directory location for your planet(.pbf) files

  • PBF_DIRECTORY: /pbfsource

###Setup Elastic Indexes Create two new indexes. To create a new index in elastic you provide a PUT request using a tool like POSTMON with the request URL following the convention ES_HOST:ES_PORT/ES_INDEX_NAME

##Import ###Configuration The mapping features that this framework imports from Openstreetmap is managed in featureTags.js. Feel free to add/remove the data attributes relevant for your scenario(s).

###Run Importer

npm install
npm run import

About

Pushes a set of OpenStreetMap planet files to an ElasticSearch Geo Tile Cluster

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published