DISCUSSION: What open datasets do you want in Pelias?

[riordan]

Pelias can only be as good as the data available to it. And so far, we've done a lot to make the best use of the best collaborative open data projects we can find. But there are plenty of other sources of data that have the potential to open geocoding up to parts of the world where we don't yet offer it.

We suspect you probably know some of the best sources already that would make Pelias/Mapzen Search substantially better. Please, leave us comment below with a particular target dataset that you'd like to see Pelias/Mapzen Search rely upon.

We'd love to see new sources of:

• Address data (so we can provide it to OpenAddresses)
• Road network data for interpolation
• Postcodes
• Venues and named points of interest

This is why we think open will win: when people can share what we know, knowledge grows faster and we can build so much more.

[riordan]

For reference here's the datasets we currently use:

• OpenStreetMap - Streets, venues, addresses, areas of interest
• OpenAddresses - addresses
• Quattroshapes - named administrative places (soon to be deprecated)
• Who's on First - named administrative places, neighborhoods, airports (polygons)
• Geonames - named places (points)

[riordan]

Also here are a few open sources we're tracking now. No promises if/when they'll all make it into the service (and if it makes sense to route them through other, broader projects, like OpenAddresses or Who's on First), but we want to be transparent in what we've been thinking about.

• US ZIP Codes | TIGER/Lines | Polygons
• UK Postal Codes | Ordinance Survey Code Point Open | Points | Scotland, Wales, England only (thx @mtmail)
• SimpleGeo Venues | Who's on First Venues | Points
• US Address Interpolation | TIGER/Lines | Road interpolation lines
• Canada Address Interpolation | Statistics Canada Road Network Data | Road interpolation lines

[mtmail]

It's GB postal codes. The Northern Ireland postcodes are managed by https://www.dfpni.gov.uk/publications/lps-copyright-licensing-digital-application-forms (kind of a separate Ordinance Survey). Those are reselling Royalmail data and require a commercial license. I know, pretty annoying, especially since you can download the NI postcodes just fine.

For Canadian postcodes watch http://geocoder.ca/?sued=1#updates (and/or add to their legal defense fund)

[riordan]

Totally, thank you for catching the NI issue, @mtmail.

And we'll keep our eyes on geocoder.ca.

[orangejulius]

I would like to propose that we build an importer for All The Places. cc @iandees

[hughcameron]

Hi @orangejulius, I noticed your Docker project for Australia. The Australian Government also makes a Geocoded National Address File each quarter which may be useful to develop an importer for.

[trescube]

Hey @hughcameron, Pelias uses openaddresses which contains that curated Australia-wide source, kept up-to-date by the inimitable @andrewharvey. Thanks for your interest in Pelias, we love new data sources!

[orangejulius]

@hughcameron, yup, what @trescube said. The excellent coverage via G-NAF data is part of the reason I made the Australia Docker project. We plan to use Australia as a mid-sized build for testing pelias going forward because we know there is great address coverage. If you have a chance to try it out let me know!

[hughcameron]

Hey @orangejulius & @trescube - thanks for letting me know. I got the Australia project up and running earlier in the week using forward geocoding for validation and parsing - really powerful stuff. Great work!

[yopaseopor]

I suggest two or three public sources:
Adresses. Directorate General for Cadastre. Spainhttp://www.catastro.minhap.es/webinspire/index.html
Institut Geològic i Cartogràfic de Catalunya. Name of streets and housenumbers http://www.icgc.cat/Descarregues/Llocs/WMS-Noms-i-numeracio-de-carrers
Barcelona city: all the adresses and housenumbers: https://opendata-ajuntament.barcelona.cat/data/ca/dataset/taula-direle

[Joxit]

I think openaddresses will be happy to have this source : https://opendata-ajuntament.barcelona.cat/data/ca/dataset/taula-direle

[trescube]

Feel free to open a PR!

[vincentfenet]

In france, you have the official https://adresse.data.gouv.fr/ too

[Joxit]

adresse.data.gouv.fr is already in OpenAddresses thanks to a member of Etalab 😄

[hennihaus]

It would be nice to have an official importer for gtfs-data.

[gelsas]

I noticed that the Canadian Postcodes are not really good in pelias yet.

This company is making monthly updated canadian zip code data (890.000 entries right now) avaiable for free: https://www.serviceobjects.com/blog/unique-us-canadian-zip-code-files-available-download/

This would make Canadian address data much more accurate.

Is the census data from here used for canada already?
https://www12.statcan.gc.ca/census-recensement/2011/geo/bound-limit/bound-limit-eng.cfm
https://www12.statcan.gc.ca/census-recensement/2011/geo/RNF-FRR/index-eng.cfm

[orangejulius]

Hi @gelsas,
Thanks for the recommendation. Open data for Canadian postcodes is hard to come by. The copyright over the data is held by Canada Post and they have defended that copyright in the past, as our fellow geocodeur Ervin from geocode.xyx knows well.

It appears that the license for that zip file from ServiceObjects is effectively proprietary, and certainly not suitable for inclusion by default in Pelias. It's likely anyone using Pelias for their own needs wouldn't want to use it either.

[gelsas]

Is this included in pelias ?
Australia:
https://data.gov.au/dataset/geocoded-national-address-file-g-naf

(This is really important question for me, since I am considering setting this up: https://github.com/mountain-pass/addressr if the data is not yet in pelias)

[gelsas]

Is this data sets already used in pelias:
iceland:
https://github.com/sveinbjornt/iceaddr

Is NZ data included already?
https://data.linz.govt.nz/

[orangejulius]

Hi @gelsas,

The G-NAF dataset from Australia, New Zealand's countrywide address dataset (and several others specific to particular cities or regions), and the Islandic address registry are all supported out of the box by Pelias through the wonderful OpenAddresses project.

The OpenAddresses folks are pretty diligent about adding new public, appropriately licensed address datasets, so they'll generally have anything like that. You can always check in their sources list and if you do happen to find one that's missing they would definitely want to know :)