[WIP] Strawman Initial Addresses Schema

[Jacob Wasserman (jwass)]

This is an initial schema for the addresses theme and is not ready to be merged. I'm hoping this sparks discussion with the schema group and the addresses folks.

Background

We recently decided to use the Open Addresses (OA) address output schema as a starting point for Overture addresses (https://github.com/openaddresses/openaddresses/blob/master/schema/layers/address_conform.json).

This Pull Request

This initial PR copies OA's by including:

• region
• district
• city
• postcode
• street
• number
• unit

We can convert OA outputs into this schema pretty straightforward. The schema also includes country and country_district which are pulled from the OA path - e.g. us/ma/... This would allow "us" as the country" and "ma" as the country_district.

Todo / Questions to Answer

• Which fields if any should be required?
• Should we always add the country from the OA top-level folder into each entry?
• What to do when an address set doesn't contain the region or city but it's probably known. For example: us/ct/city_of_plainfield-addresses-city.geojson has entries like:

{"type":"Feature","properties":{"hash":"fd84e78f7c30c1ec","number":"14","street":"PINECREST DR","unit":"","city":"","district":"","region":"","postcode":"","id":""},"geometry":{"type":"Point","coordinates":[-71.9059254,41.7584033]}}
{"type":"Feature","properties":{"hash":"3f059ec63fee9030","number":"558","street":"PUTNAM RD","unit":"","city":"","district":"","region":"","postcode":"","id":""},"geometry":{"type":"Point","coordinates":[-71.900128,41.7582586]}}

Note that city and region are blank. Should we replace all of these with "Plainfield" and "CT" here and in similar instances? Another option is to replace these in OA directly with less modification here. (We should follow up with OA).

• How to marry this with the existing address field in places
• How should we reference buildings/places to these addresses? Likely an address_ids field or similar.
• More I haven't thought of at the moment
• Would we want to link the country / region / district / city to their divisions? e.g. with country_division_id, etc?
• Would we want to link the address's street to the corresponding transportation segment(s)?

References

Other address schemas for reference:

• libpostal https://github.com/openvenues/libpostal
• opencage formatting https://github.com/OpenCageData/address-formatting
• How Pelias imports OpenAddresses https://github.com/pelias/openaddresses

[David Karlaš (DavidKarlas)]

Just came here to comment on "Would we want to link the country / region / district / city to their divisions? e.g. with country_division_id, etc?" in my opinion answer is yes, my main question is on how to do that, one possibility is to use hierarchies with same definition as in divisions theme to form some kind of standard across themes and it looks like this:

[
  [
    {
      "division_id": "085395e73fffffff01d732732c18bb4e",
      "subtype": "country",
      "name": "New Zealand"
    },
    {
      "division_id": "0857c822bfffffff01d1d4553101e1c8",
      "subtype": "region",
      "name": "Chatham Islands"
    },
    {
      "division_id": "0857c75cffffffff017471111ebf8169",
      "subtype": "county",
      "name": "Chatham Islands Territory"
    },
    {
      "division_id": "0855485affffffff01f40c7dd929bdb3",
      "subtype": "locality",
      "name": "Kāingaroa"
    }
  ]
]

Victor Schappert (@vcschapp) do you think hierarchies would be good match?

Of course this will be additional work on addresses to find matches with correct city based on raw input data, but I also think it will drive quality and value of data set up significantly.

[Jacob Wasserman (jwass)]

David Karlaš (@DavidKarlas)

Just came here to comment on "Would we want to link the country / region / district / city to their divisions? e.g. with country_division_id, etc?" in my opinion answer is yes, my main question is on how to do that, one possibility is to use hierarchies

If the divisions already contain the hierarchies, would it make sense to link to the most granular possible division? Like the locality then the hierarchy (region, country, etc) would just be derived from that? I suppose we could just copy the division's hierarchy which is what I think you're suggesting.

Another thing I thought about last night was we could also link the street to the transportation segment. Just added another bullet about it.

[Jacob Wasserman (jwass)]

Of course this will be additional work on addresses to find matches with correct city based on raw input data, but I also think it will drive quality and value of data set up significantly.

David Karlaš (@DavidKarlas) I've been doing some more digging here. It looks like Pelias actually ignores admin hierarchy data from OA and builds it itself. See https://github.com/pelias/wof-admin-lookup?tab=readme-ov-file: "So, for Pelias we actually ignore all admin hierarchy information from individual records, and generate it ourselves from the polygon data in Who's on First." I bet we could do something similar using divisions data - as you've laid out - and have a nice coherent dataset here. There's an extra step of figuring out when some addresses might go to a different admin area than the polygon that contains them too.

[Victor Schappert (vcschapp)]

Nice to see progress on addresses.

• This PR reflects a change toward a more "structured" address as informed by OA, versus the previous design idea Paul Heersink (@pheersink) socialized that had more of a free-form text field. Do we want to increase structure like this? I thought the idea behind free-form was that it would be more flexible in the face of non-Western/less structured addressing schemes.
• Consider a pattern: ^(\S.*)?\S$ validation on properties to deny leading and trailing whitespace?
• Suggested some property name tweaks to align more closely with divisions.

[Victor Schappert (vcschapp)]

heh! Address geometry?

[Victor Schappert (vcschapp)]

Suggest locality to align with subtype=locality in the divisions schema.

[Victor Schappert (vcschapp)]

This one's a bit wacky

• I don't fully understand what it means.
• Is it a subdirectory within the OpenAddresses distribution? Do we want to marry the schema to OA to that extent?

[Jacob Wasserman (jwass)]

Removed in latest PR

[Victor Schappert (vcschapp)]

Does "in the OpenAddresses distribution" add value? Can we just say it's the ISO-3166-1 alpha-2 country code?

[Victor Schappert (vcschapp)]

Suggest county to align with subtype=county within the divisions schema.

[Victor Schappert (vcschapp)]

Gar - missing NL at EOF!

[Victor Schappert (vcschapp)]

Just came here to comment on "Would we want to link the country / region / district / city to their divisions? e.g. with country_division_id, etc?" in my opinion answer is yes, my main question is on how to do that, one possibility is to use hierarchies with same definition as in divisions theme to form some kind of standard across themes and it looks like this:

[
  [
    {
      "division_id": "085395e73fffffff01d732732c18bb4e",
      "subtype": "country",
      "name": "New Zealand"
    },
    {
      "division_id": "0857c822bfffffff01d1d4553101e1c8",
      "subtype": "region",
      "name": "Chatham Islands"
    },
    {
      "division_id": "0857c75cffffffff017471111ebf8169",
      "subtype": "county",
      "name": "Chatham Islands Territory"
    },
    {
      "division_id": "0855485affffffff01f40c7dd929bdb3",
      "subtype": "locality",
      "name": "Kāingaroa"
    }
  ]
]

Victor Schappert (@vcschapp) do you think hierarchies would be good match?

Of course this will be additional work on addresses to find matches with correct city based on raw input data, but I also think it will drive quality and value of data set up significantly.

Going back to Relationships in the Schema, one thing we said is that:

Many relationships are implied by geometry ("contained by the polygon for the US"), or transitively through some standard value like an ISO country code ("US") or a highway network ("I-55"). Direct foreign key references should be used frugally, in cases where unambiguously deducing the relationship for a high priority use case is impossible or unnecessarily burdensome.

IMO the best thing to do is use country and region in exactly the same way in both addresses and divisions so you can join them at that level.

I'm reluctant to add any direct FK relationships between them because there are lots of subtleties around addresses that aren't easy to resolve, and that we probably shouldn't resolve.

e.g. City name of an address might be one thing from a postal delivery perspective and another thing from an actual municipal existence perspective.

I imagine Drew Breunig (@PreciselyDrew) could list 10 complexities.

I think the combination of the address feature's coordinates and the country/region aspect will already prove very useful and we might want to draw the line at adding more complexity above that.