Backfill wof:lang_x_* properties

[thisisaaronland]

Map wof:country to ISO2 country defined here:

https://github.com/openvenues/libpostal/blob/master/resources/language/countries/country_language.tsv

Note - we will still need to make ISO-639-2 to ISO-639-3 ourselves.

[nvkelso]

And by region, but using IDs of mixed craziness: https://github.com/openvenues/libpostal/blob/master/resources/language/regional/adm1.tsv.

[thisisaaronland]

Separate issue. Simple things, first.

[thisisaaronland]

Conveniently, GeoPlanet did most of the work for us already:

python ./issue-290-countries.py ./issue-290-countries.json  /usr/local/data/whosonfirst-data/meta/wof-country-latest.csv 
missing wof:lang for 85632231
missing wof:lang for 85632295
missing wof:lang for 85632233
missing wof:lang for 85632161
missing wof:lang for 85632709
missing wof:lang for 85632329
missing wof:lang for 85632379
missing wof:lang for 85632315
missing wof:lang for 85632593
missing wof:lang for 85632607
missing wof:lang for 85633793
missing wof:lang for 85633259
missing wof:lang for 85632495
missing wof:lang for 85632423
missing wof:lang for 85632461
missing wof:lang for 85632469
missing wof:lang for 85632437
missing wof:lang for 85632413
has wof:lang 190 missing wof:lang 18

https://whosonfirst.mapzen.com/spelunker/id/85633143/ <-- wof:lang=["fin"]

[thisisaaronland]

Note to self:

1. Apply updates from country records to regions - sudo make me a tool
2. Take a stab at updating regions per libpostal file above, in a separate issue
3. Apply updates to descendants of regions accordingly - see above inre: "a tool"

[stepps00]

Note: per https://github.com/whosonfirst/whosonfirst-properties/blob/master/properties/wof.md, the wof:lang property is now deprecated. We should use this issue to track updating wof:lang_x_official and wof:lang_x_spoken instead.

[stepps00]

Re-running the issue-290 scripts yields the following:

python .scripts/issue-290-countries.py .scripts/issue-290-countries.json ./meta/wof-country-latest.csv
missing wof:lang for 421166797
Warning 1: OGR_G_Area() called against non-surface geometry type.
missing wof:lang for 421180189
Warning 1: OGR_G_Area() called against non-surface geometry type.
missing wof:lang for 421188081
Warning 1: OGR_G_Area() called against non-surface geometry type.
missing wof:lang for 421185849
Warning 1: OGR_G_Area() called against non-surface geometry type.
missing wof:lang for 421202109
Warning 1: OGR_G_Area() called against non-surface geometry type.
has wof:lang 214 missing wof:lang 5

The five listed records are deprecated, non-current records. All top-level country records have wof:lang properties.

Rerunning on regions...

[stepps00]

Running the issue-290 script on the region metafile yields the following:

python ./scripts/issue-290-countries.py ./scripts/issue-290-countries.json ./meta/wof-region-latest.csv
missing wof:lang for 421172177
...
missing wof:lang for 1108808605
missing wof:lang for 1108810255
has wof:lang 2709 missing wof:lang 2213

and, running the issue-290 script on the macroregion metafile yields the following:

python ./scripts/issue-290-countries.py ./scripts/issue-290-countries.json ./meta/wof-macroregion-latest.csv
missing wof:lang for 404227353
missing wof:lang for 404227355
...
has wof:lang 1 missing wof:lang 108

[stepps00]

Re #290 (comment):

• Apply updates from country records to regions - sudo make me a tool
  • This is taken care of in 148401a and 167f4d4.
• Take a stab at updating regions per libpostal file above, in a separate issue
  • Not done yet. This will still require a separate issue to update records.
• Apply updates to descendants of regions accordingly - see above inre: "a tool"
  • This is also not done, but the issue-270 tools could be used to run over each of the metafiles.