Add endangerment data from ELCat

[xrotwang]

We may want to add the endangerment data from ELCat, which breaks down as shown below. The question is whether we want to merge this with the UNESCO data, or keep the possibly two assessments side by side? I'd opt for merging, since otherwise we'd be be adding not much of a service. If we merge, the question would become how to re-concile the endangerment levels below with UNESCO's.

>>> for k, v in states.most_common():
...     print k, v
... 
Threatened 900
Endangered 748
Vulnerable 548
Critically endangered 429
Severely endangered 367
Dormant 191
At risk 82
Awakening 66
>>> for k, v in certainty.most_common():
...     print k, v
... 
20 percent certain, based on the evidence available 2129
80 percent certain, based on the evidence available 337
100 percent certain, based on the evidence available 313
60 percent certain, based on the evidence available 237
40 percent certain, based on the evidence available 189
None 126

[xrotwang]

A simple merge algo would be:

1. Extend ELCat's levels by adding Extinct.
2. ELCat info overrides.
3. UNESCO is mapped to the extended levels mapping Definitely endangered to Endangered and the other levels trivially.

[chirila]

ELCat took Unesco data into account when constructing their endangerment index; Unesco endangerment indices are also in part based on Ethnologue, I think (all these sites harvest data from one another in a more or less obvious way). ELCat has better coverage for some areas than unesco though, and is probably more accurate.

…

On Thu, Feb 2, 2017 at 11:17 AM, Robert Forkel ***@***.***> wrote: We may want to add the endangerment data from ELCat, which breaks down as shown below. The question is whether we want to merge this with the UNESCO data, or keep the possibly two assessments side by side? I'd opt for merging, since otherwise we'd be be adding not much of a service. If we merge, the question would become how to re-concile the endangerment levels below with UNESCO's <http://www.unesco.org/new/en/culture/themes/endangered-languages/atlas-of-languages-in-danger/> . >>> for k, v in states.most_common(): ... print k, v ... Threatened 900 Endangered 748 Vulnerable 548 Critically endangered 429 Severely endangered 367 Dormant 191 At risk 82 Awakening 66 >>> for k, v in certainty.most_common(): ... print k, v ... 20 percent certain, based on the evidence available 2129 80 percent certain, based on the evidence available 337 100 percent certain, based on the evidence available 313 60 percent certain, based on the evidence available 237 40 percent certain, based on the evidence available 189 None 126 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#53>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AP8oRxeXoLQdwe3OQCN4iFurDxLhYoHsks5rYgF9gaJpZM4L1T5p> .

-- Claire Bowern Associate Professor Director of Graduate Studies Department of Linguistics 370 Temple St, Rm 313 New Haven, CT 06511 Support the Endangered Language Fund!: http://smile.amazon.com/ch/06-1459207 .

[xrotwang]

@chirila So is this a yes to adding and merging? The value Glottolog would be adding would be

• transparent update procedure via git (and GitHub)
• mapping to Glottolog/Glottocodes
• merged/extended coverage
• better programmatic access to the data

We will still link to both, EndangeredLanguages and UNESCO, so I think this could be considered win-win.

[chirila]

Yes, that sounds reasonable.

…

On Thu, Feb 2, 2017 at 11:29 AM, Robert Forkel ***@***.***> wrote: @chirila <https://github.com/chirila> So is this a yes to adding and merging? The value Glottolog would be adding would be - transparent update procedure via git (and GitHub) - mapping to Glottolog/Glottocodes - merged/extended coverage - better programmatic access to the data We will still link to both, EndangeredLanguages and UNESCO, so I think this could be considered win-win. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#53 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AP8oR-amhQMoW0moIvpLxyU5RzGzVw2sks5rYgRfgaJpZM4L1T5p> .

-- Claire Bowern Associate Professor Director of Graduate Studies Department of Linguistics 370 Temple St, Rm 313 New Haven, CT 06511 Support the Endangered Language Fund!: http://smile.amazon.com/ch/06-1459207 .

[d97hah]

I + some others took a closer look not too long ago and I can add the following observations: 1. Elcat, Unesco and Ethnologue differ a lot in their assessments. Thousands of languages differ if one takes one or the other as authoritative. 2. My impression is that the order of reliability (overall, different for specific regions) is ethnologue, elcat and last unesco 3. Like Claire says the different catalogues draw on each other in adhoc ways, if Glottolog becomes another silo without a principled selection procedure I am not sure if we add anything at all. but if we want to do that, Unesco and Elcat have hundreds of language each where the iso and/or glottocode assignment is wrong or non-trivial and I have corrected mappings 4. The extinct flag of Glottolog should is more correct than the corresponding status for any of the three endangerment catalogues 2017-02-02 21:54 GMT+01:00 Claire Bowern <notifications@github.com>:

…

Yes, that sounds reasonable. On Thu, Feb 2, 2017 at 11:29 AM, Robert Forkel ***@***.***> wrote: > @chirila <https://github.com/chirila> So is this a yes to adding and > merging? The value Glottolog would be adding would be > > - transparent update procedure via git (and GitHub) > - mapping to Glottolog/Glottocodes > - merged/extended coverage > - better programmatic access to the data > > We will still link to both, EndangeredLanguages and UNESCO, so I think > this could be considered win-win. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#53 (comment)>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AP8oR- amhQMoW0moIvpLxyU5RzGzVw2sks5rYgRfgaJpZM4L1T5p> > . > -- Claire Bowern Associate Professor Director of Graduate Studies Department of Linguistics 370 Temple St, Rm 313 New Haven, CT 06511 Support the Endangered Language Fund!: http://smile.amazon.com/ch/06- 1459207 . — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#53 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADUDyDSGZYsuoum2MiNP6JtEjPw4_jyLks5rYkKOgaJpZM4L1T5p> .

[xrotwang]

Merged endangerment data from multiple sources has been added in a [endangerment] section of the languoid info files.