extractUnitPatterns returns data for missing locales

[slavafomin]

Hello!

Thank you for your module, it works great.

However, it looks like extractUnitPatterns function is always returns some data, even if missing locale is specified, e.g.: extractUnitPatterns('djfhjsdhfkjhsdfkjl). Why this is happening and how do I detect if real data is present for the specified locale?

Thank you!

[papandreou]

You're getting the data from the root locale, which all locales inherit from.

require('cldr').localeIds should be a list of locales that have data available. Does that solve your problem? I could also make the extract... functions throw or return nothing (not even the root data) in this case.

[slavafomin]

I think the best course of action would be to throw exception or just return null value. I'm not sure if every locale from localeIds actually has data (for example not all locales has pluralization function).

[papandreou]

I landed a fix for this, but it turned out to be problematic when extracting from the supplemental folder where not every locale has its own file. I might revisit it later.

[rotsee]

+1 on this.

At least it would be good to have a “strict mode”, where the library does not silently fall back to root. Even if this is only partially implemented (not working on supplemental data extraction), it would be an improvement, imho.

A typical use case is when working on a fallback chain of languages, e.g. ["nn", "no", "nb"] (nn has less than 100 % coverage in CLDR).

[papandreou]

Hmm, it would be fairly easy to just throw when attempting to extract data for a locale that does not exist in CLDR at all, but I'm not sure how to implement a strict mode like that. Since nn is in there: https://github.com/papandreou/node-cldr/blob/master/3rdparty/cldr/common/main/nn.xml it's hard to tell if the fallback to root is intentional or not.

For example, en.xml does not contain an entry for the America/Coral_Harbour time zone, so that will fall back to the entry in root.xml. Which is probably perfectly fine -- then the exemplar city will be Atikokan in the Englishes.

That being said I don't have a full overview of everything in the CLDR -- it could be that there's some information in there that would help detect when the fallback isn't intentional. Please prove me wrong :)

[papandreou]

Implemented the easy one here: #73

[rotsee]

For example, en.xml does not contain an entry for the America/Coral_Harbour time zone, so that will fall back to the entry in root.xml. Which is probably perfectly fine -- then the exemplar city will be Atikokan in the Englishes.

I see, reality is more complex than I imagined. As always.
I guess if there was a way to know what fallback level was actually used, the client could use that information for decide whether the fallback was appropriate for their particular situation, but I have no idea how that would work...

[papandreou]

I dug around a bit, but CLDR docs are pretty sparse. The closest thing I could find was that some of the incomplete locales, such as nn, are littered with draft="unconfirmed" attributes:

node-cldr/3rdparty/cldr/common/main/nn.xml

Line 7976 in eeff9f2

<listPatternPart type="start" draft="unconfirmed">{0}, {1}</listPatternPart>

That doesn't really solve the <listPattern type="or"> problem though, as that is completely absent from the nn locale.

I'm out of ideas, so I'll close this for now. The "easy" fix was released in 5.0.0.