CLDR-16970 Add testData file and generator for likely subtags #3173
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…de locales > Basic to cut down the size
sffc
approved these changes
Aug 10, 2023
Comment on lines
+14
to
+17
| # RemoveFavorScript: Remove Likely Subtags, when the script is favored. | ||
| # Only included when different than AddLikely. | ||
| # RemoveFavorRegion: Remove Likely Subtags, when the region is favored. | ||
| # Only included when different than RemoveFavorScript. |
There was a problem hiding this comment.
Observation: I'm not sure we implement RemoveFavorScript in ICU4X. Good to have this test data to find a deficiency in our feature coverage that we can work toward fixing. :)
7 tasks
macchiati
deleted the
CLDR-16970-Add-test-file-and-generator-for-LikelySubtags
branch
|
It looks like it only makes a difference in a few cases (this is restricted to Basic locales):
|
|
I believe that's correct; these are the only instances where there is a
lang-region and a lang-script on the left side of the likely subtags mapping
<https://github.com/unicode-org/cldr-json/blob/main/cldr-json/cldr-core/supplemental/likelySubtags.json>
:
"sd-Deva": "sd-Deva-IN",
"sd-IN": "sd-Deva-IN",
and
"yue-CN": "yue-Hans-CN",
"yue-Hans": "yue-Hans-CN",
and
"zh-Hant": "zh-Hant-TW",
"zh-TW": "zh-Hant-TW",
I don't see a default region for "sr-Latn", for example, which means
"sr-Latn" gets expanded to "sr-Latn-RS", which cannot be minimized to
"sr-RS" because that would be changing the script.
…On Thu, Aug 10, 2023 at 5:29 PM Mark Davis ***@***.***> wrote:
It looks like it only makes a difference in a few cases (this is
restricted to Basic locales):
<style type="text/css"></style>
Source ; | AddLikely ; | RemoveFavorScript ; | RemoveFavorRegion
-- | -- | -- | --
sd-Deva ; | sd-Deva-IN ; | sd-Deva ; | sd-IN
sd-IN ; | sd-Deva-IN ; | sd-Deva ; | sd-IN
yue-CN ; | yue-Hans-CN ; | yue-Hans ; | yue-CN
yue-Hans ; | yue-Hans-CN ; | yue-Hans ; | yue-CN
und-Hant ; | zh-Hant-TW ; | zh-Hant ; | zh-TW
und-Hant-TW ; | zh-Hant-TW ; | zh-Hant ; | zh-TW
und-TW ; | zh-Hant-TW ; | zh-Hant ; | zh-TW
zh-Hant ; | zh-Hant-TW ; | zh-Hant ; | zh-TW
zh-TW ; | zh-Hant-TW ; | zh-Hant ; | zh-TW
—
Reply to this email directly, view it on GitHub
<#3173 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAIXXIUTRX2NAXWD5NDCDNTXUV4GXANCNFSM6AAAAAA3MEDNDA>
.
You are receiving this because your review was requested.Message ID:
***@***.***>
|
FrankYFTang
reviewed
Aug 11, 2023
| und-AX ; sv-Latn-AX ; sv-AX ; | ||
| und-AZ ; az-Latn-AZ ; az ; | ||
| und-Adlm ; ff-Adlm-GN ; ff-Adlm ; | ||
| und-Adlm-BF ; fr-Adlm-BF ; ; |
There was a problem hiding this comment.
Could you please explain what does the RemoveFavorRegion = "" mean?
There was a problem hiding this comment.
It's in the header. In other words, if a value is missing, it means the same as the previous field in the row (recursively).
# RemoveFavorScript: Remove Likely Subtags, when the script is favored.
# Only included when different than AddLikely.
# RemoveFavorRegion: Remove Likely Subtags, when the region is favored.
# Only included when different than RemoveFavorScript.
If that isn't clear we can improve the header.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
CLDR-16970
It became clear in today's ICU meeting that the lack of a test file has led to incompatibilities in implementations. I made a quick generator for the test file. Please check the results to see if there are any unexpected values.
ALLOW_MANY_COMMITS=true