Use versioned CLDR data #825
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Oct 29, 2020
noviluni
added
the
hacktoberfest-accepted
Codecov Report
@@ Coverage Diff @@
## master #825 +/- ##
=======================================
Coverage 98.30% 98.30%
=======================================
Files 231 231
Lines 2590 2590
=======================================
Hits 2546 2546
Misses 44 44
Continue to review full report at Codecov.
|
noviluni
force-pushed
the
fix_get_cldr_data
branch
from
22de797
to
b176f48
Compare
|
BTW, I removed the |
|
related to this: #487 |
|
I created this ticket with the step by step guide to update the CLDR in the future once we merge this: #826 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We have been using an old version of CLDR data. It has been tried to update the data from CLDR in the past but it's something really hard to do.
I changed the
dateparser_scripts.utils.get_raw_datafunction to point to a specific version and updated the files accordingly.That version was released in April 2017 and it is the most similar I found to the current data (I would say "exactly the same version"). Most of the changes you will see are related to the order, as it's important to keep all ordered to update the data easily in the future.
Once I fixed that I downloaded that version data (JSON files, script:
python dateparser_scripts/get_cldr_data.py), I updated the resources (py files, script:dateparser_scripts/write_complete_data.py) and I run the tests and all is working as expected, so I think this is a good starting point. Once we merge this we can gradually update the CLDR versions to get the most recent version.(you can review this easier by going commit by commit)
closes: #824