Add unicodedata.east_asian_width

[xiaozhiyan]

Closes #4522

This PR is a draft for further discussion.
Update: In this PR, we've adopted ucd crate to implement unicodedata.east_asian_width using database v9.0.0, while leaving unicodedata.ucd_3_2_0.unidata_version referring to unicodedata.east_asian_width as is for now.

Python Document

https://docs.python.org/3/library/unicodedata.html#unicodedata.east_asian_width

Expectation (as in CPython 3.11)

>>> import unicodedata
>>> unicodedata.unidata_version
'14.0.0'
>>> unicodedata.east_asian_width('\u231a')
'W'
>>> unicodedata.ucd_3_2_0.unidata_version
'3.2.0'
>>> unicodedata.ucd_3_2_0.east_asian_width('\u231a')
'N'

Status

• unicodedata.east_asian_width is supported by using ucd crate
• unicodedata.ucd_3_2_0.east_asian_width is not working due to 9.0 change as in test https://github.com/RustPython/RustPython/blob/main/Lib/test/test_unicodedata.py#L264

Reason

• ucd crate uses Unicode Character Database (v9.0.0)
• self.assertEqual(self.db.ucd_3_2_0.east_asian_width('\u231a'), 'N')
• https://github.com/sourtin/libucd/blob/master/src/tables/misc.rs#L768 => W

Extra Information

• unic crate uses v10.0.0 https://github.com/open-i18n/rust-unic/blob/master/unic/ucd/age/tables/unicode_version.rsv#L3
• Thus existing functions unicodedata.ucd_3_2_0.* seem using v10.0.0 instead of v3.2.0 https://github.com/RustPython/RustPython/blob/main/stdlib/src/unicodedata.rs#L68

Possible Solution

• We may (generate and) load Unicode Character Database according to unidata_version, including v3.2.0 and another major version (say, v14.0.0).
• icu crate may be a good option to load dynamic database https://github.com/unicode-org/icu4x/blob/main/components/properties/src/maps.rs#L376
• That means we may need to change all functions in unicodedata module to use database determined by unidata_version.

Example Commits

• ucd crate: 420240c
• icu crate: 38eaa75

Related Tests

• cargo run --release -- -m test test_unicodedata -v
• cargo run --release -- extra_tests/snippets/builtin_str_unicode.py

[youknowone]

Looks great. Thank you. Do you have more concerns not to make this ready?

[xiaozhiyan]

Looks great. Thank you. Do you have more concerns not to make this ready?

I've updated the PR (as in commit 7661fab) and made it ready. Please help to merge it.

[youknowone]

Thank you!