Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/text: Support UnicodeSet as per UTR35 #22920

Open
brawer opened this issue Nov 29, 2017 · 2 comments
Open

x/text: Support UnicodeSet as per UTR35 #22920

brawer opened this issue Nov 29, 2017 · 2 comments
Labels
Milestone

Comments

@brawer
Copy link

@brawer brawer commented Nov 29, 2017

Feature request: Support the UnicodeSet syntax as defined in Unicode Technical Report 35. This would be needed to implement CLDR transliteration rules which use UnicodeSets for filtering and matching; to support CLDR exemplar characters which are also defined in terms of UTR35 UnicodeSets; and other Unicode stuff such as UTR39 Unicode Security Mechanisms that make use of UnicodeSets.

See Unicode’s list-unicodeset tool for an online demo (and its documentation); and the ICU documentation for the ICU API to UnicodeSets. For reference, you might want to have a look at the C++ implementation and the Java implementation inside the ICU sources.

Not sure if this could be implemented by rewriting the string syntax to Go regular expressions, or if this would need more work.

@gopherbot gopherbot added this to the Unreleased milestone Nov 29, 2017
@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Nov 30, 2017

CC @mpvl

@mpvl

This comment has been minimized.

Copy link
Member

@mpvl mpvl commented Dec 14, 2017

I'm pretty sure the RE2 regexp package of Go is incompatible with the exact definition of UnicodeSets in TR35. It is pretty close though.

Enter package regexp/syntax. It exposes the internals of parsing and compilation of this package. It probably won't be too much effort to write an alternative regexp parser and bolt it on the existing engine.

@ALTree ALTree added the NeedsDecision label Mar 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.