\N{...} neglects formal aliases and named sequences from Unicode charnames namespace

[tchrist]

BPO	12753
Nosy	@malemburg, @gvanrossum, @loewis, @terryjreedy, @abalkin, @ezio-melotti, @florentx
Superseder	bpo-4610: Unicode case mappings are incorrect
Files	nametests.py: test case to check unicodedata.lookup and \N{} against named chars AND formal alias AND named sequences issue12753.diff: patch to add the aliases issue12753-2.diff: patch to add the aliases and named sequences issue12753-3.diff: patch to add the aliases and named sequences + tests + doc issue12753-4.diff: patch to add the aliases and named sequences + tests + doc

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ezio-melotti'
closed_at = <Date 2011-10-21.20:22:13.698>
created_at = <Date 2011-08-15.17:48:33.715>
labels = ['interpreter-core', 'type-feature', 'expert-unicode']
title = '\\N{...} neglects formal aliases and named sequences from Unicode charnames namespace'
updated_at = <Date 2013-06-23.22:32:03.145>
user = 'https://bugs.python.org/tchrist'

bugs.python.org fields:

activity = <Date 2013-06-23.22:32:03.145>
actor = 'belopolsky'
assignee = 'ezio.melotti'
closed = True
closed_date = <Date 2011-10-21.20:22:13.698>
closer = 'ezio.melotti'
components = ['Interpreter Core', 'Unicode']
creation = <Date 2011-08-15.17:48:33.715>
creator = 'tchrist'
dependencies = []
files = ['22903', '23273', '23280', '23291', '23374']
hgrepos = []
issue_num = 12753
keywords = ['patch', 'needs review']
message_count = 37.0
messages = ['142136', '142145', '142502', '142506', '142507', '142508', '143043', '144679', '144681', '144703', '144708', '144716', '144738', '144739', '144757', '144758', '144760', '144779', '144783', '144802', '144803', '144825', '144827', '144832', '144836', '144839', '145254', '145263', '145327', '145401', '146034', '146036', '146075', '146114', '146129', '146135', '191737']
nosy_count = 10.0
nosy_names = ['lemburg', 'gvanrossum', 'loewis', 'terry.reedy', 'belopolsky', 'ezio.melotti', 'mrabarnett', 'flox', 'python-dev', 'tchrist']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = '4610'
type = 'enhancement'
url = 'https://bugs.python.org/issue12753'
versions = ['Python 3.3']

[tchrist]

Unicode character names share a common namespace with formal aliases and with named sequences, but Python recognizes only the original name. That means not everything in the namespace is accessible from Python. (If this is construed to be an extant bug from than an absent feature, you probably want to change this from a wish to a bug in the ticket.)

This is a problem because aliases correct errors in the original names, and are the preferred versions. For example, ISO screwed up when they called U+01A2 LATIN CAPITAL LETTER OI. It is actually LATIN CAPITAL LETTER GHA according to the file NameAliases.txt in the Unicode Character Database. However, Python blows up when you try to use this:

% env PYTHONIOENCODING=utf8 python3.2-narrow -c 'print("\N{LATIN CAPITAL LETTER OI}")'
Ƣ

% env PYTHONIOENCODING=utf8 python3.2-narrow -c 'print("\N{LATIN CAPITAL LETTER GHA}")'
  File "<string>", line 1
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 0-27: unknown Unicode character name
Exit 1

This unfortunate, because the formal aliases correct egregious blunders, such as the Standard reading "BRAKCET" instead of "BRACKET":

$ uninames '^\s+%'
 Ƣ  01A2        LATIN CAPITAL LETTER OI
        % LATIN CAPITAL LETTER GHA
 ƣ  01A3        LATIN SMALL LETTER OI
        % LATIN SMALL LETTER GHA
        * Pan-Turkic Latin alphabets
 ೞ  0CDE        KANNADA LETTER FA
        % KANNADA LETTER LLLA
        * obsolete historic letter
        * name is a mistake for LLLA
 ຝ  0E9D        LAO LETTER FO TAM
        % LAO LETTER FO FON
        = fo fa
        * name is a mistake for fo sung
 ຟ  0E9F        LAO LETTER FO SUNG
        % LAO LETTER FO FAY
        * name is a mistake for fo tam
 ຣ  0EA3        LAO LETTER LO LING
        % LAO LETTER RO
        = ro rot
        * name is a mistake, lo ling is the mnemonic for 0EA5
 ລ  0EA5        LAO LETTER LO LOOT
        % LAO LETTER LO
        = lo ling
        * name is a mistake, lo loot is the mnemonic for 0EA3
 ࿐  0FD0        TIBETAN MARK BSKA- SHOG GI MGO RGYAN
        % TIBETAN MARK BKA- SHOG GI MGO RGYAN
        * used in Bhutan
 ꀕ A015        YI SYLLABLE WU
        % YI SYLLABLE ITERATION MARK
        * name is a misnomer
 ︘ FE18        PRESENTATION FORM FOR VERTICAL RIGHT WHITE LENTICULAR BRAKCET
        % PRESENTATION FORM FOR VERTICAL RIGHT WHITE LENTICULAR BRACKET
        * misspelling of "BRACKET" in character name is a known defect
        # <vertical> 3017
 𝃅  1D0C5       BYZANTINE MUSICAL SYMBOL FHTORA SKLIRON CHROMA VASIS
        % BYZANTINE MUSICAL SYMBOL FTHORA SKLIRON CHROMA VASIS
        * misspelling of "FTHORA" in character name is a known defect

There are only

In Perl, \N{...} grants access to the single, shared, common namespace of Unicode character names, formal aliases, and named sequences without distinction:

% env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{LATIN CAPITAL LETTER OI}")'
Ƣ
% env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{LATIN CAPITAL LETTER GHA}")'
Ƣ

% env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{LATIN CAPITAL LETTER OI}")'  | uniquote -x
\x{1A2}
% env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{LATIN CAPITAL LETTER GHA}")' | uniquote -x
\x{1A2}

It is my suggestion that Python do the same thing. There are currently only 11 of these.

The third element in this shared namespace of name, named sequences, are multiple code points masquerading under one name. They come from the NamedSequences.txt file in the Unicode Character Database. An example entry is:

LATIN CAPITAL LETTER A WITH MACRON AND GRAVE;0100 0300

There are 418 of these named sequences as of Unicode 6.0.0. This shows that Perl can also access named sequences:

  $ env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}")'
  Ā̀

  $ env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}")' | uniquote -x
  \x{100}\x{300}

  $ env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{KATAKANA LETTER AINU P}")'            
  ㇷ゚

  $ env PERL_UNICODE=S perl -Mcharnames=:full -le 'print("\N{KATAKANA LETTER AINU P}")' | uniquote -x
   \x{31F7}\x{309A}

Since it is a single namespace, it makes sense that all members of that namespace should be accessible using \N{...} as a sort of equal-opportunity accessor mechanism, and it does not make sense that they not be.

Just makes sure you take only the approved named sequences from the NamedSequences.txt file. It would be unwise to give users access to the provisional sequences located in a neighboring file I shall not name :) because those are not guaranteed never to be withdrawn the way the others are, and so you would risk introducing an incompatibility.

If you look at the ICU UCharacter class, you can see that they provide a more

[tchrist]

Here’s the right test file for the right ticket.

[terryjreedy]

I verified that the test file raises the quoted SyntaxError on 3.2 on Win7. This:

>>> "\N{LATIN CAPITAL LETTER GHA}"
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 0-27: unknown Unicode character name

is most likely a result of this:

>>> unicodedata.lookup("LATIN CAPITAL LETTER GHA")
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    unicodedata.lookup("LATIN CAPITAL LETTER GHA")
KeyError: "undefined character name 'LATIN CAPITAL LETTER GHA'"

Although the lookup comes first in nametests.py, it is never executed because of the later SyntaxError.

The Reference for string literals says"
"\N{name} Character named name in the Unicode database"

The doc for unicodedata says
"This module provides access to the Unicode Character Database (UCD) which defines character properties for all Unicode characters. The data contained in this database is compiled from the UCD version 6.0.0.

The module uses the same names and symbols as defined by Unicode Standard Annex #44, “Unicode Character Database”."
http://www.unicode.org/reports/tr44/tr44-6.html

So the question is, what are the 'names' therein defined?
All such should be valid inputs to
"unicodedata.lookup(name) Look up character by name."

The annex refers to http://www.unicode.org/Public/6.0.0/ucd/
This contains NamesList.txt, derived from UnicodeData.txt. Unicodedata must be using just the latter. The ucd directory also contains NameAliases.txt, NamedSequences.txt, and the file of provisional named sequences.

As best I can tell, the annex plus files are a bit ambiguous as to 'Unicode character name'. The following quote seems neutral: "the Unicode Character Database (UCD), a collection of data files which contain the Unicode character code points and character names." The following: "Unicode character names constitute a special case. Formally, they are values of the Name property." points toward UnicodeData.txt, which lists the Name property along with others. However, "Unicode character name, as published in the Unicode names list," indirectly points toward including aliases. NamesList.txt says it contains the "Final Unicode 6.0 names list." (but one which "should not be parsed for machine-readable information". It includes all 11 aliases in NameAliases.txt.

My current opinion is that adding the aliases might be done in current releases. It certainly would serve the any user who does not know to misspell 'FTHORA' as 'FHTORA' for just one of the 17 'FTHORA' chars.

Adding named sequences is definitely a feature request. The definition of .lookup(name) would be enlarged to "Look up character by name, alias, or named sequence" with reference to the specific files. The meaning of \N{} would also have to be enlarged.

Minimal test code might be:

from unicodedata import lookup
AssertEqual(lookup("LATIN CAPITAL LETTER GHA")), "\u01a2")
AssertEqual(lookup("LATIN CAPITAL LETTER A WITH MACRON AND GRAVE"),
   "\u0100\u0300")
plus a test that "\N{LATIN CAPITAL LETTER GHA}" and
"\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}" compile without error (I have no idea how to write that).

---

"If you look at the ICU UCharacter class, you can see that they provide a more"

More what ;-)
I presume ICU =International Components for Unicode, icu-project.org/
"Offers a portable set of C/C++ and Java libraries for Unicode support, software internationalization (I18N) and globalization (G11N)."
[appears to be free, open source, and possibly usable within Python]

[tchrist]

"Terry J. Reedy" <report@bugs.python.org> wrote
on Fri, 19 Aug 2011 22:50:58 -0000:

My current opinion is that adding the aliases might be done in current
releases. It certainly would serve the any user who does not know to
misspell 'FTHORA' as 'FHTORA' for just one of the 17 'FTHORA' chars.

Yes, I think the 11 aliases pose no problem. It's amazing the trouble
you get into from having a fat-fingered amanuesis typing your laws
into indelible stone tablets.

Adding named sequences is definitely a feature request. The definition
of .lookup(name) would be enlarged to "Look up character by name,
alias, or named sequence" with reference to the specific files. The
meaning of \N{} would also have to be enlarged.

But these do. The problem is bracketed character classes.
Yes, if you got named reference into the regex compiler as a raw
string, it could in theory rewrite

[abc\N{seq}] 

as

(?:[abc]|\N{seq})

but that doesn't help if the sequence got replaced as a string escape.
At which point you have different behavior in the two lookalike cases.

If you ask how we do this in Perl, the answer is "poorly". It really only
works well in strings, not charclasses, although there is a proposal to do
a rewrite during compilation like I've spelled out above. Seems messy for
something that might(?) not get much use. But it would be nice for \N{} to
work to access the whole namespace without prejudice. I have a feeling
this may be a case of trying to keep one's cake and eating it too, as
the two goals seem to rule each other out.

> "If you look at the ICU UCharacter class, you can see that they provide a more"

More what ;-)

More expressive set of lookup functions where it is clear which thing
you are getting. I believe the ICU regexes only support one-char returns
for \N{...}, not multis per the sequences. But I may not be looking
at the right docs for ICU; not sure.

I presume ICU =International Components for Unicode, icu-project.org/
"Offers a portable set of C/C++ and Java libraries for Unicode support,
software internationalization (I18N) and globalization (G11N)." [appears
to be free, open source, and possibly usable within Python]

Well, there are some Python bindings for ICU that I was eager to try out,
because I wanted to see whether I couild get at full/real Unicode collation
that way, but I had trouble getting the Python bindings to compile. Not
sure why. The documentation for the Python bindings isn't very um wordy,
and it isn't clear how tightly integrated it all is: there's talk about C++
strings that kind of scares me. :)

Hm, and maybe they are only for Python 2 not Python 3, which I try to do
all my Python stuff in because it seems like it has a better Unicode model.

--tom

[mrabarnett]

For the "Line_Break" property, one of the possible values is "Inseparable", with 2 permitted aliases, the shorter "IN" (which is reasonable) and "Inseperable" (ouch!).

[tchrist]

Matthew Barnett <report@bugs.python.org> wrote
on Fri, 19 Aug 2011 23:36:45 -0000:

For the "Line_Break" property, one of the possible values is
"Inseparable", with 2 permitted aliases, the shorter "IN" (which
is reasonable) and "Inseperable" (ouch!).

Yeahy, I've shaken my head at that one, too.

It's one thing to make an alias for something you typo'd in the first
place, but to have something that's correct which you then make a typo
alias for is just encouraging bad/sloppy/wrong behavior.

    Bidi_Class=Paragraph_Separator
    Bidi_Class=Common_Separator
    Bidi_Class=European_Separator
    Bidi_Class=Segment_Separator
    General_Category=Line_Separator
    General_Category=Paragraph_Separator
    General_Category=Separator
    General_Category=Space_Separator
    Line_Break=Inseparable
    Line_Break=Inseperable

And there's still set, which makes you wonder
why they couldn't spell at least *one* of them out:

    Sentence_Break=Sep SB=SE
    Sentence_Break=Sp  SB=Sp

You really have to look those up to realize they're two different things:

SB ; SE        ; Sep
SB ; SP        ; Sp

And that none of them have something like SB=Space or SB=Separator
so you know what you're talking about. Grrr.

--tom

[gvanrossum]

+1 on the feature request.

[ezio-melotti]

The attached patch changes Tools/unicode/makeunicodedata.py to create a list of names and codepoints taken from http://www.unicode.org/Public/6.0.0/ucd/NameAliases.txt and adds it to Modules/unicodename_db.h.
During the lookup the _getcode function at Modules/unicodedata.c:1055 loops over the 11 aliases and checks if any of those match.
The patch also includes tests for both unicodedata.lookup and \N{}.

I'm not sure this is the best way to implement this, and someone will probably want to review and tweak both the approach and the C code, but it works fine:
>>> "\N{LATIN CAPITAL LETTER GHA}"
'Ƣ'
>>> import unicodedata
>>> unicodedata.lookup("LATIN CAPITAL LETTER GHA")
'Ƣ'
>>> "\N{LATIN CAPITAL LETTER OI}"
'Ƣ'
>>> unicodedata.lookup("LATIN CAPITAL LETTER OI")
'Ƣ'

The patch doesn't include changes for NamedSequences.txt.

[loewis]

I propose to use a better lookup algorithm using binary search, and then integrate the NamedSequences into this as well. The search result could be a record

 struct {
   char *name;
   int len;
   Py_UCS4 chars[3]; /* no sequence is more than 3 chars */
 }

You would have two tables for these: one for the aliases, and one for the named sequences.

_getcode would continue to return a single char only, and thus not support named sequences. lookup could well return strings longer than 1, but only in 3.3.

I'm not sure that \N escapes should support named sequences: people rightfully expect that each escaped element in a string literal constitutes exactly one character.

[ezio-melotti]

Leaving named sequences for unicodedata.lookup() only (and not for \N{}) makes sense.

The list of aliases is so small (11 entries) that I'm not sure using a binary search for it would bring any advantage. Having a single lookup algorithm that looks in both tables doesn't work because the aliases lookup must be in _getcode for \N{...} to work, whereas the lookup of named sequences will happen in unicodedata_lookup (Modules/unicodedata.c:1187).
I think we can leave the for loop over aliases in _getcode and implement a separate (and binary) search in unicodedata_lookup for the named sequences. Does that sound fine?

[tchrist]

Ezio Melotti <ezio.melotti@gmail.com> added the comment:

Leaving named sequences for unicodedata.lookup() only (and not for
\N{}) makes sense.

There are certainly advantages to that strategy: you don't have to
deal with [\N{sequence}] issues. If the argument to unicode.lookup()
and be any of name, alias, or sequence, that seems ok. \N{} should
still do aliases, though, since those don't have the complication that
sequences have.

You may wish unicode.name() to return the alias in preference, however.
That's what we do. And of course, there is no issue of sequences there.

The rest of this perhaps painfully long message is just elaboration
and icing on what I've said above.

--tom

The list of aliases is so small (11 entries) that I'm not sure using a
binary search for it would bring any advantage. Having a single
lookup algorithm that looks in both tables doesn't work because the
aliases lookup must be in _getcode for \N{...} to work, whereas the
lookup of named sequences will happen in unicodedata_lookup
(Modules/unicodedata.c:1187). I think we can leave the for loop over
aliases in _getcode and implement a separate (and binary) search in
unicodedata_lookup for the named sequences. Does that sound fine?

If you mean, is it ok to add just the aliases and not the named sequences to
\N{}, it is certainly better than not doing so at all. Plus that way you do
*not* have to figure out what in the world to to do with [^a-c\\N{sequence}],
since that would have be something like (?!\N{sequence})[^a-c]), which is
hardly obvious, especially if \N{sequence} actually starts with [a-c].

However, because the one namespace comprises all three of names,
aliases, and named sequences, it might be best to have a functional
(meaning, non-regex) API that allows one to do a fetch on the whole
namespace, or on each individual component.

The ICU library supports this sort of thing. In ICU4J's Java bindings,
we find this:

    static int getCharFromExtendedName(String name) 
       [icu] Find a Unicode character by either its name and return its code point value.
    static int	getCharFromName(String name) 
       [icu] Finds a Unicode code point by its most current Unicode name and return its code point value.
    static int	getCharFromName1_0(String name) 
       [icu] Find a Unicode character by its version 1.0 Unicode name and return its code point value.
    static int	getCharFromNameAlias(String name) 
       [icu] Find a Unicode character by its corrected name alias and return its code point value.

The first one obviously has a bug in its definition, as the English
doesn't scan. Looking at the full definition is even worse. Rather
than dig out the src jar, I looked at ICU4C, but its own bindings are
completely different. There you have only one function, with an enum to
say what namespace to access:

UChar32 u_charFromName  (       UCharNameChoice         nameChoice, 
	    const char *    name, 
	    UErrorCode *    pErrorCode 
    )

The UCharNameChoice enum tells what sort of thing you want:

U_UNICODE_CHAR_NAME,
U_UNICODE_10_CHAR_NAME,
U_EXTENDED_CHAR_NAME,
U_CHAR_NAME_ALIAS,          
U_CHAR_NAME_CHOICE_COUNT

Looking at the src for the Java is no more immediately illuminating,
but I think that "extended" may refer to a union of the old 1.0 names
with the current names.

Now I'll tell you what Perl does. I do this not to say it is "right",
but just to show you one possible strategy. I also am in the middle
of writing about this for the Camel, so it is in my head.

Perl does not provide the old 1.0 names at all. We don't have a Unicode
1.0 legacy to support, which makes this cleaner. However, we do provide
for the names of the C0 and C1 Control Codes, because apart from Unicode
1.0, they don't condescend to name the ASCII or Latin1 control codes.

We also provide for certain well known aliases from the Names file:
anything that says "* commonly abbreviated as ...", so things like LRO
and ZWJ and such.

Perl makes no distinction between anything in the namespace when using
the \N{} form for string and regex escapes. That means when you use
"\N{...}" or /\N{...}/, you don't know which it is, nor can you.
(And yes, the bracketed character class issue is annoying and unsolved.)

However, the "functional" API does make a slight distinction.

-- charnames::vianame() takes a name or alias (as a string) and returns a single
integer code point.

eg: This therefore converts "LATIN SMALL LETTER A" into 0x61.
    It also converts both 
	BYZANTINE MUSICAL SYMBOL FHTORA SKLIRON CHROMA VASIS
    and 
	BYZANTINE MUSICAL SYMBOL FTHORA SKLIRON CHROMA VASIS
    into 0x1D0C5.  See below.

-- charnames::string_vianame() takes a string name, alias, *or* sequence,
and gives back a string.

eg: This therefore converts "LATIN SMALL LETTER A" into "a".
        Since it has a string return instead of an int, it now also
        handles everything from NamedSequences file as well. (See below.)

-- charnames::viacode() takes an integer can gives back the official alias
if there is one, and the official name if there is not.

eg: This converts 0x61 into "LATIN SMALL LETTER A".
        It also converts 0x1D0C5 into "BYZANTINE MUSICAL SYMBOL FTHORA
        SKLIRON CHROMA VASIS".

Consider

BYZANTINE MUSICAL SYMBOL FHTORA SKLIRON CHROMA VASIS

That was an error, and there is an official alias fixing it:

BYZANTINE MUSICAL SYMBOL FTHORA SKLIRON CHROMA VASIS

(That's FHTORA vs FTHORA.)

You may use either as the name, and if you reverse the code
point to name, you get the replacement alias.

% perl -mcharnames -wle 'printf "%04X\n", charnames::vianame("BYZANTINE MUSICAL SYMBOL FHTORA SKLIRON CHROMA VASIS")'
1D0C5

% perl -mcharnames -wle 'printf "%04X\n", charnames::vianame("BYZANTINE MUSICAL SYMBOL FTHORA SKLIRON CHROMA VASIS")'
1D0C5

% perl -mcharnames -wle 'print charnames::viacode(charnames::vianame("BYZANTINE MUSICAL SYMBOL FHTORA SKLIRON CHROMA VASIS"))'
BYZANTINE MUSICAL SYMBOL FTHORA SKLIRON CHROMA VASIS

So on round-tripping, I gave it the "wrong" one (the original) and it gave
me back the "right" one (the replacement).

Using the \N{} thing, it again doesn't matter:

% perl -mcharnames=:full -wle 'printf "%04X\n", ord "\N{BYZANTINE MUSICAL SYMBOL FHTORA SKLIRON CHROMA VASIS}"'
1D0C5

% perl -mcharnames=:full -wle 'printf "%04X\n", ord "\N{BYZANTINE MUSICAL SYMBOL FTHORA SKLIRON CHROMA VASIS}"'
1D0C5

The interesting thing is the named sequences. string_vianame() works just fine on those:

% perl -mcharnames -wle 'print length charnames::string_vianame("LATIN CAPITAL LETTER A WITH MACRON AND GRAVE")'
2

% perl -mcharnames -wle 'printf "U+%v04X\n", charnames::string_vianame("LATIN CAPITAL LETTER A WITH MACRON AND GRAVE")'
U+0100.0300

And that works fine with \N{} as well (provided you don't try charclasses):

% perl -mcharnames=:full -wle 'print "\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}"'
Ā̀

% perl -mcharnames=:full -wle 'print "\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}"' | uniquote -v
\N{LATIN CAPITAL LETTER A WITH MACRON}\N{COMBINING GRAVE ACCENT}

% perl -mcharnames=:full -wle 'print length "\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}"'
2

% perl -mcharnames=:full -wle 'printf "U+%v04X\n", "\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}"'
U+0100.0300

It's kinda sad that for \N{} and sequneces you can't just "do the right
thing" with strings and say that charclass stuff just isn't supported.
But my guess is that this simply won't work because you don't have
first class regexes. If you pass both of these to the regex engine,
they should behave the same (and would, assuming the regex compiler
knows about \N{} escapes):

"\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}"
r'\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}'

However, that falls part if you do

"[^\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}]"
r'[^\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}]'

Because the compiler will do the substitution early on the first
one but not the second. This seems a problem, eh? So I guess
you can't do it at all? Or could you document it? I think there
is no good solution here. Perl can and does actually do something
quite reasonable in the noncharclass case, but that is because we
know that we are compiling a regex in virtually all scenarios.

% perl -Mcharnames=:full -le 'print qr/\N{LATIN SMALL LETTER A}/'
(?^u:\N{U+61})

% perl -Mcharnames=:full -le 'print qr/\N{LATIN CAPITAL LETTER A WITH MACRON}/'
(?^u:\N{U+100})

% perl -Mcharnames=:full -le 'print qr/\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}/'
(?^u:\N{U+100.300})

So you can do:

% perl -Mcharnames=:full -le 'print "\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}" =~ /\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}/'
1

And it is just fine. The issue is that there are ways for you to get
yoruself into trouble if you do string-string stuff:

% perl -Mcharnames=:full -le 'print "\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}" =~ "\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}"'
1
% perl -Mcharnames=:full -le 'print "\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}" =~ "^[\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}]+\$"'
1

That works, but only accidentally, because of course U+0100.0300 contains
nothing but either U+0100 or U+0300.

This is not a solved problem.

I hope this helps.

--tom