[[:punct:]] and \p{Punct}

[k-takata]

Perl's document (perlrecharclass) says that:

\p{PosixPunct} and [[:punct:]] in the ASCII range match all non-controls, non-alphanumeric, non-space characters: [-!"#$%&'()*+,./:;<=>?@[\\\]^_{|}~]`

The similarly named property, \p{Punct} , matches a somewhat different set in the ASCII range, namely [-!"#%&'()*,./:;?@[\\\]_{}]. That is, it is missing the nine characters [$+<=>^|~]`.

In current Onigmo, [[:punct:]] and \p{Punct} is the same in the ASCII range and they depend on the encoding.
If the encoding is Unicode encoding, [[:punct:]] and \p{Punct} don't match the nine characters.
If the encoding is not Unicode encoding, [[:punct:]] and \p{Punct} match the nine characters.

Is it OK?

[tom-lord]

I think this is wrong; both Unicode and non-Unicode should match the nine characters.

http://search.cpan.org/~shay/perl-5.20.2/pod/perlreref.pod

I believe the difference should actually be that under Unicode enoding, [[:punct:]] should additionally match non-ASCII punctuation. The "symbols" ($+<=>^|~`) should always be matched.

[k-takata]

I have decided to change the behavior of [[:punct:]] on Unicode encodings, and already committed into devel-6.0 branch.
Now [[:punct:]] matches the nine characters $+<=>^`|~ on all encodings.
New property \p{XPosixPunct} can be used on Unicode encodings. This is the same as (?u)[[:punct:]].
However \p{Punct} still works differently on Unicode encodings and non-Unicode encodings. It matches the nine characters on non-Unicode encodings, and doesn't match on Unicode encodings.

[k-takata]

Closing.

[k-takata]

Related: https://bugs.ruby-lang.org/issues/12577