on unicode [UNI] string handling

[p6rt]

Migrated from rt.perl.org#128705 (status was 'resolved')

Searchable as RT128705$

[p6rt]

From @ajs

On rakudo version 2016-04 and also the current m​: bot version on IRC, this
code segfaults​:

$ perl6 -e 'for 0..0xffff -> $i { say $i if $i %% 100; my $c = try {
:16(uniprop($i.chr, "Bidi_Mirroring_Glyph")).chr }; say "{$i.fmt("%04x")}​:
{$i.chr} ~ $c" if $c.defined}'
0
100
200
300
400
500
600
700
800
900
1000
1100
Segmentation fault (core dumped)

I think that it's defaulting to uniprop-int, but the odd thing is that
uniprop seems to be returning consecutive integers that aren't related to
the value being tested.

If I call uniprop-str, this works as expected.

--
Aaron Sherman, M.​:
P​: 617-440-4332 Google Talk, Email and Google Plus​: ajs@​ajs.com
Toolsmith, developer, gamer and life-long student.

[p6rt]

From @nwc10

On Sat, Jul 23, 2016 at 10​:48​:40AM -0700, Aaron Sherman wrote​:

$ perl6 -e 'for 0..0xffff -> $i { say $i if $i %% 100; my $c = try {
:16(uniprop($i.chr, "Bidi_Mirroring_Glyph")).chr }; say "{$i.fmt("%04x")}​:
{$i.chr} ~ $c" if $c.defined}'
0
100
200
300
400
500
600
700
800
900
1000
1100
Segmentation fault (core dumped)

I get the SEGV earlier​:

$ ./perl6-m -Ilib -e 'for 0..0xffff -> $i { say $i if $i %% 100; my $c = try { :16(uniprop($i.chr, "Bidi_Mirroring_Glyph")).chr }; say "{$i.fmt("%04x")}​: {$i.chr} ~ $c" if $c.defined}'
0
100
200
ASAN​:SIGSEGV

==20272==ERROR​: AddressSanitizer​: SEGV on unknown address 0x000000000000 (pc 0x7fa2e639f022 sp 0x7fff92696ac0 bp 0x7fff92698660 T0)
  #​0 0x7fa2e639f021 in MVM_interp_run src/core/interp.c​:2859
  #​1 0x7fa2e666f7c4 in MVM_vm_run_file src/moar.c​:304
  #​2 0x401a4f in main src/main.c​:191
  #​3 0x7fa2e5bb0d5c in __libc_start_main (/lib64/libc.so.6+0x1ed5c)
  #​4 0x401058 (/home/nicholas/Sandpit/moar-san/bin/moar+0x401058)

AddressSanitizer can not provide additional info.
SUMMARY​: AddressSanitizer​: SEGV src/core/interp.c​:2859 MVM_interp_run
==20272==ABORTING

Looks like "just" a direct NULL pointer dereference, not the fallout from
earlier more subtle undefined behaviour.

Nicholas Clark

[p6rt]

The RT System itself - Status changed from 'new' to 'open'

[p6rt]

From @samcv

On Sat, 23 Jul 2016 10​:48​:40 -0700, ajs wrote​:

On rakudo version 2016-04 and also the current m​: bot version on IRC, this
code segfaults​:

$ perl6 -e 'for 0..0xffff -> $i { say $i if $i %% 100; my $c = try {
:16(uniprop($i.chr, "Bidi_Mirroring_Glyph")).chr }; say "{$i.fmt("%04x")}​:
{$i.chr} ~ $c" if $c.defined}'
...
I think that it's defaulting to uniprop-int, but the odd thing is that
uniprop seems to be returning consecutive integers that aren't related to
the value being tested.

If I call uniprop-str, this works as expected.

Support has been added for the Bidi_Mirroring_Glyph property since this was reported. Before it would give you a string which was a hex value. Now uniprop will give you the character as a string. If you query it with uniprop-int you will get the codepoint as an integer.

Closing this ticket.

[p6rt]

@samcv - Status changed from 'open' to 'resolved'