v5.25.5-184-ga5540cf breaks texinfo

[p5pRT]

Migrated from rt.perl.org#130010 (status was 'resolved')

Searchable as RT130010$

[p5pRT]

From @jkeenan

[Transferring to RT a report of blead breaking a downstream program
originally filed on perl.perl5.porters by Mathieu Arnold. -- jkeenan]

#####

Like every week, I updated the perl5-devel FreeBSD port (this week, from
v5.25.5-115-g4eadd82 to v5.25.6-72-g92c843f). Perl built just fine, and
make test runs just fine fine too.

Today, I discovered that texinfo could not be built with the new Perl
version, after a quick git bisect (git bisect is your god), I ended up
with v5.25.5-183-g8204e83 being the last good, and v5.25.5-184-ga5540cf
being the first bad.

The error I get in texinfo is​:

Modification of a read-only value attempted at
../tp/Texinfo/Convert/Line.pm line 309.

That line being​:

} elsif ($text =~ s/^(([^\s\p{InFullwidth}]|[\x{202f}\x{00a0}])+)//) {

When that fails, $text contains " Texinfo documentation system".

Adding use diagnostics to that file, I get [pastebin.com link has
expired]

It turns out that the problem is \p{InFullwidth} (comes from
Unicode​::EastAsianWidth) and its usage, in that case (I don't know
texinfo at all) got broken.

[For reference, Texinfo​::Convert​::Line is part of the GNU texinfo
library​: http​://ftpmirror.gnu.org/texinfo/texinfo-6.1.tar.gz]

Mat​: Could you please provide the output of ./perl -lib -V from the
machine on which you detected this problem? Also, since it appears that
your original pastebin link has expired, can you provide its content in
a reply to this ticket?

Thank you very much.
Jim Keenan

[perl -V info omitted because this is not being sent from the
problematic environment.]

[p5pRT]

From @khwilliamson

As a BBC ticket, I've marked this as a blocker for 5.26
--
Karl Williamson

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From mat@cpan.org

Content of the pastebin​:

Modification of a read-only value attempted at ../tp/Texinfo/Convert/Line.pm
  line 309 (#1)
  (F) You tried, directly or indirectly, to change the value of a
  constant. You didn't, of course, try "2 = 1", because the compiler
  catches that. But an easy way to do the same thing is​:

  sub mod { $_[0] = 1 }
  mod(2);

  Another way is to assign to a substr() that's off the end of the string.

  Yet another way is to assign to a foreach loop VAR when VAR
  is aliased to a constant in the look LIST​:

  $x = 1;
  foreach my $n ($x, 2) {
  $n *= 2; # modifies the $x, but fails on attempt to
  } # modify the 2

Uncaught exception from user code​:
  Modification of a read-only value attempted at ../tp/Texinfo/Convert/Line.pm line 309.
  Texinfo​::Convert​::Line​::add_text(Texinfo​::Convert​::Line=HASH(0x80872f7f8), "Texinfo documentation system") called at ../tp/Texinfo/Convert/Plaintext.pm line 1668
  Texinfo​::Convert​::Plaintext​::_convert(Texinfo​::Convert​::Info=HASH(0x808701900), HASH(0x805486360)) called at ../tp/Texinfo/Convert/Plaintext.pm line 3187
  Texinfo​::Convert​::Plaintext​::_convert(Texinfo​::Convert​::Info=HASH(0x808701900), HASH(0x808708030)) called at ../tp/Texinfo/Convert/Plaintext.pm line 705
  Texinfo​::Convert​::Plaintext​::convert_line(Texinfo​::Convert​::Info=HASH(0x808701900), HASH(0x808708030)) called at ../tp/Texinfo/Convert/Info.pm line 340
  Texinfo​::Convert​::Info​::_info_header(Texinfo​::Convert​::Info=HASH(0x808701900)) called at ../tp/Texinfo/Convert/Info.pm line 81
  Texinfo​::Convert​::Info​::output(Texinfo​::Convert​::Info=HASH(0x808701900), HASH(0x8053d33c0)) called at ../tp/texi2any line 1348

[p5pRT]

From mat@cpan.org

root@​11amd64-ports-perl5-devel​:~ # perl -lib -V
-i used with no filenames on the command line, reading from STDIN.
Summary of my perl5 (revision 5 version 25 subversion 7) configuration​:

  Platform​:
  osname=freebsd
  osvers=11.0-release-p3
  archname=amd64-freebsd-thread-multi
  uname='freebsd 11amd64-ports-perl5-devel-job-02 11.0-release-p3 freebsd 11.0-release-p3 amd64 '
  config_args='-sde -Dprefix=/usr/local -Dlibperl=libperl.so.5.25.6.144 -Darchlib=/usr/local/lib/perl5/5.25/mach -Dprivlib=/usr/local/lib/perl5/5.25 -Dman3dir=/usr/local/lib/perl5/5.25/perl/man/man3 -Dman1dir=/usr/local/lib/perl5/5.25/perl/man/man1 -Dsitearch=/usr/local/lib/p
erl5/site_perl/mach/5.25 -Dsitelib=/usr/local/lib/perl5/site_perl -Dscriptdir=/usr/local/bin -Dsiteman3dir=/usr/local/lib/perl5/site_perl/man/man3 -Dsiteman1dir=/usr/local/lib/perl5/site_perl/man/man1 -Ui_malloc -Ui_iconv -Uinstallusrbinperl -Dusenm=n -Dcc=cc -Duseshrplib -Dinc
_version_list=none -Dcf_by=mat -Dcf_email=mat@​FreeBSD.org -Dcf_time=Wed Nov 02 10​:24​:46 UTC 2016 -Alddlflags=-L/wrkdirs/usr/ports/lang/perl5-devel/work/perl5-5.25.6-144-g26e9d72 -L/usr/local/lib/perl5/5.25/mach/CORE -lperl -Dshrpldflags=$(LDDLFLAGS​:N-L/wrkdirs/usr/ports/lang/pe
rl5-devel/work/perl5-5.25.6-144-g26e9d72​:N-L/usr/local/lib/perl5/5.25/mach/CORE​:N-lperl) -Wl,-soname,$(LIBPERL) -Dusedevel -Uversiononly -Doptimize=-O2 -pipe -fstack-protector -fno-strict-aliasing -Ui_gdbm -Dusemultiplicity=y -Duse64bitint -Dusethreads=y -Dusemymalloc=n'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=define
  usemultiplicity=define
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
  optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing'
  cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='4.2.1 Compatible FreeBSD Clang 3.8.0 (tags/RELEASE_380/final 262564)'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags ='-lpthread -Wl,-E -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/lib /usr/local/lib /usr/bin/../lib/clang/3.8.0/lib /usr/lib
  libs=-lpthread -lm -lcrypt -lutil
  perllibs=-lpthread -lm -lcrypt -lutil
  libc=
  so=so
  useshrplib=true
  libperl=libperl.so.5.25.6.144
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags=' -Wl,-R/usr/local/lib/perl5/5.25/mach/CORE'
  cccdlflags='-DPIC -fPIC'
  lddlflags='-shared -L/usr/local/lib/perl5/5.25/mach/CORE -lperl -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:

  Compile-time options​:

  HAS_TIMES

  MULTIPLICITY

  PERLIO_LAYERS

  PERL_COPY_ON_WRITE

  PERL_DONT_CREATE_GVSV

  PERL_HASH_FUNC_ONE_AT_A_TIME_HARD

  PERL_IMPLICIT_CONTEXT

  PERL_MALLOC_WRAP

  PERL_OP_PARENT

  PERL_PRESERVE_IVUV

  PERL_USE_DEVEL

  USE_64_BIT_ALL

  USE_64_BIT_INT

  USE_ITHREADS

  USE_LARGE_FILES

  USE_LOCALE

  USE_LOCALE_COLLATE

  USE_LOCALE_CTYPE

  USE_LOCALE_NUMERIC

  USE_LOCALE_TIME

  USE_PERLIO

  USE_PERL_ATOF

  USE_REENTRANT_API

  Built under freebsd

  @​INC​:

  /usr/local/lib/perl5/site_perl/mach/5.25

  /usr/local/lib/perl5/site_perl

  /usr/local/lib/perl5/5.25/mach

  /usr/local/lib/perl5/5.25

  .

[p5pRT]

From @jkeenan

On Wed Nov 02 20​:33​:30 2016, jkeen@​verizon.net wrote​:

[Transferring to RT a report of blead breaking a downstream program
originally filed on perl.perl5.porters by Mathieu Arnold. -- jkeenan]

[For reference, Texinfo​::Convert​::Line is part of the GNU texinfo
library​: http​://ftpmirror.gnu.org/texinfo/texinfo-6.1.tar.gz]

Today I downloaded this tarball and attempted to configure/build/test it following the instructions in its INSTALL.generic file.

I experienced a problem as early as './configure'. As that shell script ran, at one point I got output like this​:

#####
/bin/bash ./libtool --tag=CC --mode=link cc -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DVERSION=\"6.0\" -DXS_VERSION=\"6.0\" "-I/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/lib/5.24.0/x86_64-linux/CORE" -no-undefined -L/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/lib/5.24.0/x86_64-linux/CORE -lperl -avoid-version -module -Wl,-E -o TestXS.la -rpath /usr/local/lib/texinfo TestXS_la-TestXS.lo
libtool​: link​: cc -shared -fPIC -DPIC .libs/TestXS_la-TestXS.o -L/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/lib/5.24.0/x86_64-linux/CORE -lperl -Wl,-E -Wl,-soname -Wl,TestXS.so -o .libs/TestXS.so
/usr/bin/ld​: /home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/lib/5.24.0/x86_64-linux/CORE/libperl.a(op.o)​: relocation R_X86_64_32S against `PL_opargs' can not be used when making a shared object; recompile with -fPIC
/home/jkeenan/perl5/perlbrew/perls/perl-5.24.0/lib/5.24.0/x86_64-linux/CORE/libperl.a​: error adding symbols​: Bad value
collect2​: error​: ld returned 1 exit status
Makefile​:1124​: recipe for target 'TestXS.la' failed
make​: *** [TestXS.la] Error 1
checking whether we can build Perl extension (XS) modules... no
#####

(Full typescript available upon request.)

texinfo's ./configure helpfully writes a 'config.log' file. grepping that file, I observe repeated instances of the following​:

#####
configure​: failed program was​:
| /* confdefs.h */
| #define PACKAGE_NAME "GNU Texinfo"
| #define PACKAGE_TARNAME "texinfo"
| #define PACKAGE_VERSION "6.1"
| #define PACKAGE_STRING "GNU Texinfo 6.1"
| #define PACKAGE_BUGREPORT "bug-texinfo@​gnu.org"
| #define PACKAGE_URL "http​://www.gnu.org/software/texinfo/"
| #define PACKAGE "texinfo"
| #define VERSION "6.1"
| /* end confdefs.h. */
| #include <ac_nonexistent.h>
#####

Preliminary inferences​:

1. texinfo's problems are not limited to FreeBSD.

2. texinfo's problems are not limited to that introduced at the commit point in 5.25 reported by mat. The program's ./configure appears to have problems on perl-5.24.0.

3. But since once I ran 'make' I had what appeared to be a valid 'Makefile' -- and since I know little about texinfo itself -- I'm not sure of the real importance of (1) and (2) or its relation to the blead breakage reported by mat.

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

From @jkeenan

On Thu Nov 03 11​:20​:48 2016, jkeenan wrote​:
[snip]

Preliminary inferences​:

1. texinfo's problems are not limited to FreeBSD.

2. texinfo's problems are not limited to that introduced at the commit
point in 5.25 reported by mat. The program's ./configure appears to
have problems on perl-5.24.0.

3. But since once I ran 'make' I had what appeared to be a valid
'Makefile' -- and since I know little about texinfo itself -- I'm not
sure of the real importance of (1) and (2) or its relation to the
blead breakage reported by mat.

I subsequently installed blead​:

#####
This is perl 5, version 25, subversion 7 (v5.25.7 (v5.25.6-172-g3e8592f)) built for x86_64-linux
(with 56 registered patches, see perl -V for more detail)
#####

... and then, before calling texinfo's './Configure', I said​:

#####
export PERL=/home/jkeenan/testing/blead/bin/perl
#####

I got similar errors in config.log, but, once again, they didn't seem to prevent me from getting a valid 'Makefile'.

I then called 'make check' and logged the output. texinfo's 'make check' runs quite a few shell scripts.

#####
FAIL​: test_scripts/formatting_cond_info.sh
FAIL​: test_scripts/formatting_cond_info_no-ifhtml_no-ifinfo_no-iftex.sh
FAIL​: test_scripts/formatting_cond_info_ifhtml_ifinfo_iftex.sh
FAIL​: test_scripts/formatting_direntry_dircategory_info_split.sh
FAIL​: test_scripts/formatting_split_nocopying.sh
FAIL​: test_scripts/formatting_split_nocopying_split.sh
FAIL​: test_scripts/formatting_split_nocopying_split_dev_null.sh
FAIL​: test_scripts/formatting_simplest_test_prefix_info.sh
FAIL​: test_scripts/formatting_documentlanguage_set_option_info.sh
FAIL​: test_scripts/formatting_simple_with_menu_docbook_info.sh
FAIL​: t/stdout.sh
FAIL​: t/stdout_split.sh
#####

After some digging, I came up with this ack​:

#####
$ ack -A1 '^Modification' ./tp/tests
tp/tests/formatting/out_parser/simplest_test_prefix_info/simplest.2
1​:Modification of a read-only value attempted at ../../tp/Texinfo/Convert/ParagraphNonXS.pm line 327.
2-panic​: POPSTACK

tp/tests/formatting/out_parser/simple_with_menu_docbook_info/simple_with_menu.2
1​:Modification of a read-only value attempted at ../../tp/Texinfo/Convert/ParagraphNonXS.pm line 327.
2-panic​: POPSTACK

tp/tests/formatting/out_parser/cond_info_no-ifhtml_no-ifinfo_no-iftex/cond.2
1​:Modification of a read-only value attempted at ../../tp/Texinfo/Convert/ParagraphNonXS.pm line 327.
2-panic​: POPSTACK

tp/tests/formatting/out_parser/direntry_dircategory_info_split/direntry_dircategory.2
3​:Modification of a read-only value attempted at ../../tp/Texinfo/Convert/ParagraphNonXS.pm line 327.
4-panic​: POPSTACK

tp/tests/formatting/out_parser/split_nocopying_split_dev_null/split_nocopying.2
1​:Modification of a read-only value attempted at ../../tp/Texinfo/Convert/ParagraphNonXS.pm line 327.
2-panic​: POPSTACK

tp/tests/formatting/out_parser/cond_info_ifhtml_ifinfo_iftex/cond.2
1​:Modification of a read-only value attempted at ../../tp/Texinfo/Convert/ParagraphNonXS.pm line 327.
2-panic​: POPSTACK

tp/tests/formatting/out_parser/split_nocopying_split/split_nocopying.2
1​:Modification of a read-only value attempted at ../../tp/Texinfo/Convert/ParagraphNonXS.pm line 327.
2-panic​: POPSTACK

tp/tests/formatting/out_parser/split_nocopying/split_nocopying.2
1​:Modification of a read-only value attempted at ../../tp/Texinfo/Convert/ParagraphNonXS.pm line 327.
2-panic​: POPSTACK

tp/tests/formatting/out_parser/documentlanguage_set_option_info/documentlanguage_set.2
1​:Modification of a read-only value attempted at ../../tp/Texinfo/Convert/ParagraphNonXS.pm line 327.
2-panic​: POPSTACK

tp/tests/formatting/out_parser/cond_info/cond.2
1​:Modification of a read-only value attempted at ../../tp/Texinfo/Convert/ParagraphNonXS.pm line 327.
2-panic​: POPSTACK
#####

This is the same kind of failure reported by mat, but reported from a different location.

Here is the relevant part of texinfo's tp/Texinfo/Convert/ParagraphNonXS.pm

#####
sub add_text($$)
{
  my $paragraph = shift;
  my $text = shift;
  $paragraph->{'end_line_count'} = 0;
  my $result = '';

  my $protect_spaces_flag = $paragraph->{'protect_spaces'};

  ##### below is line 327 #####
  my @​segments = split
/([^\S\x{202f}\x{00a0}]+)|(\p{InFullwidth})|((?​:[^\s\p{InFullwidth}]|[\x{202f}\x{00a0}])+)/,
  $text;
#####

I don't see any attempt to modify a read-only value at that line -- but, again, I know little about texinfo.

Here's the code (excerpted) from tp/Texinfo/Convert/Line.pm that is relevant to mat's original report.

#####
263 sub add_text($$)
264 {
265 my $line = shift;
266 my $text = shift;
267 $line->{'end_line_count'} = 0;
268 my $result = '';
269
270 while ($text ne '') {
271 if ($line->{'DEBUG'}) {
272 my $word = 'UNDEF';
273 $word = $line->{'word'} if (defined($line->{'word'}));
274 print STDERR "s `$line->{'space'}', w `$word'\n";
275 }
276 # \x{202f}\x{00a0} are non breaking spaces
277 if ($text =~ s/^([^\S\x{202f}\x{00a0}\n]+)//) {
...
309 } elsif ($text =~ s/^(([^\s\p{InFullwidth}]|[\x{202f}\x{00a0}])+)//) {
310 my $added_word = $1;
...
#####

In each case there's some heavy-duty regex action going on. Perhaps we've recently introduced some regex changes that are intefering with texinfo's assumptions. Any ideas?

Thank you very much.
--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

From @jkeenan

On Thu Nov 03 12​:28​:06 2016, jkeenan wrote​:

After some digging, I came up with this ack​:

#####
$ ack -A1 '^Modification' ./tp/tests
tp/tests/formatting/out_parser/simplest_test_prefix_info/simplest.2
1​:Modification of a read-only value attempted at
../../tp/Texinfo/Convert/ParagraphNonXS.pm line 327.
2-panic​: POPSTACK

I can now reproduce panics at both the location reported by mat and the location I noted above.

#####
$ cat as_in_paragraphnonxs.pl
my $text = "This is ";
say STDERR "\$text​: <$text>";

my @​segments = split
/([^\S\x{202f}\x{00a0}]+)|(\p{InFullwidth})|((?​:[^\s\p{InFullwidth}]|[\x{202f}\x{00a0}])+)/,
  $text;
#####
$ cat as_in_line.pl
my $text = "This is ";
say STDERR "\$text​: before​: <$text>";

$text =~ s/^(([^\s\p{InFullwidth}]|[\x{202f}\x{00a0}])+)//;
say STDERR "\$text​: after​: <$text>";
#####

Running each against blead​:

#####
./perl -v

This is perl 5, version 25, subversion 7 (v5.25.7 (v5.25.6-172-g3e8592f)) built for x86_64-linux
(with 56 registered patches, see perl -V for more detail)
#####
$ ./perl -Ilib texinfo/as_in_paragraphnonxs.pl
$text​: <This is >
Can't find Unicode property definition "InFullwidth" at /home/jkeenan/learn/perl/p5p/texinfo/as_in_paragraphnonxs.pl line 4.
panic​: POPSTACK
#####
$ ./perl -Ilib texinfo/as_in_line.pl
$text​: before​: <This is >
Can't find Unicode property definition "InFullwidth" at /home/jkeenan/learn/perl/p5p/texinfo/as_in_line.pl line 4.
#####

FBOW, I cannot find "InFullwidth" in the Perl 5 core distribution.

#####
$ ack -i InFullwidth .
$
#####

Hope that helps.

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

From @khwilliamson

On 11/03/2016 02​:00 PM, James E Keenan via RT wrote​:

FBOW, I cannot find "InFullwidth" in the Perl 5 core distribution.

That's because it is a subroutine in Unicode​::EastAsianWidth

[p5pRT]

From @jkeenan

On Thu Nov 03 13​:00​:51 2016, jkeenan wrote​:

On Thu Nov 03 12​:28​:06 2016, jkeenan wrote​:

[snip]

#####
$ cat as_in_paragraphnonxs.pl
my $text = "This is ";
say STDERR "\$text​: <$text>";

my @​segments = split
/([^\S\x{202f}\x{00a0}]+)|(\p{InFullwidth})|((?​:[^\s\p{InFullwidth}]|[\x{202f}\x{00a0}])+)/,
$text;
#####
$ cat as_in_line.pl
my $text = "This is ";
say STDERR "\$text​: before​: <$text>";

$text =~ s/^(([^\s\p{InFullwidth}]|[\x{202f}\x{00a0}])+)//;
say STDERR "\$text​: after​: <$text>";
#####

Running each against blead​:

#####
./perl -v

This is perl 5, version 25, subversion 7 (v5.25.7 (v5.25.6-172-
g3e8592f)) built for x86_64-linux
(with 56 registered patches, see perl -V for more detail)
#####
$ ./perl -Ilib texinfo/as_in_paragraphnonxs.pl
$text​: <This is >
Can't find Unicode property definition "InFullwidth" at
/home/jkeenan/learn/perl/p5p/texinfo/as_in_paragraphnonxs.pl line 4.
panic​: POPSTACK
#####
$ ./perl -Ilib texinfo/as_in_line.pl
$text​: before​: <This is >
Can't find Unicode property definition "InFullwidth" at
/home/jkeenan/learn/perl/p5p/texinfo/as_in_line.pl line 4.
#####

FBOW, I cannot find "InFullwidth" in the Perl 5 core distribution.

#####
$ ack -i InFullwidth .
$
#####

Hope that helps.

A DDG search suggests that 'InFullwidth' is a property settable in these CPAN distributions, neither of which is part of the Perl 5 core distribution.

#####
https://metacpan.org/pod/Unicode::EastAsianWidth
https://metacpan.org/pod/Unicode::Property::XS
#####

And an 'ack' on the texinfo source code quickly shows that it has a dependency on Unicode​::EastAsianWidth​:

#####
$ ack -l 'Unicode(​::|-|\/)(EastAsianWidth)' | wc -l
49

$ ack -l 'Unicode(​::|-|\/)(Property​::XS)' | wc -l
0
#####

If, once I have built blead, installed Unicode​::EastAsianWidth and then required that module in my two little scripts above, the panics go away.

#####
$ ./bin/perl /home/jkeenan/learn/perl/p5p/texinfo/as_in_paragraphnonxs.pl
$text​: <This is >

$ ./bin/perl /home/jkeenan/learn/perl/p5p/texinfo/as_in_line.pl
$text​: before​: <This is >
$text​: after​: < is >
#####

mat​: Could you explore what happens when you try to configure/make/make check texinfo in an environment where Unicode​::EastAsianWidth is already installed against what texinfo's ./configure believes to be $PERL?

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

From mat@cpan.org

mat​: Could you explore what happens when you try to
configure/make/make check texinfo in an environment where
Unicode​::EastAsianWidth is already installed against what texinfo's
./configure believes to be $PERL?

Mmmm, Unicode​::EastAsianWidth is bundled with the texinfo FreeBSD uses, it comes from​:

ftp​://ftp.stack.nl/pub/users/johans/texinfo/20160425/texinfo-6.1.tar.xz

(I don't quite understand why it is not the official tarball, from what I gather, there are some datafiles that are updated)

I will try to see what happens if I have Unicode​::EastAsianWidth present when building texinfo tomorrow.

[p5pRT]

From @jkeenan

On Thu, 03 Nov 2016 23​:06​:13 GMT, mat@​cpan.org wrote​:

mat​: Could you explore what happens when you try to
configure/make/make check texinfo in an environment where
Unicode​::EastAsianWidth is already installed against what texinfo's
./configure believes to be $PERL?

Mmmm, Unicode​::EastAsianWidth is bundled with the texinfo FreeBSD
uses, it comes from​:

ftp​://ftp.stack.nl/pub/users/johans/texinfo/20160425/texinfo-
6.1.tar.xz

(I don't quite understand why it is not the official tarball, from
what I gather, there are some datafiles that are updated)

I will try to see what happens if I have Unicode​::EastAsianWidth
present when building texinfo tomorrow.

mat, thanks for looking into this. texinfo is a *massive* distribution -- over 4200 files in just one of the top-level directories -- and its test suite is not easy to understand. I found that I had to insert 'print' statements for debugging, run 'make check', and then grep the distro for instances of the "Modification ..." failures we've been investigating. I can't reproduce those failures/crashes in simple Perl programs where Unicode​::EastAsianWidth is clearly 'use'd. So the challenge will be to get a reproducible failure that can be run outside of 'make check'.

The only thing I'm fairly certain of now is that this is not FreeBSD-specific.

Since texinfo is maintained upstream of you by GNU, I feel we should alert the GNU maintainers about this problem.

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

From mat@cpan.org

Le Thu, 03 Nov 2016 13​:52​:10 -0700, jkeenan a écrit :

mat​: Could you explore what happens when you try to
configure/make/make check texinfo in an environment where
Unicode​::EastAsianWidth is already installed against what texinfo's
./configure believes to be $PERL?

Friday, RT was down, so I could not comment, but installing Unicode​::EastAsianWidth does not help.
There is only one Perl installed in the testbed, it is the problematic Perl 5.25, and it is /usr/local/bin/perl, and ./configure does find it without problems.

[p5pRT]

From @jkeenan

On Mon, 07 Nov 2016 10​:21​:33 GMT, mat@​cpan.org wrote​:

Le Thu, 03 Nov 2016 13​:52​:10 -0700, jkeenan a écrit :

mat​: Could you explore what happens when you try to
configure/make/make check texinfo in an environment where
Unicode​::EastAsianWidth is already installed against what texinfo's
./configure believes to be $PERL?

Friday, RT was down, so I could not comment, but installing
Unicode​::EastAsianWidth does not help.
There is only one Perl installed in the testbed, it is the problematic
Perl 5.25, and it is /usr/local/bin/perl, and ./configure does find it
without problems.

Agreed. In the hope that others may help with this debugging effort, I am attaching several files​:

* 130010_debugging_procedures.pod, which has step-by-step instructions as to how to reproduce the failures in the texinfo test suite.

* install_branch_for_testing.sh​: Shell script (adapted from rafl and khw) for installing a perl built at the failing commit.

* dollar-tree.txt and dollar-converter.txt​: Referenced in 130010_debugging_procedures.pod

The question is​: What is it about this the pattern​:

#####
/([^\S\x{202f}\x{00a0}]+)|(\p{InFullwidth})|((?​:[^\s\p{InFullwidth}]|[\x{202f}\x{00a0}])+)/
#####

... that (a) as of commit C<a5540cf> but not previously; and (b) in the context of this test suite but not in isolation, perceives something to be a read-only value not subject to modification?

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

From @jkeenan

130010_debugging_procedure.pod