"(P) substitution loop" with long string

[p5pRT]

Migrated from rt.perl.org#103260 (status was 'resolved')

Searchable as RT103260$

[p5pRT]

From @daxim

Created by @daxim

Munging a 1 GiB XML file before parsing.

Following program triggers the grade (P) diagnostic
"Substitution loop at � line 5".

#!/usr/bin/env perl5.14.2
my $xml = qq(<?xml version="1.0" encoding="UTF-8"?>\n<foobar ) . ('a' x
2**30); # any string length >= 1073741818

$xml =~
s{(<[?]xml[^>]+>)}
{$1\n<!DOCTYPE fnord SYSTEM "dummy.dtd" []>}ms;

print substr $xml, 0, 250; # did it work?

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.14.2:

Configured by daxim at Mon Oct 17 14:10:33 CEST 2011.

Summary of my perl5 (revision 5 version 14 subversion 2) configuration:
   
  Platform:
    osname=linux, osvers=2.6.37.6-0.7-desktop, archname=x86_64-linux
    uname='linux notruescotsman 2.6.37.6-0.7-desktop #1 smp preempt
2011-07-21 02:17:24 +0200 x86_64 x86_64 x86_64 gnulinux '
config_args='-de
-Dprefix=/home/daxim/local/share/perlbrew/perls/perl-5.14.2'
hint=recommended, useposix=true, d_sigaction=define useithreads=undef,
usemultiplicity=undef useperlio=define, d_sfio=undef,
uselargefiles=define, usesocks=undef use64bitint=define,
use64bitall=define, uselongdouble=undef usemymalloc=n,
bincompat5005=undef Compiler: cc='cc', ccflags ='-fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64', optimize='-O2', cppflags='-fno-strict-aliasing
-pipe -fstack-protector -I/usr/local/include' ccversion='',
gccversion='4.5.1 20101208 [gcc-4_5-branch revision 167585]',
gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8,
byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define,
longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8,
Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and
Libraries: ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/../lib64 /usr/lib/../lib64 /lib /usr/lib /lib64 /usr/lib64 /usr/local/lib64
    libs=-lnsl -lndbm -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
-lgdbm_compat perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
    libc=/lib/libc-2.11.3.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.14'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib
-fstack-protector'

Locally applied patches:
    


@INC for perl 5.14.2:
    /home/daxim/local/share/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2/x86_64-linux
    /home/daxim/local/share/perlbrew/perls/perl-5.14.2/lib/site_perl/5.14.2
    /home/daxim/local/share/perlbrew/perls/perl-5.14.2/lib/5.14.2/x86_64-linux
    /home/daxim/local/share/perlbrew/perls/perl-5.14.2/lib/5.14.2
    .


Environment for perl 5.14.2:
    HOME=/home/daxim
    LANG=de_DE.UTF-8
    LANGUAGE=
    LD_LIBRARY_PATH=/usr/lib64/mpi/gcc/openmpi/lib64
    LOGDIR (unset)
    PATH=/home/daxim/local/share/perlbrew/bin:/home/daxim/local/share/perlbrew/perls/perl-5.14.2/bin:/home/daxim/local/bin:/usr/lib64/mpi/gcc/openmpi/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/kde3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin
    PERLBREW_HOME=/home/daxim/.perlbrew
    PERLBREW_PATH=/home/daxim/local/share/perlbrew/bin:/home/daxim/local/share/perlbrew/perls/perl-5.14.2/bin
    PERLBREW_PERL=perl-5.14.2
    PERLBREW_ROOT=/home/daxim/local/share/perlbrew
    PERLBREW_VERSION=0.23
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[p5pRT]

From @tonycoz

On Wed, Nov 09, 2011 at 10​:02​:46AM -0800, Lars Dɪá´�á´�á´�á´�á´¡ 迪æ��æ�¯ wrote​:

Munging a 1 GiB XML file before parsing.

Following program triggers the grade (P) diagnostic
"Substitution loop at � line 5".

#!/usr/bin/env perl5.14.2
my $xml = qq(<?xml version="1.0" encoding="UTF-8"?>\n<foobar ) . ('a' x
2**30); # any string length >= 1073741818

$xml =~
s{(<[?]xml[^>]+>)}
{$1\n<!DOCTYPE fnord SYSTEM "dummy.dtd" []>}ms;

print substr $xml, 0, 250; # did it work?

One-liner​:

perl -Ilib -E '$x = "a" . ("b" x 2**30); $x =~ s/([^b]+b)/$1xyz/;'

Changing maxiters in pp_subst and sbu_maxiters in struct subst to
SSize_t fixes this particular case, but I don't think it's a complete
fix, in particular​:

perl -e '$x = "a" . ("b" x 2**31); $x =~ s/b/a/g'

changes from "Substitution loop at -e line 1" to what may be an
infinite loop (or it could just be copying that string 2**31 times).

Tony

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @cpansprout

Fixed in 3c6ef0a.

--

Father Chrysostomos

[p5pRT]

@cpansprout - Status changed from 'open' to 'pending release'

[p5pRT]

From @khwilliamson

Thanks for submitting this ticket

The issue should be resolved with the release today of Perl v5.22. If you find that the problem persists, feel free to reopen this ticket

--
Karl Williamson for the Perl 5 porters team

[p5pRT]

@khwilliamson - Status changed from 'pending release' to 'resolved'