{n}+ possessive quantifier broken in 5.20

[p5pRT]

Migrated from rt.perl.org#125825 (status was 'resolved')

Searchable as RT125825$

[p5pRT]

From rolf.langsdorf@googlemail.com

Created by perl.lanx@gmail.com

Within regexes "{n}+" is like in "(sub-expression){n}+" a so called
"possessive quantifier" to mark sub-expressions atomic and thus
avoid back-tracking.

And according to perlre the following equivalence applies​:

PAT{min,max}+ (?>PAT{min,max})

this is true for 5.14 but not anymore for 5.20

------ Test in 5.20

  DB<2> print "aaa" =~ /(?>(a+))a/

  DB<3> print "aaa" =~ /(a+){1}+a/
aa
  DB<4> print "aaa" =~ /(a+){1}a/
aa
  DB<5> print $]
5.020002

------- Test in 5.14

  DB<1> print "aaa" =~ /(?>(a+))a/

  DB<2> print "aaa" =~ /(a+){1}+a/

  DB<3> print "aaa" =~ /(a+){1}a/
aa
  DB<4> print $]
5.014002

-------

Furthermore from 5.20 on a non-sensical warning is emitted,
telling that the "possessive" quantifier {n}+ may be a redundant use
of "greedy" quantifier.

example​:

Useless use of greediness modifier '+' in regex; marked by <-- HERE in m/\A ((a|
+b){2}){1}+ <-- HERE \z/ at 1345_SoPW.pl line 27, <DATA> line 2.

see http​://www.perlmonks.org/?node_id=1138774 and surrounding thread
for more details and tests, including a "use re 'debug'" output.

cheers
  Rolf

Perl Info

Flags:
    category=core
    severity=critical

Site configuration information for perl 5.20.2:

Configured by lanx at Sun Aug 16 23:36:56 CEST 2015.

Summary of my perl5 (revision 5 version 20 subversion 2) configuration:

  Platform:
    osname=linux, osvers=3.13.0-61-generic, archname=i686-linux
    uname='linux lanx-1005ha 3.13.0-61-generic #100~precise1-ubuntu
smp wed jul 29 12:07:07 utc 2015 i686 i686 i386 gnulinux '
    config_args='-de
-Dprefix=/home/lanx/perl5/perlbrew/perls/perl-5.20.2
-Aeval:scriptdir=/home/lanx/perl5/perlbrew/perls/perl-5.20.2/bin'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fwrapv -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector
-I/usr/local/include'
    ccversion='', gccversion='4.6.3', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib
/usr/lib/gcc/i686-linux-gnu/4.6/include-fixed
/usr/include/i386-linux-gnu /usr/lib /lib/i386-linux-gnu /lib/../lib
/usr/lib/i386-linux-gnu /usr/lib/../lib /lib
    libs=-lnsl -ldl -lm -lcrypt -lutil -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.15.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.15'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib
-fstack-protector'



@INC for perl 5.20.2:
    /home/lanx/perl5/perlbrew/perls/perl-5.20.2/lib/site_perl/5.20.2/i686-linux
    /home/lanx/perl5/perlbrew/perls/perl-5.20.2/lib/site_perl/5.20.2
    /home/lanx/perl5/perlbrew/perls/perl-5.20.2/lib/5.20.2/i686-linux
    /home/lanx/perl5/perlbrew/perls/perl-5.20.2/lib/5.20.2
    .


Environment for perl 5.20.2:
    HOME=/home/lanx
    LANG=de_DE.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/lanx/perl5/perlbrew/bin:/home/lanx/perl5/perlbrew/perls/perl-5.20.2/bin:/home/lanx/perl/bin:/home/lanx/bin:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
    PERLBREW_BASHRC_VERSION=0.71
    PERLBREW_HOME=/home/lanx/.perlbrew
    PERLBREW_MANPATH=/home/lanx/perl5/perlbrew/perls/perl-5.20.2/man
    PERLBREW_PATH=/home/lanx/perl5/perlbrew/bin:/home/lanx/perl5/perlbrew/perls/perl-5.20.2/bin
    PERLBREW_PERL=perl-5.20.2
    PERLBREW_ROOT=/home/lanx/perl5/perlbrew
    PERLBREW_VERSION=0.71
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[p5pRT]

From @iabyn

On Sun, Aug 16, 2015 at 05​:15​:46PM -0700, rolf langsdorf wrote​:

Within regexes "{n}+" is like in "(sub-expression){n}+" a so called
"possessive quantifier" to mark sub-expressions atomic and thus
avoid back-tracking.

And according to perlre the following equivalence applies​:

PAT{min,max}+ (?>PAT{min,max})

this is true for 5.14 but not anymore for 5.20

------ Test in 5.20

DB<2> print "aaa" =~ /(?>(a+))a/

DB<3> print "aaa" =~ /(a+){1}+a/
aa
DB<4> print "aaa" =~ /(a+){1}a/
aa
DB<5> print $]
5.020002

------- Test in 5.14

DB<1> print "aaa" =~ /(?>(a+))a/

DB<2> print "aaa" =~ /(a+){1}+a/

DB<3> print "aaa" =~ /(a+){1}a/
aa
DB<4> print $]
5.014002

-------

Furthermore from 5.20 on a non-sensical warning is emitted,
telling that the "possessive" quantifier {n}+ may be a redundant use
of "greedy" quantifier.

example​:

Useless use of greediness modifier '+' in regex; marked by <-- HERE in m/\A ((a|
+b){2}){1}+ <-- HERE \z/ at 1345_SoPW.pl line 27, <DATA> line 2.

see http​://www.perlmonks.org/?node_id=1138774 and surrounding thread
for more details and tests, including a "use re 'debug'" output.

It bisects to this​:

commit 4fa6dd1
Author​: Karl Williamson <public@​khwilliamson.com>
Date​: Fri Dec 6 15​:50​:52 2013 -0700

  PATCH​: [Perl #42957] Suggesting warning for useless greediness operator
 
  This adds the requested warning. Now we'll see if anything breaks as a
  result.

--
My get-up-and-go just got up and went.

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @khwilliamson

Thanks for reporting this. Now fixed in blead by commit
9a7bb2f
--
Karl Williamson

[p5pRT]

@khwilliamson - Status changed from 'open' to 'pending release'

[p5pRT]

@mauke - Status changed from 'pending release' to 'resolved'