Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

regex optimizer bug in qr//flags #8457

Closed
p5pRT opened this issue May 22, 2006 · 5 comments
Closed

regex optimizer bug in qr//flags #8457

p5pRT opened this issue May 22, 2006 · 5 comments

Comments

@p5pRT
Copy link
Collaborator

@p5pRT p5pRT commented May 22, 2006

Migrated from rt.perl.org#39185 (status was 'resolved')

Searchable as RT39185$

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented May 22, 2006

From johnpc@xs4all.nl

Created by johnpc@xs4all.nl

When using flags on the qr// regex-creation operator, some regexes
are badly optimized and thus fail to match. This example illustrates
the problem (problem originally presented by TTY as "buzz8" during the
Dutch Perl Workshop).

#!/usr/bin/env perl
use strict;
use warnings;

my $test = <<'END';
{ bla }
{ bla bla }

END

my $re = qr/^\{(.*?)\}$/m;
$test =~ s/$re/<$1>/g; ### match rejected by optimizer
### workarounds are​:
# $re = qr/(?m​:^\{(.*?)\}$)/; ### OK
# however​: $re = qr/(?m​:^\{(.*?)\}$)/m; ### FAILS too???
# chomp $test; ### OK (now $test =~ /}\n\z/)
# $test .= "}" ### OK (now $test =~ /}\z/)
# $test =~ s/^\{(.*?)\}$/<$1>/gm; ### OK
# $test =~ s/$re/<$1>/gm; ### OK

print $test; ### expect​: "< bla >\n< bla bla >\n\n"
__END__

The problem can be witnessed if this is run with -Mre=debug. The
regex is rejected by the optimizer, but any of the suggested changes
(marked "### OK") give the expected result, and make the regex match.

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl v5.8.8:

Configured by Debian Project at Tue Apr  4 22:34:25 UTC 2006.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration:
  Platform:
    osname=linux, osvers=2.6.15.4, archname=i486-linux-gnu-thread-multi
    uname='linux ninsei 2.6.15.4 #1 smp preempt mon feb 20 09:48:53 pst 2006 i686 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i486-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.8.8 -Dsitearch=/usr/local/lib/perl/5.8.8 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dlibperl=libperl.so.5.8.8 -Dd_dosuid -des'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='4.0.3 (Debian 4.0.3-1)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.3.6.so, so=so, useshrplib=true, libperl=libperl.so.5.8.8
    gnulibc_version='2.3.6'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.8:
    /etc/perl
    /usr/local/lib/perl/5.8.8
    /usr/local/share/perl/5.8.8
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.8
    /usr/share/perl/5.8
    /usr/local/lib/site_perl
    /usr/local/lib/perl/5.8.4
    /usr/local/share/perl/5.8.4
    /usr/local/lib/perl/5.8.3
    /usr/local/share/perl/5.8.3
    /usr/local/lib/perl/5.8.0
    /usr/local/share/perl/5.8.0
    .


Environment for perl v5.8.8:
    HOME=/home/johnpc
    LANG (unset)
    LANGUAGE (unset)
    LC_CTYPE=en_US.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/johnpc/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin:/usr/games:/usr/X11R6/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented May 22, 2006

From @iabyn

On Mon, May 22, 2006 at 02​:26​:40PM -0700, johnpc @​ xs4all. nl wrote​:

When using flags on the qr// regex-creation operator, some regexes
are badly optimized and thus fail to match. This example illustrates
the problem (problem originally presented by TTY as "buzz8" during the
Dutch Perl Workshop).

#!/usr/bin/env perl
use strict;
use warnings;

my $test = <<'END';
{ bla }
{ bla bla }

END

my $re = qr/^\{(.*?)\}$/m;
$test =~ s/$re/<$1>/g; ### match rejected by optimizer
### workarounds are​:
# $re = qr/(?m​:^\{(.*?)\}$)/; ### OK
# however​: $re = qr/(?m​:^\{(.*?)\}$)/m; ### FAILS too???
# chomp $test; ### OK (now $test =~ /}\n\z/)
# $test .= "}" ### OK (now $test =~ /}\z/)
# $test =~ s/^\{(.*?)\}$/<$1>/gm; ### OK
# $test =~ s/$re/<$1>/gm; ### OK

print $test; ### expect​: "< bla >\n< bla bla >\n\n"
__END__

The problem can be witnessed if this is run with -Mre=debug. The
regex is rejected by the optimizer, but any of the suggested changes
(marked "### OK") give the expected result, and make the regex match.

This appears to have been fixed between perl 5.9.1 and 5.9.2, but the fix
presumably hasn't been backported to 5.8.x.

--
This email is confidential, and now that you have read it you are legally
obliged to shoot yourself. Or shoot a lawyer, if you prefer. If you have
received this email in error, place it in its original wrapping and return
for a full refund. By opening this email, you accept that Elvis lives.

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented May 22, 2006

The RT System itself - Status changed from 'new' to 'open'

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented May 23, 2006

From rick@bort.ca

On Tue, May 23, 2006 at 12​:07​:59AM +0100, Dave Mitchell wrote​:

On Mon, May 22, 2006 at 02​:26​:40PM -0700, johnpc @​ xs4all. nl wrote​:

When using flags on the qr// regex-creation operator, some regexes
are badly optimized and thus fail to match. This example illustrates
the problem (problem originally presented by TTY as "buzz8" during the
Dutch Perl Workshop).

#!/usr/bin/env perl
use strict;
use warnings;

my $test = <<'END';
{ bla }
{ bla bla }

END

my $re = qr/^\{(.*?)\}$/m;
$test =~ s/$re/<$1>/g; ### match rejected by optimizer
### workarounds are​:
# $re = qr/(?m​:^\{(.*?)\}$)/; ### OK
# however​: $re = qr/(?m​:^\{(.*?)\}$)/m; ### FAILS too???
# chomp $test; ### OK (now $test =~ /}\n\z/)
# $test .= "}" ### OK (now $test =~ /}\z/)
# $test =~ s/^\{(.*?)\}$/<$1>/gm; ### OK
# $test =~ s/$re/<$1>/gm; ### OK

print $test; ### expect​: "< bla >\n< bla bla >\n\n"
__END__

The problem can be witnessed if this is run with -Mre=debug. The
regex is rejected by the optimizer, but any of the suggested changes
(marked "### OK") give the expected result, and make the regex match.

This appears to have been fixed between perl 5.9.1 and 5.9.2, but the fix
presumably hasn't been backported to 5.8.x.

I believe it has been backported; it just didn't make it into 5.8.8. At
least, it works in my copy of 5.8.x sync'ed today. Probably fixed by
change 27604.

--
Rick Delaney
rick@​bort.ca

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented May 23, 2006

@rgs - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.