regex not evaluated in constant ?:

[p5pRT]

Migrated from rt.perl.org#20444 (status was 'resolved')

Searchable as RT20444$

[p5pRT]

From perl-5.8.0@ton.iguana.be

Created by perl-5.8.0@ton.iguana.be

Recently i was helping someone on irc who tried to use

signal !~ ( $var eq "a" ? /$test$/ : /^$test/)

and was confused that it didn't do what she expected. Of course in
reality $_ was matched and the result of that matched to $signal.

What however confused the asker most was that

$signal !~ ( 1 ? /$test$/ : /^$test/)

actually matches $test to signal. It confuses me too, but for the
opposite reason. I really don't think this should be executed
as $signal !~ /$test$/ as it is now. I'd expect it to match $_
and then apply the 1 or "" to $signal.

Perl Info

Flags:
    category=core
    severity=low

Site configuration information for perl v5.8.0:

Configured by ton at Tue Nov 12 01:56:18 CET 2002.

Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
  Platform:
    osname=linux, osvers=2.4.19, archname=i686-linux-thread-multi-64int-ld
    uname='linux quasar 2.4.19 #5 wed oct 2 02:34:25 cest 2002 i686 unknown '
    config_args=''
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=define use64bitall=undef uselongdouble=define
    usemymalloc=y, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -fomit-frame-pointer',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -I/usr/local/include'
    ccversion='', gccversion='2.95.3 20010315 (release)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long long', ivsize=8, nvtype='long double', nvsize=12, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lndbm -ldb -ldl -lm -lpthread -lc -lposix -lcrypt -lutil
    perllibs=-lnsl -ldl -lm -lpthread -lc -lposix -lcrypt -lutil
    libc=/lib/libc-2.2.4.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.2.4'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.8.0:
    /usr/lib/perl5/5.8.0/i686-linux-thread-multi-64int-ld
    /usr/lib/perl5/5.8.0
    /usr/lib/perl5/site_perl/5.8.0/i686-linux-thread-multi-64int-ld
    /usr/lib/perl5/site_perl/5.8.0
    /usr/lib/perl5/site_perl
    .


Environment for perl v5.8.0:
    HOME=/home/ton
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/ton/bin.Linux:/home/ton/bin:/home/ton/bin.SampleSetup:/usr/local/bin:/usr/local/sbin:/usr/local/jre/bin:/home/oracle/product/9.0.1/bin:/usr/local/ar/bin:/usr/games/bin:/usr/X11R6/bin:/usr/share/bin:/usr/bin:/usr/sbin:/bin:/sbin:.
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[p5pRT]

From @rgs

perl-5.8.0@​ton.iguana.be (via RT) wrote​:

Recently i was helping someone on irc who tried to use

signal !~ ( $var eq "a" ? /$test$/ : /^$test/)

and was confused that it didn't do what she expected. Of course in
reality $_ was matched and the result of that matched to $signal.

What however confused the asker most was that

$signal !~ ( 1 ? /$test$/ : /^$test/)

actually matches $test to signal. It confuses me too, but for the
opposite reason. I really don't think this should be executed
as $signal !~ /$test$/ as it is now. I'd expect it to match $_
and then apply the 1 or "" to $signal.

I don't understand _what_ is the bug you're reporting.
Here's a test script to make it clear :

use Test​::More tests => 8;
$_ = "quux";
ok( "foo" =~ (1 ? /foo/ : /bar/) );
ok( "bar" !~ (1 ? /foo/ : /bar/) );
ok( "foo" !~ (0 ? /foo/ : /bar/) );
ok( "bar" =~ (0 ? /foo/ : /bar/) );
$p = 1; $n = 0;
ok( "foo" =~ ($p ? /foo/ : /bar/) );
ok( "bar" !~ ($p ? /foo/ : /bar/) );
ok( "foo" !~ ($n ? /foo/ : /bar/) );
ok( "bar" =~ ($n ? /foo/ : /bar/) );

It produces with bleadperl and with 5.8.0 :

1..8
ok 1
ok 2
ok 3
ok 4
not ok 5
# Failed test (foo.pl at line 8)
not ok 6
# Failed test (foo.pl at line 9)
ok 7
ok 8
# Looks like you failed 2 tests of 8.

note that in the first four tests, the conditionals are
constant folded :

$ bleadperl -MO=Deparse,-p test.pl
use Test​::More ('tests', 8);
($_ = 'quux');
ok(('foo' =~ /foo/));
ok((!('bar' =~ /foo/)));
ok((!('foo' =~ /bar/)));
ok(('bar' =~ /bar/));
($p = 1);
($n = 0);
ok(('foo' =~ ($p ? /foo/ : /bar/)));
ok((!('bar' =~ ($p ? /foo/ : /bar/))));
ok((!('foo' =~ ($n ? /foo/ : /bar/))));
ok(('bar' =~ ($n ? /foo/ : /bar/)));
test.pl syntax OK

[p5pRT]

From @ysth

On Mon, 27 Jan 2003 00​:11​:43 +0100, rgarciasuarez@​free.fr wrote​:

I don't understand _what_ is the bug you're reporting.
Here's a test script to make it clear :

use Test​::More tests => 8;
$_ = "quux";
ok( "foo" =~ (1 ? /foo/ : /bar/) );
ok( "bar" !~ (1 ? /foo/ : /bar/) );
ok( "foo" !~ (0 ? /foo/ : /bar/) );
ok( "bar" =~ (0 ? /foo/ : /bar/) );
$p = 1; $n = 0;
ok( "foo" =~ ($p ? /foo/ : /bar/) );
ok( "bar" !~ ($p ? /foo/ : /bar/) );
ok( "foo" !~ ($n ? /foo/ : /bar/) );
ok( "bar" =~ ($n ? /foo/ : /bar/) );

?? In all those cases, whether the /foo/ or /bar/ path is taken, the
result of the ?​: expression should be false, so each test should
result in matching "foo" or "bar" against an empty string, which
should succeed. Why do some of your tests have !~ ?

(N.B., the doc doesn't make it clear to me whether ="" should use the
last successful regex as =// does, but in this case it doesn't matter,
since there should be no successful matches using a nonempty regex.)

[p5pRT]

From perl5-porters@ton.iguana.be

In article <20030127001143.7d5eaec2.rgarciasuarez@​_ree._r>,
  Rafael Garcia-Suarez <rgarciasuarez@​free.fr> writes​:

perl-5.8.0@​ton.iguana.be (via RT) wrote​:

Recently i was helping someone on irc who tried to use

signal !~ ( $var eq "a" ? /$test$/ : /^$test/)

and was confused that it didn't do what she expected. Of course in
reality $_ was matched and the result of that matched to $signal.

What however confused the asker most was that

$signal !~ ( 1 ? /$test$/ : /^$test/)

actually matches $test to signal. It confuses me too, but for the
opposite reason. I really don't think this should be executed
as $signal !~ /$test$/ as it is now. I'd expect it to match $_
and then apply the 1 or "" to $signal.

I don't understand _what_ is the bug you're reporting.
Here's a test script to make it clear :

use Test​::More tests => 8;
$_ = "quux";
ok( "foo" =~ (1 ? /foo/ : /bar/) );
ok( "bar" !~ (1 ? /foo/ : /bar/) );
ok( "foo" !~ (0 ? /foo/ : /bar/) );
ok( "bar" =~ (0 ? /foo/ : /bar/) );
$p = 1; $n = 0;
ok( "foo" =~ ($p ? /foo/ : /bar/) );
ok( "bar" !~ ($p ? /foo/ : /bar/) );
ok( "foo" !~ ($n ? /foo/ : /bar/) );
ok( "bar" =~ ($n ? /foo/ : /bar/) );

It produces with bleadperl and with 5.8.0 :

1..8
ok 1
ok 2
ok 3
ok 4
not ok 5
# Failed test (foo.pl at line 8)
not ok 6
# Failed test (foo.pl at line 9)
ok 7
ok 8
# Looks like you failed 2 tests of 8.

note that in the first four tests, the conditionals are
constant folded :

(I already answered this in a direct mail, but I didn't see it appear
on p5p or the bug database)

It's the way the constant folding works that I'm reporting as a bug.

$_="foo";
print "foo" =~ (1 ? /foo/ : /bar/)

(prints 1, it matches the constant "foo" to /foo/, $_ isn't involved)
and

$p=1;
$_="foo";
print "foo" =~ ($p ? /foo/ : /bar/)

(prints nothing, $_ is matched with /foo/ which gives "1", which does not match "foo")

$_="foo";
print "1" =~ (1 ? /foo/ : /bar/)

(prints nothing, "1" doesn't match "foo" (again $_ is not involved at all)

and

$p=1;
$_="foo";
print 1 =~ ($p ? /foo/ : /bar/)

(prints 1, /foo/ matches $_, giving "1" which in turn matches the 1

I don't think these should differ. And I think the ones using $p
are right. Constant folding should be an optimization, not change
the meaning of what you write.

The tests you gave in the example are not very relevant since if they
match "quux" to /foo/ and /bar/, you get "", which will when it gets
matched next always work out, so you don't see the effect.

More interesting would be​:

"a"=/a/;
$_ = "quux";
$p=1;
print "foo" = ($p ? /foo/ : /bar/);

Here the empty match is first set up to mean "match a". Now it does NOT
match anymore. If you leave out the "a"=~/a/ it WILL match, and at first
sight look as if "foo" got matched against /foo/ (which it did not).

[p5pRT]

From me-02@ton.iguana.be

On Sun, Jan 26, 2003 at 11​:06​:13PM -0000, Rafael Garcia-Suarez wrote​:

perl-5.8.0@​ton.iguana.be (via RT) wrote​:

Recently i was helping someone on irc who tried to use

signal !~ ( $var eq "a" ? /$test$/ : /^$test/)

and was confused that it didn't do what she expected. Of course in
reality $_ was matched and the result of that matched to $signal.

What however confused the asker most was that

$signal !~ ( 1 ? /$test$/ : /^$test/)

actually matches $test to signal. It confuses me too, but for the
opposite reason. I really don't think this should be executed
as $signal !~ /$test$/ as it is now. I'd expect it to match $_
and then apply the 1 or "" to $signal.

I don't understand _what_ is the bug you're reporting.
Here's a test script to make it clear :

use Test​::More tests => 8;
$_ = "quux";
ok( "foo" =~ (1 ? /foo/ : /bar/) );
ok( "bar" !~ (1 ? /foo/ : /bar/) );
ok( "foo" !~ (0 ? /foo/ : /bar/) );
ok( "bar" =~ (0 ? /foo/ : /bar/) );
$p = 1; $n = 0;
ok( "foo" =~ ($p ? /foo/ : /bar/) );
ok( "bar" !~ ($p ? /foo/ : /bar/) );
ok( "foo" !~ ($n ? /foo/ : /bar/) );
ok( "bar" =~ ($n ? /foo/ : /bar/) );

It produces with bleadperl and with 5.8.0 :

1..8
ok 1
ok 2
ok 3
ok 4
not ok 5
# Failed test (foo.pl at line 8)
not ok 6
# Failed test (foo.pl at line 9)
ok 7
ok 8
# Looks like you failed 2 tests of 8.

note that in the first four tests, the conditionals are
constant folded :

It's the constant folding that I'm reporting as a bug.

$_="foo";
print "foo" =~ (1 ? /foo/ : /bar/)

(prints 1, it matches the constant "foo" to /foo/, $_ isn't involved)
and

$p=1;
$_="foo";
print "foo" =~ ($p ? /foo/ : /bar/)

(prints nothing, $_ is matched with /foo/ which gives "1", which does not match "foo")

$_="foo";
print "1" =~ (1 ? /foo/ : /bar/)

(prints nothing, "1" doesn't match "foo" (again $_ is not involved at all)

and

$p=1;
$_="foo";
print 1 =~ ($p ? /foo/ : /bar/)

(prints 1, /foo/ matches $_, giving "1" which in turn matches the 1

I don't think these should differ. And I think the ones using $p
are right. Constant folding should be an optimization, not change
the meaning of what you write.

The tests you gave in the example are not very relevant since if they
match "quux" to /foo/ and /bar/, you get "", which will when it gets
matched next always work out, so you don't see the effect.

More interesting would be​:

"a"=/a/;
$_ = "quux";
$p=1;
print "foo" = ($p ? /foo/ : /bar/);

Here the empty match is first set up to mean "match a". Now it does NOT
match anymore. If you leave out the "a"=~/a/ it WILL match, and at first
sight look as if "foo" got matched against /foo/ (which it did not).

[p5pRT]

From @cpansprout

This patch solves the problem by marking match and subst ops as OPf_SPECIAL during constant folding, so the =~ operator can tell not to take possession of it.

[p5pRT]

From @cpansprout

Inline Patch

diff -Nup blead/op.c blead-20444-re-const-flodding/op.c
--- blead/op.c	2010-07-25 10:28:10.000000000 -0700
+++ blead-20444-re-const-flodding/op.c	2010-08-01 11:15:56.000000000 -0700
@@ -2242,9 +2242,10 @@ Perl_bind_match(pTHX_ I32 type, OP *left
 	type == OP_NOT)
 	yyerror("Using !~ with s///r doesn't make sense");
 
-    ismatchop = rtype == OP_MATCH ||
-		rtype == OP_SUBST ||
-		rtype == OP_TRANS;
+    ismatchop = (rtype == OP_MATCH ||
+		 rtype == OP_SUBST ||
+		 rtype == OP_TRANS)
+	     && !(right->op_flags & OPf_SPECIAL);
     if (ismatchop && right->op_private & OPpTARGET_MY) {
 	right->op_targ = 0;
 	right->op_private &= ~OPpTARGET_MY;
@@ -4689,6 +4690,11 @@ S_new_logop(pTHX_ I32 type, I32 flags, O
 	    op_free(first);
 	    if (other->op_type == OP_LEAVE)
 		other = newUNOP(OP_NULL, OPf_SPECIAL, other);
+	    else if (other->op_type == OP_MATCH
+	          || other->op_type == OP_SUBST
+	          || other->op_type == OP_TRANS)
+		/* Mark the op as being unbindable with =~ */
+		other->op_flags |= OPf_SPECIAL;
 	    return other;
 	}
 	else {
@@ -4827,6 +4833,10 @@ Perl_newCONDOP(pTHX_ I32 flags, OP *firs
 	}
 	if (live->op_type == OP_LEAVE)
 	    live = newUNOP(OP_NULL, OPf_SPECIAL, live);
+	else if (live->op_type == OP_MATCH || live->op_type == OP_SUBST
+	      || live->op_type == OP_TRANS)
+	    /* Mark the op as being unbindable with =~ */
+	    live->op_flags |= OPf_SPECIAL;
 	return live;
     }
     NewOp(1101, logop, 1, LOGOP);
diff -Nup blead/op.h blead-20444-re-const-flodding/op.h
--- blead/op.h	2010-07-29 03:17:10.000000000 -0700
+++ blead-20444-re-const-flodding/op.h	2010-08-01 07:06:45.000000000 -0700
@@ -142,6 +142,10 @@ Deprecated.  Use C<GIMME_V> instead.
 				/*  On OP_HELEM and OP_HSLICE, localization will be followed
 				    by assignment, so do not wipe the target if it is special
 				    (e.g. a glob or a magic SV) */
+				/*  On OP_MATCH, OP_SUBST & OP_TRANS, the
+				    operand of a logical or conditional
+				    that was optimised away, so it should
+				    not be bound via =~ */
 
 /* old names; don't use in new code, but don't break them, either */
 #define OPf_LIST	OPf_WANT_LIST
diff -Nurp blead/t/comp/fold.t blead-20444-re-const-flodding/t/comp/fold.t
--- blead/t/comp/fold.t	2009-11-19 08:51:40.000000000 -0800
+++ blead-20444-re-const-flodding/t/comp/fold.t	2010-08-01 11:15:02.000000000 -0700
@@ -4,7 +4,7 @@
 # we've not yet verified that use works.
 # use strict;
 
-print "1..13\n";
+print "1..19\n";
 my $test = 0;
 
 # Historically constant folding was performed by evaluating the ops, and if
@@ -52,6 +52,16 @@ sub is {
     failed($got, "'$expect'", $name);
 }
 
+sub ok {
+    my ($got, $name) = @_;
+    $test = $test + 1;
+    if ($got) {
+	print "ok $test - $name\n";
+	return 1;
+    }
+    failed($got, "a true value", $name);
+}
+
 my $a;
 $a = eval '$b = 0/0 if 0; 3';
 is ($a, 3, 'constants in conditionals don\'t affect constant folding');
@@ -88,3 +98,23 @@ is ($@, '', 'no error');
     like ($@, qr/division/, "eval caught division");
     is($c, 2, "missing die hook");
 }
+
+# [perl #20444] Constant folding should not change the meaning of match
+# operators.
+{
+ local *_;
+ $_="foo"; my $jing = 1;
+ ok scalar $jing =~ (1 ? /foo/ : /bar/),
+   'lone m// is not bound via =~ after ? : folding';
+ ok scalar $jing =~ (0 || /foo/),
+   'lone m// is not bound via =~ after || folding';
+ ok scalar $jing =~ (1 ? s/foo/foo/ : /bar/),
+   'lone s/// is not bound via =~ after ? : folding';
+ ok scalar $jing =~ (0 || s/foo/foo/),
+   'lone s/// is not bound via =~ after || folding';
+ $jing = 3;
+ ok scalar $jing =~ (1 ? y/fo// : /bar/),
+   'lone y/// is not bound via =~ after ? : folding';
+ ok scalar $jing =~ (0 || y/fo//),
+   'lone y/// is not bound via =~ after || folding';
+}

[p5pRT]

From @cpansprout

On Aug 1, 2010, at 12​:21 PM, Father Chrysostomos wrote​:

This patch solves the problem by marking match and subst ops as OPf_SPECIAL during constant folding, so the =~ operator can tell not to take possession of it.

With that patch​:

$ ./perl -Ilib -MO=Deparse -e'"foo" =~ (1?/foo/​:/bar/)'
'foo' =~ /foo/;
-e syntax OK

So the Deparse output no longer matches what perl does. With the patch attached to this message applied after that one​:

$ ./perl -Ilib -MO=Deparse -e'"foo" =~ (1?/foo/​:/bar/)'
'foo' =~ ($_ =~ /foo/);
-e syntax OK

[p5pRT]

From @cpansprout

Inline Patch

diff -Nurp blead-20444-re-const-flodding/dist/B-Deparse/Deparse.pm blead-20444-re-const-flodding-copy/dist/B-Deparse/Deparse.pm
--- blead-20444-re-const-flodding/dist/B-Deparse/Deparse.pm	2010-06-21 14:31:10.000000000 -0700
+++ blead-20444-re-const-flodding-copy/dist/B-Deparse/Deparse.pm	2010-08-01 18:25:57.000000000 -0700
@@ -4221,6 +4221,7 @@ sub matchop {
     }
     my $quote = 1;
     my $extended = ($op->pmflags & PMf_EXTENDED);
+    my $rhs_bound_to_defsv;
     if (null $kid) {
 	my $unbacked = re_unback($op->precomp);
 	if ($extended) {
@@ -4232,6 +4233,7 @@ sub matchop {
 	carp("found ".$kid->name." where regcomp expected");
     } else {
 	($re, $quote) = $self->regcomp($kid, 21, $extended);
+	$rhs_bound_to_defsv = 1 if $kid->first->first->flags & OPf_SPECIAL;
     }
     my $flags = "";
     $flags .= "c" if $op->pmflags & PMf_CONTINUE;
@@ -4250,7 +4252,13 @@ sub matchop {
     }
     $re = $re . $flags if $quote;
     if ($binop) {
-	return $self->maybe_parens("$var =~ $re", $cx, 20);
+	return
+	 $self->maybe_parens(
+	  $rhs_bound_to_defsv
+	   ? "$var =~ (\$_ =~ $re)"
+	   : "$var =~ $re",
+	  $cx, 20
+	 );
     } else {
 	return $re;
     }
diff -Nurp blead-20444-re-const-flodding/dist/B-Deparse/t/deparse.t blead-20444-re-const-flodding-copy/dist/B-Deparse/t/deparse.t
--- blead-20444-re-const-flodding/dist/B-Deparse/t/deparse.t	2010-05-03 14:22:11.000000000 -0700
+++ blead-20444-re-const-flodding-copy/dist/B-Deparse/t/deparse.t	2010-08-01 18:26:56.000000000 -0700
@@ -17,7 +17,7 @@ BEGIN {
     require feature;
     feature->import(':5.10');
 }
-use Test::More tests => 89;
+use Test::More tests => 90;
 use Config ();
 
 use B::Deparse;
@@ -645,3 +645,12 @@ pop;
 pop();
 ####
 pop @_;
+####
+# 82 [perl #20444]
+"foo" =~ (1 ? /foo/ : /bar/);
+"foo" =~ (1 ? y/foo// : /bar/);
+"foo" =~ (1 ? s/foo// : /bar/);
+>>>>
+'foo' =~ ($_ =~ /foo/);
+'foo' =~ ($_ =~ tr/fo//);
+'foo' =~ ($_ =~ s/foo//);

[p5pRT]

From @cpansprout

On Sun Aug 01 20​:12​:28 2010, sprout wrote​:

On Aug 1, 2010, at 12​:21 PM, Father Chrysostomos wrote​:

This patch solves the problem by marking match and subst ops as
OPf_SPECIAL during constant folding, so the =~ operator can tell not
to take possession of it.

With that patch​:

$ ./perl -Ilib -MO=Deparse -e'"foo" =~ (1?/foo/​:/bar/)'
'foo' =~ /foo/;
-e syntax OK

So the Deparse output no longer matches what perl does. With the patch
attached to this message applied after that one​:

$ ./perl -Ilib -MO=Deparse -e'"foo" =~ (1?/foo/​:/bar/)'
'foo' =~ ($_ =~ /foo/);
-e syntax OK

The two patches have been applied as
2474a78 and
a539498, respectively.

[p5pRT]

@cpansprout - Status changed from 'open' to 'resolved'