Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter::Util::Call problem with $_ #9216

Closed
p5pRT opened this issue Jan 31, 2008 · 11 comments
Closed

Filter::Util::Call problem with $_ #9216

p5pRT opened this issue Jan 31, 2008 · 11 comments

Comments

@p5pRT
Copy link
Collaborator

@p5pRT p5pRT commented Jan 31, 2008

Migrated from rt.perl.org#50430 (status was 'resolved')

Searchable as RT50430$

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jan 31, 2008

From ambrus@math.bme.hu

Created by ambrus@math.bme.hu

This is a strange problem with Filter​::Util​::Call. Here is a perl script
that installs a source filter that doesn't transform the code at all, then
follows a simple hello world using that filter.

use Filter​::Util​::Call;
BEGIN{ filter_add(sub{
  my $s = filter_read;
  my $t = $_;
  $_ = $t; # <---
  $s;
}); }
print "hello, world\n";

The above code works as expected​: it prints hello world.

However, if I modify it by deleting the line with the arrow, I get this​:

use Filter​::Util​::Call;
BEGIN{ filter_add(sub{
  my $s = filter_read;
  my $t = $_;
  $s;
}); }
print "hello, world\n";

This code does not work. It exits with 0 exit code without printing
anything. I can't see why they should be different, the line I deleted
shouldn't change the way the filter transforms the code.

I could reproduce this bug on both perl 5.10.0 linux-x86 and perl 5.8.8
linux-x86, both compiled from vanilla source.

I've first posted the bug at "http​://www.perlmonks.com/?node_id=664944" --
see the replies there.

Thanks and keep up the good work,

Ambrus

Perl Info

Flags:
    category=library
    severity=medium

Site configuration information for perl v5.8.8:

Configured by Debian Project at Sun Nov  4 15:45:42 UTC 2007.

Summary of my perl5 (revision 5 version 8 subversion 8) configuration:
  Platform:
    osname=linux, osvers=2.6.22-3-amd64, archname=x86_64-linux-gnu-thread-multi
    uname='linux deneb 2.6.22-3-amd64 #1 smp thu oct 11 15:23:23 utc
2007 x86_64 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN
-Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr
-Dprivlib=/usr/share/perl/5.8 -Darchlib=/usr/lib/perl/5.8
-Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5
-Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local
-Dsitelib=/usr/local/share/perl/5.8.8
-Dsitearch=/usr/local/lib/perl/5.8.8 -Dman1dir=/usr/share/man/man1
-Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1
-Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl
-Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Uusenm
-Duseshrplib -Dlibperl=libperl.so.5.8.8 -Dd_dosuid -des'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=define use5005threads=undef useithreads=define
usemultiplicity=define
    useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
    use64bitint=define use64bitall=define uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS
-DDEBIAN -fno-strict-aliasing -pipe -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN
-fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion='', gccversion='4.1.2 20061115 (prerelease) (Debian
4.1.1-21)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.3.6.so, so=so, useshrplib=true, libperl=libperl.so.5.8.8
    gnulibc_version='2.3.6'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:



@INC for perl v5.8.8:
    /etc/perl
    /usr/local/lib/perl/5.8.8
    /usr/local/share/perl/5.8.8
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.8
    /usr/share/perl/5.8
    /usr/local/lib/site_perl
    .


Environment for perl v5.8.8:
    HOME=/home/student/ambrus
    LANG=C
    LANGUAGE (unset)
    LC_CTYPE=hu_HU
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/student/ambrus/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/games
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jan 31, 2008

From @steve-m-hay

It is because the anonymous filter sub is supposed to return the
filtered source via $_, which will be empty unless you set it. (The
filter sub has a local copy of $_ which is always empty initially.)

The return value of the filter sub is simply a success/failure indicator
(< 0 for error, 0 for EOF, > 0 for OK).

This is all documented in the Filter​::Util​::Call manpage​:

http​://search.cpan.org/~pmqs/Filter-1.34/Call/Call.pm#filter()_and_anonymous_sub

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Jan 31, 2008

@steve-m-hay - Status changed from 'new' to 'rejected'

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 6, 2008

From ambrus@math.bme.hu

(note​: this is a dupe of a message I sent in email to
"perlbug-followup\x40perl.org"​: as that mail doesn't seem to appear in
the web interface rt.perl.org I think I'd messed up something and repost
it using the web interface. sorry if you get this in dupe.)

On Thu Jan 31 08​:33​:12 2008, shay wrote​:

It is because the anonymous filter sub is supposed to return the
filtered source via $_, which will be empty unless you set it. (The
filter sub has a local copy of $_ which is always empty initially.)

The return value of the filter sub is simply a success/failure
indicator
(< 0 for error, 0 for EOF, > 0 for OK).

This is all documented in the Filter​::Util​::Call manpage​:

http​://search.cpan.org/~pmqs/Filter-
1.34/Call/Call.pm#filter()_and_anonymous_sub

Dear shay,

I'm sorry but I don't think your argument stands, and must insist on
my bug report. Please look at the source again.

1 use Filter​::Util​::Call;
2 BEGIN{ filter_add(sub{
3 my $s = filter_read;
4 my $t = $_;
5 $_ = $t; # <---
6 $s;
7 }); }
8 print "hello, world\n";

The variable $_ is filled with the original source by the filter_read
function, just like it's documented. If you remove both line 4 and
line 5, the code works and acts like a trivial source filter. Line 3
reads an original source line to $_ and saves the status to $s, and
line 6 returns this status from the filter sub, just like all the
simple examples in the Filter​::Util​::Call manpage. Thus, on exit from
the filter sub, $_ contains the original source line unchanged so this
should act as a source filter. Indeed, if you run this code you see
that it prints hello world because line 8 is not changed by the
filter.

The first snippet of the code in my bug report (with both line 4 and
line 5 included) doesn't seem much different from the above​: $_ gets
set to the source line by line 3, and line 5 copies the same value it
already have to $_. Indeed, this snippet appears to work just

The second code snippet in the bug report differs from this only in
that line 5 is omitted. Without that line, $_ should still contain
the source line that line 3 put there, and line 4 should not change
this in any way. Thus, I would think that this snippet should work
all the same. However, if I run it with perl 5.10.0 or 5.8.8, it
turns out that it doesn't print anything.

Finally, please note that BrowserUK has a suspicion on what causes
this bug at "http​://www.perlmonks.com/?node_id=664998"; and tye says
he believes it might be a reference count problem. I don't understand
the XS code so I can't say anything about these.

Thanks for your patience,

Ambrus

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 6, 2008

ambrus@math.bme.hu - Status changed from 'rejected' to 'new'

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 6, 2008

From ambrus@math.bme.hu

On ticket #50430, shay comments at Thu Jan 31 08​:33​:12 2008​:

It is because the anonymous filter sub is supposed to return the
filtered source via $_, which will be empty unless you set it. (The
filter sub has a local copy of $_ which is always empty initially.)

The return value of the filter sub is simply a success/failure indicator
(< 0 for error, 0 for EOF, > 0 for OK).
This is all documented in the Filter​::Util​::Call manpage​:
http​://search.cpan.org/~pmqs/Filter-1.34/Call/Call.pm#filter()_and_anonymous_sub

Dear shay,

I'm sorry but I don't think your argument stands, and must insist on
my bug report. Please look at the source again.

1 use Filter​::Util​::Call;
2 BEGIN{ filter_add(sub{
3 my $s = filter_read;
4 my $t = $_;
5 $_ = $t; # <---
6 $s;
7 }); }
8 print "hello, world\n";

The variable $_ is filled with the original source by the filter_read
function, just like it's documented. If you remove both line 4 and
line 5, the code works and acts like a trivial source filter. Line 3
reads an original source line to $_ and saves the status to $s, and
line 6 returns this status from the filter sub, just like all the
simple examples in the Filter​::Util​::Call manpage. Thus, on exit from
the filter sub, $_ contains the original source line unchanged so this
should act as a source filter. Indeed, if you run this code you see
that it prints hello world because line 8 is not changed by the
filter.

The first snippet of the code in my bug report (with both line 4 and
line 5 included) doesn't seem much different from the above​: $_ gets
set to the source line by line 3, and line 5 copies the same value it
already have to $_. Indeed, this snippet appears to work just

The second code snippet in the bug report differs from this only in
that line 5 is omitted. Without that line, $_ should still contain
the source line that line 3 put there, and line 4 should not change
this in any way. Thus, I would think that this snippet should work
all the same. However, if I run it with perl 5.10.0 or 5.8.8, it
turns out that it doesn't print anything.

Finally, please note that BrowserUK has a suspicion on what causes
this bug at "http​://www.perlmonks.com/?node_id=664998"; and tye says
he believes it might be a reference count problem. I don't understand
the XS code so I can't say anything about these.

Thanks for your patience,

Ambrus

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 6, 2008

From @steve-m-hay

Zsbán Ambrus wrote​:

On ticket #50430, shay comments at Thu Jan 31 08​:33​:12 2008​:

It is because the anonymous filter sub is supposed to return the
filtered source via $_, which will be empty unless you set it. (The
filter sub has a local copy of $_ which is always empty initially.)

The return value of the filter sub is simply a success/failure
indicator (< 0 for error, 0 for EOF, > 0 for OK).
This is all documented in the Filter​::Util​::Call manpage​:
http​://search.cpan.org/~pmqs/Filter-1.34/Call/Call.pm#filter()_and_anonymous_sub

Dear shay,

I'm sorry but I don't think your argument stands, and must insist on
my bug report. Please look at the source again.

1 use Filter​::Util​::Call;
2 BEGIN{ filter_add(sub{
3 my $s = filter_read;
4 my $t = $_;
5 $_ = $t; # <---
6 $s;
7 }); }
8 print "hello, world\n";

The variable $_ is filled with the original source by the filter_read
function, just like it's documented.

You're quite correct. My apologies for being too hasty before. I'd missed the fact that filter_read sets $_ for you, despite it also being clearly documented.

Finally, please note that BrowserUK has a suspicion on what causes
this bug at "http​://www.perlmonks.com/?node_id=664998"; and tye says
he believes it might be a reference count problem. I don't understand
the XS code so I can't say anything about these.

I had a quick look and I wonder if it is to do with the "local" $_ that is created prior to the perl_call_sv() being made mortal. The debugger shows DEFSV is cleared by the time perl_call_sv() returns, so perhaps a FREETMPS has happened somewhere? (Can ENTER+SAVETMPS / FREETMPS+LEAVE blocks be nested? I've always been a little uneasy with this stuff, and am currently rather rusty too.)

Anyway, the attached patch moves the mortalization of the local $_ until just before the FREETMPS, well after the perl_call_sv(), and fixes the bug without breaking any tests.

Would somebody with more knowledge of these matters care to comment?

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 6, 2008

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 6, 2008

The RT System itself - Status changed from 'new' to 'open'

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 21, 2008

From @steve-m-hay

My previous patch is now applied to bleadperl in change 33341.

@p5pRT

This comment has been minimized.

Copy link
Collaborator Author

@p5pRT p5pRT commented Feb 21, 2008

@steve-m-hay - Status changed from 'open' to 'resolved'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.