Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

binmode clobbers $@ under certain circumstances #10800

Closed
p5pRT opened this issue Nov 5, 2010 · 10 comments
Closed

binmode clobbers $@ under certain circumstances #10800

p5pRT opened this issue Nov 5, 2010 · 10 comments

Comments

@p5pRT
Copy link
Collaborator

@p5pRT p5pRT commented Nov 5, 2010

Migrated from rt.perl.org#78844 (status was 'resolved')

Searchable as RT78844$

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Nov 5, 2010

From muir@arubanetworks.com

Created by muir@arubanetworks.com

This is a bug report for perl from muir@​arubanetworks.com,
generated with the help of perlbug 1.39 running under perl 5.10.1.

-----------------------------------------------------------------

binmode() can clobber $@​ under certain circumstances. This reproduces it​:

$@​ = 'foo';

open(my $fh, '>/tmp/foo');
binmode($fh, '​:encoding(UTF-8)');

warn $@​;

Note that setting the binmode to '​:raw' does not clobber $@​.

If this turns out to not be a bug for some reason, the perldoc for binmode
should mention this behavior.

In case you are curious, the first symptom I saw was die() being unusually
silent (but still exiting the program). We had an object with a DESTROY
method hitting binmode in this way, which of course clobbered the exception
after it was thrown but before it was output to STDERR.

Perl Info

Flags:
    category=core
    severity=low

Site configuration information for perl 5.10.1:

Configured by Red Hat, Inc. at Tue Jun 29 15:55:54 PDT 2010.

Summary of my perl5 (revision 5 version 10 subversion 1) configuration:
   
  Platform:
    osname=linux, osvers=2.6.18-194.3.1.el5, archname=x86_64-linux-thread-multi
    uname='linux kilo.corp.airwave.com 2.6.18-194.3.1.el5 #1 smp thu may 13 13:08:30 edt 2010 x86_64 x86_64 x86_64 gnulinux '
    config_args='-des -Doptimize=-O2 -g -m64 -mtune=generic -Dversion=5.10.1 -Dmyhostname=localhost -Dperladmin=root@localhost -Dcc=gcc -Dman1dir=/opt/airwave/share/man/man1 -Dman3dir=/opt/airwave/share/man/man3 -Dcf_by=Red Hat, Inc. -Dinstallprefix=/opt/airwave -Dprefix=/opt/airwave -Dlibpth=/usr/lib64 /lib64 /opt/airwave/lib64 /usr/lib /lib /opt/airwave/lib -Dprivlib=/opt/airwave/lib/perl5/5.10.1 -Dsitelib=/opt/airwave/local/lib/perl5/site_perl/5.10.1 -Dvendorlib=/opt/airwave/lib/perl5/vendor_perl/5.10.1 -Darchlib=/opt/airwave/lib64/perl5/5.10.1/x86_64-linux-thread-multi -Dsitearch=/opt/airwave/local/lib64/perl5/site_perl/5.10.1/x86_64-linux-thread-multi -Dvendorarch=/opt/airwave/lib64/perl5/vendor_perl/5.10.1/x86_64-linux-thread-multi -Dotherlibdirs=/opt/airwave/local/lib/perl5/site_perl/5.10.0:/opt/airwave/lib/perl5/vendor_perl/5.10.0 -Darchname=x86_64-linux-thread-multi -Dvendorprefix=/opt/airwave -Dsiteprefix=/opt/airwave/local -Duseshrplib -Dusethreads -Duseithreads -D
 uselargefiles -Dd_dosuid -Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm -Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly -Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto -Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto -Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/opt/airwave/bin'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g -m64 -mtune=generic',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.1.2 20080704 (Red Hat 4.1.2-48)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -fstack-protector'
    libpth=/usr/lib64 /lib64 /opt/airwave/lib64 /usr/lib /lib /opt/airwave/lib
    libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
    perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
    libc=/lib/libc-2.5.so, so=so, useshrplib=true, libperl=libperl.so
    gnulibc_version='2.5'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/opt/airwave/lib64/perl5/5.10.1/x86_64-linux-thread-multi/CORE'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -m64 -mtune=generic -fstack-protector'

Locally applied patches:
    


@INC for perl 5.10.1:
    /root/svn/mercury/lib/perl
    /root/svn/devtoys/rally/lib
    /root/svn/devtoys/bug_emails
    /root/svn/devtoys/farm
    /opt/airwave/lib64/perl5/5.10.1/x86_64-linux-thread-multi
    /opt/airwave/lib/perl5/5.10.1
    /opt/airwave/local/lib64/perl5/site_perl/5.10.1/x86_64-linux-thread-multi
    /opt/airwave/local/lib/perl5/site_perl/5.10.1
    /opt/airwave/local/lib/perl5/site_perl/5.10.0
    /opt/airwave/local/lib/perl5/site_perl
    /opt/airwave/lib64/perl5/vendor_perl/5.10.1/x86_64-linux-thread-multi
    /opt/airwave/lib/perl5/vendor_perl/5.10.1
    /opt/airwave/lib/perl5/vendor_perl/5.10.0
    /opt/airwave/lib/perl5/vendor_perl
    /opt/airwave/local/lib/perl5/site_perl/5.10.0
    /opt/airwave/lib/perl5/vendor_perl/5.10.0
    .


Environment for perl 5.10.1:
    HOME=/root
    LANG=en_US
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/opt/airwave/local/bin:/opt/airwave/bin:/usr/local/sbin:/root/svn/mercury/bin:/usr/local/airwave/bin:/usr/java/jre/bin:/usr/java/jdk/bin:/var/airwave/support:/opt/condor/sbin:/opt/condor/bin:/opt/flex/bin:/opt/ant/bin:/usr/sbin:/sbin:/usr/local/bin::/usr/kerberos/sbin:/usr/kerberos/bin:/bin:/usr/bin:/root/bin
    PERL5LIB=/root/svn/mercury/lib/perl:/root/svn/devtoys/rally/lib:/root/svn/devtoys/bug_emails:/root/svn/devtoys/farm
    PERL_BADLANG (unset)
    SHELL=/bin/bash

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Nov 6, 2010

From @xdg

On Fri, Nov 5, 2010 at 5​:59 PM, muir@​arubanetworks.com
<perlbug-followup@​perl.org> wrote​:

binmode() can clobber $@​ under certain circumstances. This reproduces it​:

Added t/io/layers.t TODO test as commit 54f3160

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Nov 6, 2010

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Nov 6, 2010

From zefram@fysh.org

muir@​arubanetworks.com wrote​:

binmode() can clobber $@​ under certain circumstances.

Any place that you call into arbitrarily complex code can clobber $@​,
due to internal use of eval. PerlIO layers are arbitrarily complex code,
so it's not surprising that binmode would do this. It's as if you'd
called a function from a library module. We don't generally treat this
as a bug, but the specific problem you ran into is a different matter​:

In case you are curious, the first symptom I saw was die() being unusually
silent (but still exiting the program). We had an object with a DESTROY
method hitting binmode in this way, which of course clobbered the exception
after it was thrown but before it was output to STDERR.

In Perl 5.13 we have made a change so that a DESTROY method clobbering $@​
won't interfere with flight of an exception. This problem won't occur
in this form on Perl 5.14 when it is released.

In any case, for a DESTROY method to clobber $@​ or other status variables
is generally a bug in that method, because it intereferes with the flow
of code that doesn't know the method is being called. Same goes for
signal handlers, and this is in contrast to the situation of normal
library functions/methods where the dynamically surrounding code knows
it's calling an arbitrarily complex function. If a DESTROY method (or
signal handler) calls arbitrarily complex code, it must mediate between
these two sets of expectations by ensuring that the status variables
don't get clobbered, which it can easily do by

  local($., $@​, $!, $^E, $?);

This will avoid your problem on the Perl that you are using, and avoid
other forms of the problem on all Perl versions. There has been some
talk of this possibly happening automatically around DESTROY methods,
but nothing has been resolved there.

-zefram

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Nov 6, 2010

From @druud62

On 2010-11-05 22​:59, muir @​ arubanetworks . com wrote​:

binmode() can clobber $@​ under certain circumstances. This reproduces it​:

$@​ = 'foo';

open(my $fh, '>/tmp/foo');
binmode($fh, '​:encoding(UTF-8)');

warn $@​;

Note that setting the binmode to '​:raw' does not clobber $@​.

If this turns out to not be a bug for some reason, the perldoc for binmode
should mention this behavior.

In case you are curious, the first symptom I saw was die() being unusually
silent (but still exiting the program). We had an object with a DESTROY
method hitting binmode in this way, which of course clobbered the exception
after it was thrown but before it was output to STDERR.

No bug. A quality of global variables is, well ... that they are global.

Decent Perl pattern​:

  my $result;

  eval {
  $result = myfunc();
  1; # success
  }
  or do {
  my $eval_error = $@​ || "unknown";
  ...;
  die $eval_error;
  };

To preserve $@​ you can localize it in your DESTROY.

See also Try​::Tiny and friends.

--
Ruud

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Nov 6, 2010

From @xdg

On Sat, Nov 6, 2010 at 8​:03 AM, Zefram <zefram@​fysh.org> wrote​:

muir@​arubanetworks.com wrote​:

binmode() can clobber $@​ under certain circumstances.

Any place that you call into arbitrarily complex code can clobber $@​,
due to internal use of eval.  PerlIO layers are arbitrarily complex code,
so it's not surprising that binmode would do this.  It's as if you'd
called a function from a library module.  We don't generally treat this
as a bug, but the specific problem you ran into is a different matter​:

That's true, but it's not immediately obvious to a non-expert how
arbitrarily complex binmode() can be. Why not localize $@​ for the
duration of the binmode call?

(Though I agree with other comments that it's best practice to
localize $@​ and inspect it or copy it immediately after an eval)

-- David

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Nov 6, 2010

From @cpansprout

Fixed in 523a494.

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Nov 6, 2010

@cpansprout - Status changed from 'open' to 'resolved'

@p5pRT p5pRT closed this Nov 6, 2010
@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Nov 7, 2010

From @jandubois

On Sat, 06 Nov 2010, David Golden wrote​:

On Sat, Nov 6, 2010 at 8​:03 AM, Zefram <zefram@​fysh.org> wrote​:
That's true, but it's not immediately obvious to a non-expert how
arbitrarily complex binmode() can be. Why not localize $@​ for the
duration of the binmode call?

It is also not obvious to non-experts how arbitrarily complex even the
most simple Perl expressions can be due to tying and overloading.

I would like to register _some_ protest against adding gratuitous cruft
to the Perl internals to deal with non-bugs (I don't feel strongly about
this individual case, but the cumulative effect of many of these
special-purpose hacks over time is what I'm concerned about).

Cheers,
-Jan

@p5pRT
Copy link
Collaborator Author

@p5pRT p5pRT commented Nov 7, 2010

From @xdg

On Sat, Nov 6, 2010 at 10​:29 PM, Jan Dubois <jand@​activestate.com> wrote​:

On Sat, 06 Nov 2010, David Golden wrote​:

On Sat, Nov 6, 2010 at 8​:03 AM, Zefram <zefram@​fysh.org> wrote​:
That's true, but it's not immediately obvious to a non-expert how
arbitrarily complex binmode() can be.  Why not localize $@​ for the
duration of the binmode call?

It is also not obvious to non-experts how arbitrarily complex even the
most simple Perl expressions can be due to tying and overloading.

Yes, but tying and overloading is usually done by the user, whereas a
non-expert doesn't realize exactly how many modules are loaded and
methods fired by perl to invoke binmode with an encoding. Most
built-in functions are not acting like arbitrarily complex function
calls.

I would like to register _some_ protest against adding gratuitous cruft
to the Perl internals to deal with non-bugs (I don't feel strongly about
this individual case, but the cumulative effect of many of these
special-purpose hacks over time is what I'm concerned about).

I understand the sentiment, entirely. I still hope someday for
something better than $@​ for managing exceptions.

-- David

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.