[PATCH] Cleanup two argument open usage.

[p5pRT]

Migrated from rt.perl.org#130122 (status was 'resolved')

Searchable as RT130122$

[p5pRT]

From @lightsey

Created by @lightsey

This is a bug report for perl from john@​nixnuts.net,
generated with the help of perlbug 1.40 running under perl 5.25.7.

-----------------------------------------------------------------
The attached patch removes most two argument open() calls in the
Perl codebase.

The 'cpan' and 't' directories were skipped.
Filehandle cloning was left as-is for 5.6.2 compatibility.

Notable changes​:
- Storable
- Devel​::SelfStubber
- Unicode​::UCD
- win32/bin/search.pl

Perl Info

Flags:
    category=core
    severity=wishlist
    Type=Patch
    PatchStatus=HasPatch

Site configuration information for perl 5.25.7:

Configured by jd at Wed Nov  9 09:22:44 CST 2016.

Summary of my perl5 (revision 5 version 25 subversion 7) configuration:
  Snapshot of: cc2b46b563e1a9147700b04d0d2e529ae63e2e12
  Platform:
    osname=linux
    osvers=4.8.0-1-amd64
    archname=x86_64-linux
    uname='linux slug 4.8.0-1-amd64 #1 smp debian 4.8.5-1 (2016-10-28) x86_64 gnulinux '
    config_args='-de -Dprefix=/home/jd/perl5/perlbrew/perls/perl-blead -Dcc=/usr/bin/afl-gcc -DDEBUGGING -Dusedevel -Aeval:scriptdir=/home/jd/perl5/perlbrew/perls/perl-blead/bin'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    bincompat5005=undef
  Compiler:
    cc='/usr/bin/afl-gcc'
    ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
    optimize='-O2 -g'
    cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='6.2.0 20161103'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='/usr/bin/afl-gcc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
    libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.24.so
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.24'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-strong'

Locally applied patches:
    Devel::PatchPerl 1.44


@INC for perl 5.25.7:
    /home/jd/perl5/perlbrew/perls/perl-blead/lib/site_perl/5.25.7/x86_64-linux
    /home/jd/perl5/perlbrew/perls/perl-blead/lib/site_perl/5.25.7
    /home/jd/perl5/perlbrew/perls/perl-blead/lib/5.25.7/x86_64-linux
    /home/jd/perl5/perlbrew/perls/perl-blead/lib/5.25.7


Environment for perl 5.25.7:
    HOME=/home/jd
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/jd/perl5/perlbrew/bin:/home/jd/perl5/perlbrew/perls/perl-blead/bin:/home/jd/.gems/bin:/home/jd/bin:/home/jd/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
    PERLBREW_BASHRC_VERSION=0.76
    PERLBREW_HOME=/home/jd/.perlbrew
    PERLBREW_MANPATH=/home/jd/perl5/perlbrew/perls/perl-blead/man
    PERLBREW_PATH=/home/jd/perl5/perlbrew/bin:/home/jd/perl5/perlbrew/perls/perl-blead/bin
    PERLBREW_PERL=perl-blead
    PERLBREW_ROOT=/home/jd/perl5/perlbrew
    PERLBREW_VERSION=0.76
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[p5pRT]

From @lightsey

0001-Remove-most-usage-of-the-two-argument-open.patch

[p5pRT]

From @jkeenan

On Thu, 17 Nov 2016 17​:52​:28 GMT, john@​nixnuts.net wrote​:

This is a bug report for perl from john@​nixnuts.net,
generated with the help of perlbug 1.40 running under perl 5.25.7.

-----------------------------------------------------------------
The attached patch removes most two argument open() calls in the
Perl codebase.

The 'cpan' and 't' directories were skipped.
Filehandle cloning was left as-is for 5.6.2 compatibility.

Notable changes​:
- Storable
- Devel​::SelfStubber
- Unicode​::UCD
- win32/bin/search.pl

1. The patch is large but contains the same changes over and over again -- which makes me nod off while reviewing it. :-) Could I ask that we have multiple people look at this, say, in 1000-line chunks? (I've looked over lines 1-1000.)

2. I'm in favor of this, but I recognize that there are people who may object on the basis of "Why fix something that isn't broken?". If you are one of that group, please raise your objection now.

Thank you very much.
Jim Keenan

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags​:
category=core
severity=wishlist
Type=Patch
PatchStatus=HasPatch
---
Site configuration information for perl 5.25.7​:

Configured by jd at Wed Nov 9 09​:22​:44 CST 2016.

Summary of my perl5 (revision 5 version 25 subversion 7)
configuration​:
Snapshot of​: cc2b46b
Platform​:
osname=linux
osvers=4.8.0-1-amd64
archname=x86_64-linux
uname='linux slug 4.8.0-1-amd64 #1 smp debian 4.8.5-1 (2016-10-28)
x86_64 gnulinux '
config_args='-de -Dprefix=/home/jd/perl5/perlbrew/perls/perl-blead
-Dcc=/usr/bin/afl-gcc -DDEBUGGING -Dusedevel
-Aeval​:scriptdir=/home/jd/perl5/perlbrew/perls/perl-blead/bin'
hint=recommended
useposix=true
d_sigaction=define
useithreads=undef
usemultiplicity=undef
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
bincompat5005=undef
Compiler​:
cc='/usr/bin/afl-gcc'
ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-
protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64'
optimize='-O2 -g'
cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-
protector-strong -I/usr/local/include'
ccversion=''
gccversion='6.2.0 20161103'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries​:
ld='/usr/bin/afl-gcc'
ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/6/include-
fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu
/lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.24.so
so=so
useshrplib=false
libperl=libperl.a
gnulibc_version='2.24'
Dynamic Linking​:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E'
cccdlflags='-fPIC'
lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-
strong'

Locally applied patches​:
Devel​::PatchPerl 1.44

---
@​INC for perl 5.25.7​:
/home/jd/perl5/perlbrew/perls/perl-
blead/lib/site_perl/5.25.7/x86_64-linux
/home/jd/perl5/perlbrew/perls/perl-blead/lib/site_perl/5.25.7
/home/jd/perl5/perlbrew/perls/perl-blead/lib/5.25.7/x86_64-linux
/home/jd/perl5/perlbrew/perls/perl-blead/lib/5.25.7

---
Environment for perl 5.25.7​:
HOME=/home/jd
LANG=en_US.UTF-8
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/home/jd/perl5/perlbrew/bin​:/home/jd/perl5/perlbrew/perls/perl-
blead/bin​:/home/jd/.gems/bin​:/home/jd/bin​:/home/jd/bin​:/usr/local/bin​:/usr/bin​:/bin​:/usr/local/games​:/usr/games
PERLBREW_BASHRC_VERSION=0.76
PERLBREW_HOME=/home/jd/.perlbrew
PERLBREW_MANPATH=/home/jd/perl5/perlbrew/perls/perl-blead/man
PERLBREW_PATH=/home/jd/perl5/perlbrew/bin​:/home/jd/perl5/perlbrew/perls/perl-
blead/bin
PERLBREW_PERL=perl-blead
PERLBREW_ROOT=/home/jd/perl5/perlbrew
PERLBREW_VERSION=0.76
PERL_BADLANG (unset)
SHELL=/bin/bash

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @jkeenan

On Thu, 17 Nov 2016 17​:52​:28 GMT, john@​nixnuts.net wrote​:

This is a bug report for perl from john@​nixnuts.net,
generated with the help of perlbug 1.40 running under perl 5.25.7.

-----------------------------------------------------------------
The attached patch removes most two argument open() calls in the
Perl codebase.

The 'cpan' and 't' directories were skipped.
Filehandle cloning was left as-is for 5.6.2 compatibility.

Notable changes​:
- Storable
- Devel​::SelfStubber
- Unicode​::UCD
- win32/bin/search.pl

John, there are places in this patch where I believe we can do a bit more cleanup, i.e., places where, after re-writing an 'open' call in 3-arg form, we have double-quotes around a scalar holding a file name.

#####
$ grep -n -E '^\+.*?open[^"]+"\$' 130122-0001-Remove-most-usage-of-the-two-argument-open.patch | grep -v -E 'or (die|return) "\$'
232​:+open(H,'>',"$file.new") || die "Cannot open $file.new​:$!";
1762​:+ if (open(TMPC, '>', "$tmp.c")) {
1771​:+ open( CMDFILE, '>', "$tmp.com" );
2408​:+open (CODE, '>', "$tmp.c") || die "$iam​: cannot create $tmp.c​: $!\n";
2421​:+ if (!open (INCLUDE,'<',"$LIB/$include")) {
2434​:+ open(H, '<', "$file.html") ||
2443​:+ open(H, '>', "$file.html") ||
2461​:+ open(my $in, '<', "$dir/$file") ||
2479​:+ open(SPLITOUT, '>', "$poddir/$file") ||
3060​:+open(OUT,'>',"${outdir}config.sh") || die "Can't open ${outdir}config.sh​: $!\n";
3069​:+if (open(SH,'<',"${outdir}config_h.SH")) {
3078​:+if (open(PL,'<',"${outdir}patchlevel.h")) {
3091​:+ open(MAN, '>', "$myname.3pl");
3104​:+ open (POD, '>', "$name.pod") || die "can't open $name.pod​: $!";
3461​:+ if (open(my $fh, '>', "$AppName.app")) {
3470​:+ if (open(my $fh, '>', "$AppName.rsc")) {
4354​:+ open my $fhdyna, '>', "$opts{distdir}/lib/Dynaloader.pm";
#####

I believe that in these cases we don't need the double-quotes around the 3rd arg -- correct?

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

From @csjewell

On Fri, 23 Dec 2016 05​:46​:03 -0800, jkeenan wrote​:

On Thu, 17 Nov 2016 17​:52​:28 GMT, john@​nixnuts.net wrote​:

This is a bug report for perl from john@​nixnuts.net,
generated with the help of perlbug 1.40 running under perl 5.25.7.

-----------------------------------------------------------------
The attached patch removes most two argument open() calls in the
Perl codebase.

The 'cpan' and 't' directories were skipped.
Filehandle cloning was left as-is for 5.6.2 compatibility.

Notable changes​:
- Storable
- Devel​::SelfStubber
- Unicode​::UCD
- win32/bin/search.pl

John, there are places in this patch where I believe we can do a bit
more cleanup, i.e., places where, after re-writing an 'open' call in
3-arg form, we have double-quotes around a scalar holding a file name.

#####
$ grep -n -E '^\+.*?open[^"]+"\$' 130122-0001-Remove-most-usage-of-
the-two-argument-open.patch | grep -v -E 'or (die|return) "\$'
232​:+open(H,'>',"$file.new") || die "Cannot open $file.new​:$!";
1762​:+ if (open(TMPC, '>', "$tmp.c")) {
1771​:+ open( CMDFILE, '>', "$tmp.com" );
2408​:+open (CODE, '>', "$tmp.c") || die "$iam​: cannot create $tmp.c​:
$!\n";
2421​:+ if (!open (INCLUDE,'<',"$LIB/$include")) {
2434​:+ open(H, '<', "$file.html") ||
2443​:+ open(H, '>', "$file.html") ||
2461​:+ open(my $in, '<', "$dir/$file") ||
2479​:+ open(SPLITOUT, '>', "$poddir/$file") ||
3060​:+open(OUT,'>',"${outdir}config.sh") || die "Can't open
${outdir}config.sh​: $!\n";
3069​:+if (open(SH,'<',"${outdir}config_h.SH")) {
3078​:+if (open(PL,'<',"${outdir}patchlevel.h")) {
3091​:+ open(MAN, '>', "$myname.3pl");
3104​:+ open (POD, '>', "$name.pod") || die "can't open $name.pod​:
$!";
3461​:+ if (open(my $fh, '>', "$AppName.app")) {
3470​:+ if (open(my $fh, '>', "$AppName.rsc")) {
4354​:+ open my $fhdyna, '>', "$opts{distdir}/lib/Dynaloader.pm";
#####

I believe that in these cases we don't need the double-quotes around
the 3rd arg -- correct?

Thank you very much.

Incorrect. In all those cases, they are either appending an extension or filename on, or a second scalar. Therefore, they all need the double quotes, as an alternative to something like $AppName . '.app'

[p5pRT]

From @jkeenan

On Fri, 23 Dec 2016 14​:16​:30 GMT, csjewell wrote​:

On Fri, 23 Dec 2016 05​:46​:03 -0800, jkeenan wrote​:

On Thu, 17 Nov 2016 17​:52​:28 GMT, john@​nixnuts.net wrote​:

This is a bug report for perl from john@​nixnuts.net,
generated with the help of perlbug 1.40 running under perl 5.25.7.

-----------------------------------------------------------------
The attached patch removes most two argument open() calls in the
Perl codebase.

The 'cpan' and 't' directories were skipped.
Filehandle cloning was left as-is for 5.6.2 compatibility.

Notable changes​:
- Storable
- Devel​::SelfStubber
- Unicode​::UCD
- win32/bin/search.pl

John, there are places in this patch where I believe we can do a bit
more cleanup, i.e., places where, after re-writing an 'open' call in
3-arg form, we have double-quotes around a scalar holding a file
name.

#####
$ grep -n -E '^\+.*?open[^"]+"\$' 130122-0001-Remove-most-usage-of-
the-two-argument-open.patch | grep -v -E 'or (die|return) "\$'
232​:+open(H,'>',"$file.new") || die "Cannot open $file.new​:$!";
1762​:+ if (open(TMPC, '>', "$tmp.c")) {
1771​:+ open( CMDFILE, '>', "$tmp.com" );
2408​:+open (CODE, '>', "$tmp.c") || die "$iam​: cannot create $tmp.c​:
$!\n";
2421​:+ if (!open (INCLUDE,'<',"$LIB/$include")) {
2434​:+ open(H, '<', "$file.html") ||
2443​:+ open(H, '>', "$file.html") ||
2461​:+ open(my $in, '<', "$dir/$file") ||
2479​:+ open(SPLITOUT, '>', "$poddir/$file") ||
3060​:+open(OUT,'>',"${outdir}config.sh") || die "Can't open
${outdir}config.sh​: $!\n";
3069​:+if (open(SH,'<',"${outdir}config_h.SH")) {
3078​:+if (open(PL,'<',"${outdir}patchlevel.h")) {
3091​:+ open(MAN, '>', "$myname.3pl");
3104​:+ open (POD, '>', "$name.pod") || die "can't open $name.pod​:
$!";
3461​:+ if (open(my $fh, '>', "$AppName.app")) {
3470​:+ if (open(my $fh, '>', "$AppName.rsc")) {
4354​:+ open my $fhdyna, '>', "$opts{distdir}/lib/Dynaloader.pm";
#####

I believe that in these cases we don't need the double-quotes around
the 3rd arg -- correct?

Thank you very much.

Incorrect. In all those cases, they are either appending an extension
or filename on, or a second scalar. Therefore, they all need the
double quotes, as an alternative to something like $AppName . '.app'

Ah! The value of additional eyes on a problem. curtis++.

I've now reviewed the first 2000 lines of the patch. Only 3500 to go! Anyone want to help out?

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

From @jkeenan

On Thu, 17 Nov 2016 17​:52​:28 GMT, john@​nixnuts.net wrote​:

This is a bug report for perl from john@​nixnuts.net,
generated with the help of perlbug 1.40 running under perl
5.25.7.

-----------------------------------------------------------------
The attached patch removes most two argument open() calls in the
Perl codebase.

The 'cpan' and 't' directories were skipped.
Filehandle cloning was left as-is for 5.6.2 compatibility.

Notable changes​:
- Storable
- Devel​::SelfStubber
- Unicode​::UCD
- win32/bin/search.pl

Pushed to blead in commit 1ae6ead.

John, can you double-check that commit against your original patch?

Since the patch was large and submitted some time ago, blead had moved. So the original patch did not apply cleanly. I essentially took it apart and applied it in chunks via 'patch'. Then I had to increment certain $VERSION numbers and run certain programs to pass the porting tests. So your proofreading would be helpful to make sure I didn't miss anything crucial.

Also, you might want to submit -- in a new perlbug ticket -- a patch to clean up 2-arg open() calls like the following which still remain.

#####
$> cat dist/Storable/t/code.t

193 open(SAVEERR, ">&STDERR");
194 open(STDERR, '>', $devnull) or
195 ( print SAVEERR "Unable to redirect STDERR​: $!\n" and exit(1) );
196
197 eval { $freezed = freeze $obj[0]->[0] };
198
199 open(STDERR, ">&SAVEERR");
#####

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

From @lightsey

On Fri, 2016-12-23 at 11​:06 -0800, James E Keenan via RT wrote​:

Pushed to blead in commit 1ae6ead.

John, can you double-check that commit against your original patch?

The minor code differences look fine to me. Thanks for following through on
this.

Also, you might want to submit -- in a new perlbug ticket -- a patch to clean
up 2-arg open() calls like the following which still remain.

#####
$> cat dist/Storable/t/code.t

193     open(SAVEERR, ">&STDERR");
194     open(STDERR, '>', $devnull) or

I left the filehandle-copying opens in the two-argument format for compatibility
with Perl 5.6.2.

If you don't think compatibility with 5.6.2 is worth keeping, particularly in
the /dist/ code, I can make another pass and remove them.

[p5pRT]

From @jkeenan

On Fri, 23 Dec 2016 23​:14​:31 GMT, john@​nixnuts.net wrote​:

On Fri, 2016-12-23 at 11​:06 -0800, James E Keenan via RT wrote​:

Pushed to blead in commit 1ae6ead.

John, can you double-check that commit against your original patch?

The minor code differences look fine to me. Thanks for following
through on
this.

Also, you might want to submit -- in a new perlbug ticket -- a patch
to clean
up 2-arg open() calls like the following which still remain.

#####
$> cat dist/Storable/t/code.t

193     open(SAVEERR, ">&STDERR");
194     open(STDERR, '>', $devnull) or

I left the filehandle-copying opens in the two-argument format for
compatibility
with Perl 5.6.2.

If you don't think compatibility with 5.6.2 is worth keeping,
particularly in
the /dist/ code, I can make another pass and remove them.

No, you can leave them as is. I overlooked your rationale for leaving them alone.

Resolving ticket.

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

@jkeenan - Status changed from 'open' to 'resolved'