Cwd: different return values between pure perl and XS variants #16338

p5pRT · 2017-12-23T20:32:52Z

Migrated from rt.perl.org#132648 (status was 'resolved')

Searchable as RT132648$

p5pRT · 2017-12-23T20:32:52Z

From @eserte

This is a bug report for perl from slaven@rezic.de,
generated with the help of perlbug 1.40 running under perl 5.26.1.

The functions getcwd and abs_path return different values for the
XS and pure perl variants if a directory is non-existent. The XS
variant returns undef, while the PP variant returns an empty string.

A test program:

#!/usr/bin/perl

use strict;
use warnings;
use Test::More 'no_plan';
use File::Temp 'tempdir';
use Cwd 'getcwd', 'abs_path';

my $tmp = tempdir(CLEANUP => 1);
mkdir "$tmp/testdir" or die $!;
chdir "$tmp/testdir" or die $!;
rmdir "$tmp/testdir" or die $!;
is getcwd, undef, 'XS getcwd on non-existent directory';
is abs_path("."), undef, 'XS abs_path on non-existent directory';
{
no warnings 'redefine';
# force pure perl variants
local *Cwd::abs_path = \&Cwd::_perl_abs_path;
local *Cwd::getcwd = \&Cwd::_perl_getcwd;
is Cwd::getcwd, undef, 'PP getcwd on non-existent directory';
is Cwd::abs_path("."), undef, 'PP abs_path on non-existent directory';
}
chdir "/";

__END__

Output:

ok 1 - XS getcwd on non-existent directory
ok 2 - XS abs_path on non-existent directory
readdir(./..): No such file or directory at /tmp/cwd.pl line 20.
not ok 3 - PP getcwd on non-existent directory
# Failed test 'PP getcwd on non-existent directory'
# at /tmp/cwd.pl line 20.
# got: ''
# expected: undef
readdir(./..): No such file or directory at /tmp/cwd.pl line 21.
not ok 4 - PP abs_path on non-existent directory
# Failed test 'PP abs_path on non-existent directory'
# at /tmp/cwd.pl line 21.
# got: ''
# expected: undef
1..4
# Looks like you failed 2 tests of 4.

Additionally, the PP variants are quite loud while the XS variant is
silent in this situation. But this might be acceptable.

Probably the fix is to change all "return ''" statements in _perl_abs_path
to "return undef".

Regards,
Slaven

Flags:
category=library
severity=low
module=Cwd

Site configuration information for perl 5.26.1:

Configured by eserte at Sat Sep 23 09:36:11 CEST 2017.

Summary of my perl5 (revision 5 version 26 subversion 1) configuration:

Platform:
osname=linux
osvers=3.16.0-4-amd64
archname=x86_64-linux
uname='linux cabulja 3.16.0-4-amd64 #1 smp debian 3.16.39-1+deb8u2 (2017-03-07) x86_64 gnulinux '
config_args='-ds -e -Dprefix=/opt/perl-5.26.1 -Dcf_email=srezic@cpan.org'
hint=recommended
useposix=true
d_sigaction=define
useithreads=undef
usemultiplicity=undef
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler:
cc='cc'
ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
optimize='-O2'
cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion=''
gccversion='4.9.2'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries:
ld='cc'
ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.19.so
so=so
useshrplib=false
libperl=libperl.a
gnulibc_version='2.19'
Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E'
cccdlflags='-fPIC'
lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'

@INC for perl 5.26.1:
/opt/perl-5.26.1/lib/site_perl/5.26.1/x86_64-linux
/opt/perl-5.26.1/lib/site_perl/5.26.1
/opt/perl-5.26.1/lib/5.26.1/x86_64-linux
/opt/perl-5.26.1/lib/5.26.1

Environment for perl 5.26.1:
HOME=/home/eserte
LANG=en_US.UTF-8
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/eserte/bin/linux-gnu:/home/eserte/bin/sh:/home/eserte/bin:/home/eserte/bin/pistachio-perl/bin:/usr/games:/home/eserte/devel
PERLDOC=-MPod::Perldoc::ToTextOverstrike
PERL_BADLANG (unset)
SHELL=/bin/zsh

p5pRT · 2017-12-24T11:16:06Z

From zefram@fysh.org

Fixed in commit d2e38af.

-zefram

p5pRT · 2017-12-24T11:16:06Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2017-12-26T16:59:16Z

From @jkeenan

On Sun, 24 Dec 2017 11:16:06 GMT, zefram@fysh.org wrote:

Fixed in commit d2e38af.

-zefram

This does appear to be fixed on FreeBSD, but we've got a smoke test report from Cygwin explicitly FAILing on dist/PathTools/t/cwd_enoent.t. Please see:

http://perl.develop-help.com/raw/?id=204663

Thank you very much.
--
James E Keenan (jkeenan@cpan.org)

p5pRT · 2017-12-26T17:00:28Z

From @jkeenan

On Tue, 26 Dec 2017 16:59:16 GMT, jkeenan wrote:

On Sun, 24 Dec 2017 11:16:06 GMT, zefram@fysh.org wrote:

Fixed in commit d2e38af.

-zefram

This does appear to be fixed on FreeBSD, but we've got a smoke test
report from Cygwin explicitly FAILing on
dist/PathTools/t/cwd_enoent.t. Please see:

http://perl.develop-help.com/raw/?id=204663

Thank you very much.

Please ignore the above. Wrong ticket.

--
James E Keenan (jkeenan@cpan.org)

p5pRT · 2017-12-28T12:02:10Z

@iabyn - Status changed from 'open' to 'pending release'

p5pRT · 2018-01-15T02:32:28Z

From @tonycoz

On Sun, Dec 24, 2017 at 11:15:47AM +0000, Zefram wrote:

Fixed in commit d2e38af.

This new test has been failing on cygwin since it was introduced.

tony@saturn ~/dev/perl/git/perl/t
$ ./perl harness -v ../dist/PathTools/t/cwd_enoent.t
../dist/PathTools/t/cwd_enoent.t ..
1..8
not ok 1 - regular getcwd result on non-existent directory
# Failed test 'regular getcwd result on non-existent directory'
# at t/cwd_enoent.t line 30.
# got: '/tmp/4WuHD3C4IY/testdir'
# expected: undef
not ok 2 - regular getcwd errno on non-existent directory
# Failed test 'regular getcwd errno on non-existent directory'
# at t/cwd_enoent.t line 31.
# got: '0'
# expected: '2'
.: No such file or directory at t/cwd_enoent.t line 33.
# Looks like your test exited with 2 just after 2.
.: No such file or directory at ../../lib/File/Temp.pm line 777.
END failed--call queue aborted.
Dubious, test returned 2 (wstat 512, 0x200)
Failed 8/8 subtests

Test Summary Report

../dist/PathTools/t/cwd_enoent.t (Wstat: 512 Tests: 2 Failed: 2)
Failed tests: 1-2
Non-zero exit status: 2
Parse errors: Bad plan. You planned 8 tests but ran 2.
Files=1, Tests=2, 0 wallclock secs ( 0.00 usr 0.00 sys + 0.09 cusr 0.00 csys = 0.09 CPU)
Result: FAIL

tony@saturn ~/dev/perl/git/perl/t
$ git describe
v5.27.7-132-g22803c6a19

Tony

p5pRT · 2018-01-15T19:31:25Z

From @eserte

Dana Sun, 14 Jan 2018 18:32:28 -0800, tonyc reče:

On Sun, Dec 24, 2017 at 11:15:47AM +0000, Zefram wrote:

Fixed in commit d2e38af.

This new test has been failing on cygwin since it was introduced.

tony@saturn ~/dev/perl/git/perl/t
$ ./perl harness -v ../dist/PathTools/t/cwd_enoent.t
../dist/PathTools/t/cwd_enoent.t ..
1..8
not ok 1 - regular getcwd result on non-existent directory
# Failed test 'regular getcwd result on non-existent directory'
# at t/cwd_enoent.t line 30.
# got: '/tmp/4WuHD3C4IY/testdir'
# expected: undef
not ok 2 - regular getcwd errno on non-existent directory
# Failed test 'regular getcwd errno on non-existent directory'
# at t/cwd_enoent.t line 31.
# got: '0'
# expected: '2'
.: No such file or directory at t/cwd_enoent.t line 33.
# Looks like your test exited with 2 just after 2.
.: No such file or directory at ../../lib/File/Temp.pm line 777.
END failed--call queue aborted.
Dubious, test returned 2 (wstat 512, 0x200)
Failed 8/8 subtests

Test Summary Report
-------------------
../dist/PathTools/t/cwd_enoent.t (Wstat: 512 Tests: 2 Failed: 2)
Failed tests: 1-2
Non-zero exit status: 2
Parse errors: Bad plan. You planned 8 tests but ran 2.
Files=1, Tests=2, 0 wallclock secs ( 0.00 usr 0.00 sys + 0.09 cusr
0.00 csys = 0.09 CPU)
Result: FAIL

tony@saturn ~/dev/perl/git/perl/t
$ git describe
v5.27.7-132-g22803c6a19

I guess that's because Cygwin's getcwd() is implemented with _backtick_pwd(), and Cygwin's pwd command behaves differently than on Unix --- in this situation it returns the non-existent directory.

Regards,
Slaven

p5pRT · 2018-01-15T23:28:26Z

From zefram@fysh.org

Tony Cook wrote:

# Failed test 'regular getcwd result on non-existent directory'
# at t/cwd_enoent.t line 30.
# got: '/tmp/4WuHD3C4IY/testdir'
# expected: undef

For getcwd() to yield a non-error return for a non-existent directory
seems like a bug. Cygwin seems to prefer _backtick_pwd() as its
implementation of getcwd(), so maybe this is a bug in Cygwin's pwd(1).
Would someone on Cygwin please check, firstly, whether pwd(1) actually
exhibits this behaviour for a removed directory, and secondly, whether
the removed directory can actually be addressed by this name. (If it
can be addressed by name, that would seem to be a bug in rmdir().)

.: No such file or directory at t/cwd_enoent.t line 33.
# Looks like your test exited with 2 just after 2.

This is another problem. That's abs_path() dying, which it shouldn't:
it's documented to return errors as undef+$!. Cygwin actually uses
fast_abs_path() as its abs_path(), and fast_abs_path() is coded to die
rather than yield undef+$!, so that's a clear bug in fast_abs_path().

-zefram

p5pRT · 2018-01-16T05:26:38Z

From zefram@fysh.org

I wrote:

fast_abs_path() as its abs_path(), and fast_abs_path() is coded to die
rather than yield undef+$!, so that's a clear bug in fast_abs_path().

This bit fixed in commit a97021b.

-zefram

p5pRT · 2018-01-16T05:30:48Z

From @tonycoz

On Mon, Jan 15, 2018 at 11:28:06PM +0000, Zefram wrote:

Tony Cook wrote:

# Failed test 'regular getcwd result on non-existent directory'
# at t/cwd_enoent.t line 30.
# got: '/tmp/4WuHD3C4IY/testdir'
# expected: undef

For getcwd() to yield a non-error return for a non-existent directory
seems like a bug. Cygwin seems to prefer _backtick_pwd() as its
implementation of getcwd(), so maybe this is a bug in Cygwin's pwd(1).
Would someone on Cygwin please check, firstly, whether pwd(1) actually
exhibits this behaviour for a removed directory, and secondly, whether
the removed directory can actually be addressed by this name. (If it
can be addressed by name, that would seem to be a bug in rmdir().)

I tested with:

tony@saturn ~/dev/perl/git
$ cat 132648.c
#include <stdio.h>
#include <unistd.h>
#include <sys/stat.h>
#include <sys/types.h>

int main(void) {
char buf[1000]; /* keeping this simple */
if (mkdir("foo", 0700) < 0) {
perror("mkdir");
return 1;
}
if (chdir("foo") < 0) {
perror("chdir");
return 1;
}
if (rmdir("../foo") < 0) {
perror("rmdir");
return 1;
}

if (!getcwd(buf, sizeof(buf))) {
perror("getcwd");
return 1;
}
puts(buf);

return 0;
}

tony@saturn ~/dev/perl/git
$ cc -o132648.exe 132648.c

tony@saturn ~/dev/perl/git
$ ./132648
/home/tony/dev/perl/git/foo

I've asked about it on the cygwin mailing list.

I suspect this will be WONTFIX at the cygwin level.

Tony

p5pRT · 2018-01-16T05:51:05Z

From zefram@fysh.org

Tony Cook wrote:

I suspect this will be WONTFIX at the cygwin level.

I'd still like to know which bug it is that they won't fix.

-zefram

p5pRT · 2018-01-16T07:58:09Z

From @tonycoz

On Tue, Jan 16, 2018 at 05:50:47AM +0000, Zefram wrote:

Tony Cook wrote:

I suspect this will be WONTFIX at the cygwin level.

I'd still like to know which bug it is that they won't fix.

Well, remembering cygwin is built on top of Win32, and if I'm reading
the code correctly, rather than calling the RemoveDirectory() API,
which fails on "open" directories, cygwin calls NtSetInformationFile()
to mark the directory to be deleted once it's closed.

Having to record that the directory (or file) is marked deleted and
not make it available to getcwd(), open(), opendir(), unlink() etc
might be considered too much work for too little return.

Tony

p5pRT · 2018-01-16T08:24:17Z

From zefram@fysh.org

Tony Cook wrote:

Having to record that the directory (or file) is marked deleted and
not make it available to getcwd(), open(), opendir(), unlink() etc

That's not what I'm proposing, and would be totally unworkable.

If rmdir(2) doesn't actually succeed in removing the directory
(immediately), then I'll add a check for that in cwd_enoent.t, skipping
the tests if the directory didn't go away. If rmdir() does remove the
directory and pwd(1) emits a name that can't be used, then I'll add a
check for that in _backtick_pwd(), to prevent it returning an unusable
name.

-zefram

p5pRT · 2018-01-16T10:27:46Z

From @tonycoz

On Tue, Jan 16, 2018 at 08:23:58AM +0000, Zefram wrote:

Tony Cook wrote:

Having to record that the directory (or file) is marked deleted and
not make it available to getcwd(), open(), opendir(), unlink() etc

That's not what I'm proposing, and would be totally unworkable.

If rmdir(2) doesn't actually succeed in removing the directory
(immediately), then I'll add a check for that in cwd_enoent.t, skipping
the tests if the directory didn't go away. If rmdir() does remove the
directory and pwd(1) emits a name that can't be used, then I'll add a
check for that in _backtick_pwd(), to prevent it returning an unusable
name.

If this what you need?

tony@saturn ~/dev/perl/git
$ perl 132648.pl
/home/tony/dev/perl/git/foo

tony@saturn ~/dev/perl/git
$ cat 132648.pl
#!perl
use Cwd;

mkdir "foo", 0700
or die "mkdir $!";
chdir "foo"
or die "chdir $!";
rmdir "../foo"
or die "rmdir $!";
my $cwd = getcwd
or die "getcwd $!";
print "$cwd\n";
print "$cwd is a dir\n" if -d $cwd;

Tony

p5pRT · 2018-01-16T10:45:55Z

From zefram@fysh.org

Tony Cook wrote:

If this what you need?

Nearly. Instead of the -d line, please do

print stat($cwd) ? join",",stat(_) : $!, "\n";

and please show the output of Cwd::_backtick_pwd(), not just getcwd().
For good measure, please do

print $_, " ", \&{"Cwd::$_"}, "\n"
foreach qw(cwd getcwd fastgetcwd fastcwd _backtick_pwd _perl_getcwd);

-zefram

p5pRT · 2018-01-16T23:23:23Z

From @tonycoz

On Tue, Jan 16, 2018 at 10:45:38AM +0000, Zefram wrote:

Tony Cook wrote:

If this what you need?

Nearly. Instead of the -d line, please do
print stat$$cwd$ ? join"\,"\,stat$\_$ : $\!\, "\\n";
and please show the output of Cwd::_backtick_pwd(), not just getcwd().
For good measure, please do
print $\_\, " "\, \\&\{"Cwd&#8203;::$\_"\}\, "\\n"
foreach qw$cwd getcwd fastgetcwd fastcwd \_backtick\_pwd \_perl\_getcwd$;

tony@saturn ~/dev/perl/git
$ perl 132648.pl
/home/tony/dev/perl/git/foo
No such file or directory
cwd /home/tony/dev/perl/git/foo
getcwd /home/tony/dev/perl/git/foo
fastgetcwd /home/tony/dev/perl/git/foo
fastcwd /home/tony/dev/perl/git/foo
_backtick_pwd /home/tony/dev/perl/git/foo
.: No such file or directory at 132648.pl line 15.

tony@saturn ~/dev/perl/git
$ cat 132648.pl
#!perl
use Cwd;

mkdir "foo", 0700
or die "mkdir $!";
chdir "foo"
or die "chdir $!";
rmdir "../foo"
or die "rmdir $!";
my $cwd = getcwd
or die "getcwd $!";
print "$cwd\n";
print stat($cwd) ? join",",stat(_) : $!, "\n";
print $_, " ", &{"Cwd::$_"}, "\n"
foreach qw(cwd getcwd fastgetcwd fastcwd _backtick_pwd _perl_getcwd);

I assumed you wanted the Cwd::foo functions called rather than a bunch
of CODE(...) output.

Tony

p5pRT · 2018-01-16T23:44:15Z

From zefram@fysh.org

Tony Cook wrote:

I assumed you wanted the Cwd::foo functions called rather than a bunch
of CODE(...) output.

I wanted the CODE(...) output. I want to see which names are aliased,
and hence which implementations are hiding behind each name. Though the
outputs of the functions are also interesting data points.

-zefram

p5pRT · 2018-01-16T23:49:13Z

From @tonycoz

On Tue, Jan 16, 2018 at 11:44:00PM +0000, Zefram wrote:

Tony Cook wrote:

I assumed you wanted the Cwd::foo functions called rather than a bunch
of CODE(...) output.

I wanted the CODE(...) output. I want to see which names are aliased,
and hence which implementations are hiding behind each name. Though the
outputs of the functions are also interesting data points.

$ perl 132648.pl
/home/tony/dev/perl/git/foo
cwd CODE(0x800f9630)
getcwd CODE(0x800f9630)
fastgetcwd CODE(0x800f9630)
fastcwd CODE(0x800f9630)
_backtick_pwd CODE(0x800a3a20)
_perl_getcwd CODE(0x800a35d0)
No such file or directory

Tony

p5pRT · 2018-01-17T21:16:04Z

From zefram@fysh.org

Findings from debugging output provided by Tony:

* rmdir() succeeds in making the directory's former name unresolvable.

* Cygwin uses the XS implementation for its getcwd(), not _backtick_pwd()
as the code seems to prefer.

* both the XS getcwd() and _backtick_pwd() produce the erroneous result.

* behind the XS getcwd(), the core's getcwd_sv() and libc's getcwd(3)
produce the erroneous result.

This is fixable, but it's got more complicated than I'm willing to
squeeze into the present ticket. I've therefore made cwd_enoent.t just
skip on Cygwin, and opened [perl #132733] regarding getcwd() erroneously
producing a non-error result.

-zefram

p5pRT · 2018-06-23T15:32:52Z

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release yesterday of Perl 5.28.0, this and 185 other issues have been
resolved.

Perl 5.28.0 may be downloaded via:
https://metacpan.org/release/XSAWYERX/perl-5.28.0

If you find that the problem persists, feel free to reopen this ticket.

p5pRT · 2018-06-23T15:32:53Z

@khwilliamson - Status changed from 'pending release' to 'resolved'

p5pRT closed this as completed Jun 23, 2018

p5pRT added the Severity Low label Oct 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cwd: different return values between pure perl and XS variants #16338

Cwd: different return values between pure perl and XS variants #16338

p5pRT commented Dec 23, 2017

p5pRT commented Dec 23, 2017

p5pRT commented Dec 24, 2017

p5pRT commented Dec 24, 2017

p5pRT commented Dec 26, 2017

p5pRT commented Dec 26, 2017

p5pRT commented Dec 28, 2017

p5pRT commented Jan 15, 2018

p5pRT commented Jan 15, 2018

p5pRT commented Jan 15, 2018

p5pRT commented Jan 16, 2018

p5pRT commented Jan 16, 2018

p5pRT commented Jan 16, 2018

p5pRT commented Jan 16, 2018

p5pRT commented Jan 16, 2018

p5pRT commented Jan 16, 2018

p5pRT commented Jan 16, 2018

p5pRT commented Jan 16, 2018

p5pRT commented Jan 16, 2018

p5pRT commented Jan 16, 2018

p5pRT commented Jan 17, 2018

p5pRT commented Jun 23, 2018

p5pRT commented Jun 23, 2018

Cwd: different return values between pure perl and XS variants #16338

Cwd: different return values between pure perl and XS variants #16338

Comments

p5pRT commented Dec 23, 2017

p5pRT commented Dec 23, 2017

From @eserte

p5pRT commented Dec 24, 2017

From zefram@fysh.org

p5pRT commented Dec 24, 2017

p5pRT commented Dec 26, 2017

From @jkeenan

p5pRT commented Dec 26, 2017

From @jkeenan

p5pRT commented Dec 28, 2017

p5pRT commented Jan 15, 2018

From @tonycoz

p5pRT commented Jan 15, 2018

From @eserte

p5pRT commented Jan 15, 2018

From zefram@fysh.org

p5pRT commented Jan 16, 2018

From zefram@fysh.org

p5pRT commented Jan 16, 2018

From @tonycoz

p5pRT commented Jan 16, 2018

From zefram@fysh.org

p5pRT commented Jan 16, 2018

From @tonycoz

p5pRT commented Jan 16, 2018

From zefram@fysh.org

p5pRT commented Jan 16, 2018

From @tonycoz

p5pRT commented Jan 16, 2018

From zefram@fysh.org

p5pRT commented Jan 16, 2018

From @tonycoz

p5pRT commented Jan 16, 2018

From zefram@fysh.org

p5pRT commented Jan 16, 2018

From @tonycoz

p5pRT commented Jan 17, 2018

From zefram@fysh.org

p5pRT commented Jun 23, 2018

From @khwilliamson

p5pRT commented Jun 23, 2018