lib/File/Copy.t fails when running in non-root

[p5pRT]

Migrated from rt.perl.org#127316 (status was 'rejected')

Searchable as RT127316$

[p5pRT]

From bitcard@jatu.online

This happens for example when using perlbrew. The build is done as a regular user.

The copy-test copies files with various umasks, removes the file and continues. However, when running as non-root, it is not possible to remove file​:
---------- 1 ep ep 0 Jan 19 12​:34 copy1-129533

Also, it won't return any errors on unlink().

This is a non-issue when running as root. A root can remove such files. As regular user, if the file is owned by the user (as it is in this case), a chmod u+w and then unlink() will succeed.

[p5pRT]

From @tonycoz

On Tue Jan 19 06​:10​:43 2016, bitcard@​jatu.online wrote​:

This happens for example when using perlbrew. The build is done as a
regular user.

The copy-test copies files with various umasks, removes the file and
continues. However, when running as non-root, it is not possible to
remove file​:
---------- 1 ep ep 0 Jan 19 12​:34 copy1-129533

Also, it won't return any errors on unlink().

This is a non-issue when running as root. A root can remove such
files. As regular user, if the file is owned by the user (as it is in
this case), a chmod u+w and then unlink() will succeed.

Hi,

I regularly run perl's tests, including that one as non-root on several systems (Linux, darwin, NetBSD, Solaris).

You haven't provided any of the following useful information​:

- which test in particular is failing,
- the operating system you're using (including the distribution
  name and it's release number), or
- the perl version you're building.

Please provide that information and maybe we can see what's happening.

You might want to provide a verbose run of the test script​:

  make TEST_ARGS=-v TEST_FILES=../lib/File/Copy.t

Tony

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From bitcard@jatu.online

On Tue Jan 19 15​:10​:27 2016, tonyc wrote​:

On Tue Jan 19 06​:10​:43 2016, bitcard@​jatu.online wrote​:

This happens for example when using perlbrew. The build is done as a
regular user.

The copy-test copies files with various umasks, removes the file and
continues. However, when running as non-root, it is not possible to
remove file​:
---------- 1 ep ep 0 Jan 19 12​:34 copy1-129533

Also, it won't return any errors on unlink().

This is a non-issue when running as root. A root can remove such
files. As regular user, if the file is owned by the user (as it is in
this case), a chmod u+w and then unlink() will succeed.

Hi,

I regularly run perl's tests, including that one as non-root on
several systems (Linux, darwin, NetBSD, Solaris).

You haven't provided any of the following useful information​:

- which test in particular is failing,
- the operating system you're using (including the distribution
name and it's release number), or
- the perl version you're building.

Please provide that information and maybe we can see what's happening.

You might want to provide a verbose run of the test script​:

make TEST_ARGS=-v TEST_FILES=../lib/File/Copy.t

Tony

Sir, you acknowledged running "that test" and are claiming, that I didn't mention which test is failing. The test failing is​: lib/File/Copy.t

I'm running on a Ubuntu 14.04.1 LTS and am installing a perl-5.20.3.

Here is a test run​:
$ make test TEST_ARGS=-v TEST_FILES=../lib/File/Copy.t
...
cd t && (rm -f perl; /bin/ln -s ../perl perl)
./runtests choose
t/../lib/File/Copy ... 1..466
ok 1 - 'copy()' is a usage error
ok 2 - 'copy('arg')' is a usage error
ok 3 - 'copy('arg', 'arg', 'arg', 'arg')' is a usage error
ok 4 - 'move()' is a usage error
ok 5 - 'move('arg')' is a usage error
ok 6 - 'move('arg', 'arg', 'arg')' is a usage error
...
ok 356 - Permission bits set correctly
ok 357 - Permission bits set correctly
ok 358 - Permission bits set correctly
ok 359 - Permission bits not modified
ok 360 - Permission bits not modified
ok 361 - Permission bits set correctly
ok 362 - Permission bits set correctly
ok 363 - Permission bits set correctly
ok 364 - Permission bits set correctly
ok 365 - Permission bits not modified
ok 366 - Permission bits not modified
Can't open /perl5/perlbrew/build/perl-5.20.3/t/copy1-70189​: Permission denied at ../lib/File/Copy.t line 327.
# Looks like you planned 466 tests but ran 366.
# Looks like your test exited with 13 just after 366.
FAILED--expected 466 tests, saw 366
Failed 1 test out of 1, 0.00% okay.
  ../lib/File/Copy.t

Does that help?

Regards,
Jari Turkia

[p5pRT]

From zefram@fysh.org

Jari Turkia via RT wrote​:

I'm running on a Ubuntu 14.04.1 LTS and am installing a perl-5.20.3.
...
Can't open /perl5/perlbrew/build/perl-5.20.3/t/copy1-70189​: Permission denied at ../lib/File/Copy.t line 327.

There's something funny going on with that line number. Line 327 of that
file in 5.20.3 is a line that performs the chmod, and cannot itself be
the location in a diagnostic.

Anyway, the "Can't open" message indicates that the failing operation is
opening the file for writing (line 325). If unlink failed, the message
would be "Can't unlink". That misapprehension made your bug report
initially quite bizarre, because on Unix permissions on the file don't
impede unlinking it.

-zefram

[p5pRT]

From bitcard@jatu.online

On Wed Jan 20 06​:20​:09 2016, zefram@​fysh.org wrote​:

Jari Turkia via RT wrote​:

I'm running on a Ubuntu 14.04.1 LTS and am installing a perl-5.20.3.
...
Can't open /perl5/perlbrew/build/perl-5.20.3/t/copy1-70189​:
Permission denied at ../lib/File/Copy.t line 327.

There's something funny going on with that line number. Line 327 of
that
file in 5.20.3 is a line that performs the chmod, and cannot itself be
the location in a diagnostic.

Anyway, the "Can't open" message indicates that the failing operation
is
opening the file for writing (line 325). If unlink failed, the
message
would be "Can't unlink". That misapprehension made your bug report
initially quite bizarre, because on Unix permissions on the file don't
impede unlinking it.

-zefram

Sir, you are correct about the line number. I had modified the file to debug the issue. Correct error with original file is (pretty much what you said)​:
Can't open copy1-53250​: Permission denied at ../lib/File/Copy.t line 325.

This is the strace of the test​:
write(6, "ok 366 - Permission bits not mod"..., 38) = 38
close(3) = 0
stat("copy1-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
lstat("copy1-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
unlink("copy1-63677") = 0
stat("copy2-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
lstat("copy2-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
unlink("copy2-63677") = 0
stat("copy4-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
lstat("copy4-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
unlink("copy4-63677") = 0
stat("copy5-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
lstat("copy5-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
unlink("copy5-63677") = 0
mkdir("copy1-63677", 0777) = 0
chmod("copy1-63677", 01751) = 0
rmdir("copy1-63677") = 0
open("copy1-63677", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1 EACCES (Permission denied)
write(2, "Can't open copy1-63677​: Permissi"..., 74) = 74

After the test​:
$ ls -l t/copy1-63677
---------- 1 ep ep 0 Jan 20 18​:46 t/copy1-63677

The file is there. However, the sequence doesn't make any sense. First it unlinks the file, then it creates a directory with same name, removes it and after all that, the file still exists. Huh!?

[p5pRT]

From zefram@fysh.org

Jari Turkia via RT wrote​:

rmdir("copy1-63677") = 0
open("copy1-63677", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1 EACCES (Permission denied)
write(2, "Can't open copy1-63677​: Permissi"..., 74) = 74

After the test​:
$ ls -l t/copy1-63677
---------- 1 ep ep 0 Jan 20 18​:46 t/copy1-63677

That is a very strange sequence. Regardless of umask, creating the
file should not fail there. The rmdir shows that you can write to the
directory, and that the name should be free. Guessing a bit, it looks
as though the open call has actually created the file and then failed
with EACCES. What kind of filesystem are you doing this on? If this
were to happen just once, I could explain it as a failure of idempotence
in NFS​: if the open were to succeed but the response be lost, then the
open retried, we'd see this result. But if it's happening reliably
that's a different matter. Perhaps something in the filesystem layer
is consistently taking two attempts at the operation?

The chmod immediately preceding the rmdir suggests that this is the second
time round the test script's umask 0777 test set. The umask handling is
a bit funny​: I'm not clear what the intent is, but the behaviour is that
umask isn't set to the selected value until after this part of the test
set, creating and removing the directory and creating the regular file.
The umask isn't restored until the very end, so this part of each test
set operates with the umask setting of the previous iteration. So this
second iteration of the umask 0777 test set is actually the first time
that this regular file creation runs with umask 0777. It does seem to
be the case that the umask-altered file permissions are being applied
to the open operation, causing the EACCES.

-zefram

[p5pRT]

From @Abigail

On Wed, Jan 20, 2016 at 10​:52​:35AM -0800, Jari Turkia via RT wrote​:

On Wed Jan 20 06​:20​:09 2016, zefram@​fysh.org wrote​:

Jari Turkia via RT wrote​:

I'm running on a Ubuntu 14.04.1 LTS and am installing a perl-5.20.3.
...
Can't open /perl5/perlbrew/build/perl-5.20.3/t/copy1-70189​:
Permission denied at ../lib/File/Copy.t line 327.

There's something funny going on with that line number. Line 327 of
that
file in 5.20.3 is a line that performs the chmod, and cannot itself be
the location in a diagnostic.

Anyway, the "Can't open" message indicates that the failing operation
is
opening the file for writing (line 325). If unlink failed, the
message
would be "Can't unlink". That misapprehension made your bug report
initially quite bizarre, because on Unix permissions on the file don't
impede unlinking it.

-zefram

Sir, you are correct about the line number. I had modified the file to debug the issue. Correct error with original file is (pretty much what you said)​:
Can't open copy1-53250​: Permission denied at ../lib/File/Copy.t line 325.

This is the strace of the test​:
write(6, "ok 366 - Permission bits not mod"..., 38) = 38
close(3) = 0
stat("copy1-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
lstat("copy1-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
unlink("copy1-63677") = 0
stat("copy2-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
lstat("copy2-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
unlink("copy2-63677") = 0
stat("copy4-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
lstat("copy4-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
unlink("copy4-63677") = 0
stat("copy5-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
lstat("copy5-63677", {st_mode=S_IFREG, st_size=0, ...}) = 0
unlink("copy5-63677") = 0
mkdir("copy1-63677", 0777) = 0
chmod("copy1-63677", 01751) = 0
rmdir("copy1-63677") = 0
open("copy1-63677", O_WRONLY|O_CREAT|O_TRUNC, 0666) = -1 EACCES (Permission denied)
write(2, "Can't open copy1-63677​: Permissi"..., 74) = 74

After the test​:
$ ls -l t/copy1-63677
---------- 1 ep ep 0 Jan 20 18​:46 t/copy1-63677

The file is there. However, the sequence doesn't make any sense. First it unlinks the file, then it creates a directory with same name, removes it and after all that, the file still exists. Huh!?

This is a long shot, but could there be some ACL in effect? Even if standard
permissions says you have access, an ACL may still prevent it.

Abigail

[p5pRT]

From bitcard@jatu.online

On Thu Jan 21 06​:37​:34 2016, zefram@​fysh.org wrote​:

that's a different matter. Perhaps something in the filesystem layer
is consistently taking two attempts at the operation?

I checked ACLs and anything that I could think of. Then I found it!

The bug is reproducible every single time on one box only. That one has /home mounted over NFS. On an identical Ubuntu running XFS, /home as a directory on the root-partition, the test succeeds. It was a filesystem issue after all.

Regards,
Jari Turkia

[p5pRT]

From @wolfsage

On Fri, Jan 22, 2016 at 3​:43 AM, Jari Turkia via RT <
perlbug-followup@​perl.org> wrote​:

I checked ACLs and anything that I could think of. Then I found it!

The bug is reproducible every single time on one box only. That one has
/home mounted over NFS. On an identical Ubuntu running XFS, /home as a
directory on the root-partition, the test succeeds. It was a filesystem
issue after all.

Should this be closed then? Or is this NFS issue something we need to look
into?

Thanks,

-- Matthew Horsfall (alh)

[p5pRT]

From zefram@fysh.org

Matthew Horsfall (alh) wrote​:

Should this be closed then? Or is this NFS issue something we need to look
into?

It should be closed. We can't expect to allow for filesystem misbehaviour
that so seriously breaks syscall semantics.

-zefram

[p5pRT]

From @wolfsage

Closing then.

[p5pRT]

@wolfsage - Status changed from 'open' to 'rejected'