PPPort breaks the build from sources of 5.32 on Ubuntu18.04 (WSLv1)

[vadimkantorov]

I'm running the following script on Ubuntu 18.04 / WSLv1 / case-sensitive mount point. It produces the following error:

chmod 755 ../../lib/auto/Devel/PPPort/PPPort.so
"../../miniperl" "-I../../lib" "-I../../lib" mktests.PL t/01_test.t
t/01_test.t: Permission denied
generating t/01_test.t
Makefile:550: recipe for target 't/01_test.t' failed
make[1]: *** [t/01_test.t] Error 13
make[1]: Leaving directory '/home/vadimkantorov/wipbiber/perl/dist/Devel-PPPort'
make[1]: Entering directory '/home/vadimkantorov/wipbiber/perl/dist/Devel-PPPort'
/home/vadimkantorov/wipbiber/perl/dist/Devel-PPPort/../../miniperl "-I../../lib" -MExtUtils::Command::MM -e 'cp_nonempty' -- PPPort.bs ../../lib/auto/Devel/PPPort/PPPort.bs 644
"../../miniperl" "-I../../lib" "-I../../lib" mktests.PL t/01_test.t
t/01_test.t: Permission denied
generating t/01_test.t
Makefile:550: recipe for target 't/01_test.t' failed
make[1]: *** [t/01_test.t] Error 13
make[1]: Leaving directory '/home/vadimkantorov/wipbiber/perl/dist/Devel-PPPort'
Unsuccessful make(dist/Devel-PPPort): code=512 at make_ext.pl line 588.
makefile:576: recipe for target 'lib/auto/Devel/PPPort/PPPort.so' failed
make: *** [lib/auto/Devel/PPPort/PPPort.so] Error 25

PERL=https://www.cpan.org/src/5.0/perl-5.32.0.tar.gz
EXTR=perl
PREFIX=$PWD/perlprefix

mkdir -p $PREFIX
wget -nc $PERL

mkdir -p perl_
tar -xf $(basename $PERL) --strip-components=1 --directory=perl_
rm -rf $EXTR
cp -r perl_ $EXTR

pushd $EXTR
bash +x ./Configure -sde -Dprefix=$PREFIX
make
#make install
popd

Full log: log.zip

P.S. Can one also configure perl / modules build to skip tests?

Thank you!

[atoomic]

This is strange, I'm not sure why the test need to be executable. @khwilliamson any ideas what went wrong there?

Note that since v5.33.1 dynamic tests from PPPort are not shipped anymore and this file t/01_test.t is not part of the tarball anymore.

[khwilliamson]

No clue. @pali any ideas?

[pali]

I guess that problem is with write permissions. mktests.PL cannot create file t/01_test.t. I have already seen similar issues not related to Perl and it looks like a bug in WSL.

You could try to add permissions: chmod -R a+rw perl

[vadimkantorov]

It generates the object files in-tree without a problem, so it should have write permissions in perl sources directory. But I'll try this out.

[vadimkantorov]

Note that since v5.33.1 dynamic tests from PPPort are not shipped anymore and this file t/01_test.t is not part of the tarball anymore.

The same error is produced on v5.33.1.

after chown and re-launching make, everything works. but this is quite a strange bug.

[pali]

As I said, I have already seen this problem in WSL and I suspect that there is a bug in WSL permission layer.

[vadimkantorov]

Maybe. Just for information, before the build the permissions seem identical:

vadimkantorov@DESKTOP-4UF8FID:~/wipbiber$ ls -lah perl
total 21M
drwxr-xr-x 1 vadimkantorov vadimkantorov 4.0K Oct 19 19:45 .

...

drwxr-xr-x 1 vadimkantorov vadimkantorov 4.0K Oct 19 19:45 t

I wonder if permissions get changed somehow. I'll see if they change after the build

[vadimkantorov]

What happens is:

cd perl/dist/Devel-PPPort # working directory of failing line
"../../miniperl" "-I../../lib" "-I../../lib" mktests.PL t/01_test.t # the failing line
# generating t/01_test.t
# t/01_test.t: Permission denied

From strace:

openat(AT_FDCWD, "t/01_test.t", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = -1 EACCES (Permission denied)
write(2, "t/01_test.t: Permission denied\n", 31t/01_test.t: Permission denied

Indeed, this fails as well: python3 -c 'import os; os.open("t/01_test.t", os.O_WRONLY | os.O_CREAT | os.O_TRUNC | os.O_CLOEXEC, 0o666)'

[vadimkantorov]

This fails as well: python3 -c 'import os; os.open("t/01_test.t", os.O_TRUNC , 0o666)'

[vadimkantorov]

This looks strange enough, reporting to WSL, let's see

[pali]

From strace:

openat(AT_FDCWD, "t/01_test.t", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = -1 EACCES (Permission denied)
write(2, "t/01_test.t: Permission denied\n", 31t/01_test.t: Permission denied

Exactly same issue which I have already seen in other projects.

This looks strange enough, reporting to WSL, let's see

Let us know if you get some information. Thanks!

[vadimkantorov]

A-ha! I found that this file is somehow write-protected:

By that point where it is generated:

vadimkantorov@DESKTOP-4UF8FID:~/wipbiber/perl/dist/Devel-PPPort$ rm t/01_test.t
rm: remove write-protected regular file 't/01_test.t'

[vadimkantorov]

That's why WSL doesn't want to truncate or open it for writing

[vadimkantorov]

Why would it be write-protected?

[vadimkantorov]

It seems that the t directory is write-protected, and files generated there-in are write-protected as well. So rewriting or re-generating them fails.

[vadimkantorov]

Yep, basically Perl build process wants to open as write-only/truncate already existing readonly files. Why is that happening?

Is it still a WSL's problem that it fails? Is it supposed to work in Linux "proper"? To me it looks a correct behavior

[brodafly]

Hi. I just wanted to add that I'm experiencing the exact same issue when building perl 5.32.0 on HP-UX (IA-64).

[tonycoz]

I don't see the problem working with the tar extracted tree, but that has the same modification time for mktests.PL and t/01_test.t and I think all of the other dependencies for t/01_test.t

../dist/Devel-PPPort/t/01_test.t 1592175685
../dist/Devel-PPPort/mktests.PL 1592175685

I expect your copy of the tree is giving one or more of those dependencies a later modification time than t/01_test.t and so the make rule to rebuild the tests is run.

Do you still see the problem if you "touch" dist/Devel-PPPort/t/01_test.t after the copy and before the make?

If this does fix the problem I think it's a case of the build process being a bit too fragile in the tarball, and should be fixed at our end, possibly by only having the rule to regenerate the tests if .git exists.

[khwilliamson]

@vadimkantorov @brodafly

Did you try out this suggestion? If not could you?

[davvid]

I ran into this same issue while building perl 5.32.1 from a tarball. I can confirm that the following workaround avoids the issue when invoked before running make:

touch dist/Devel-PPPort/t/01_test.t

[tonycoz]

I believe this was fixed by c8799af which removed the test files from the tarball, so they are always generated and are writable.

Do you see this problem with 5.34 and later?

[brodafly]

Personally I am no longer able to try and reproduce the issue, so I can't give you relevant input whether or not it persists.

[vadimkantorov]

Same for me - I moved on from this, I think it stopped reproducing at some point, but I didn't investigate why. So I'll close this issue. But if anyone stumbles on it again, please mention me an I will reopen it again!