Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Created by email@example.com
When building perl with highly parallel options (e.g. -j 32) on a machine
The scripts pod2man, pod2text, podchecker, podselect, and pod2usage all use
From 26cf06121b03ef60fd49af6f0f56e743b332f6b0 Mon Sep 17 00:00:00 2001 From: Dan Dedrick <firstname.lastname@example.org> Date: Thu, 12 Oct 2017 21:49:39 -0400 Subject: [PATCH] Fix missing build dependency for pods When building perl with highly parallel options (e.g. -j 32) on a machine with many cores (e.g. 32) ocassionaly it fails with the following type of error. make: Entering directory '.../cpan/podlators' Can't locate Getopt/Long.pm in @INC (you may need to install the Getopt::Long module) (@INC contains: .../cpan/AutoLoader/lib .../dist/Carp/lib .../dist/PathTools .../dist/PathTools/lib .../cpan/ExtUtils-Install/lib .../cpan/ExtUtils-MakeMaker/lib .../cpan/ExtUtils-Manifest/lib .../cpan/File-Path/lib .../ext/re .../dist/Term-ReadLine/lib .../dist/Exporter/lib .../ext/File-Find/lib .../cpan/Text-Tabs/lib .../dist/constant/lib .../cpan/version/lib .../lib ../../lib .) at .../cpan/ExtUtils-MakeMaker/lib/ExtUtils/Command/MM.pm line 109. Makefile:445: recipe for target 'manifypods' failed make: *** [manifypods] Error 2 The scripts pod2man, pod2text, podchecker, podselect, and pod2usage all use Getopt-Long and since they are all part of nonxs modules this needs to be added here to prevent these build races. --- write_buildcustomize.pl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/write_buildcustomize.pl b/write_buildcustomize.pl index 8666a6bce2..7a55405029 100644 --- a/write_buildcustomize.pl +++ b/write_buildcustomize.pl @@ -39,6 +39,7 @@ my @toolchain = qw(cpan/AutoLoader/lib cpan/Text-Tabs/lib dist/constant/lib cpan/version/lib + cpan/Getopt-Long/lib ); # Text-ParseWords used only in ExtUtils::Liblist::Kid::_win32_ext() @@ -47,7 +48,6 @@ my @toolchain = qw(cpan/AutoLoader/lib push @toolchain, qw( cpan/Text-ParseWords/lib dist/ExtUtils-ParseXS/lib - cpan/Getopt-Long/lib cpan/parent/lib cpan/ExtUtils-Constant/lib ) if $^O eq 'MSWin32'; -- 2.13.6
On Fri, 27 Oct 2017 00:47:58 GMT, email@example.com wrote:
Can you explain a bit more as to what you think is the sequence of events involved in the race condition?
Thank you very much.
On Fri, 27 Oct 2017 17:28:22 -0700, jkeenan wrote:
If cpan/podlators starts building before cpan/Getopt-Long has finished it will attempt to run the pod2man, which contains "require Getopt::Long;" before Getopt-Long has been published to the lib dir. Therefore when it runs it searches its path and finds no GetOpt-Long and fails. This can naturally happen when running a very parallel make invocation (-j 32) but it is still fairly rare. It can be artificially induced by reordering the nonxs_ext variable such that cpan/podlators/pm_to_blib is before cpan/Getopt-Long/pm_to_blib (this needs to be done in a fresh workspace right after the Configure is run).
On Fri, 27 Oct 2017 18:40:10 -0700, firstname.lastname@example.org wrote:
I have also seen this problem for perl 5.26.1 on a parallel build with -j33 on a machine with 32 cores.
With this above reproduction hack, the problem can even be reproduced with '-j5'.
--- output/build/host-perl-5.26.1/Makefile 2018-06-06 15:18:34.905011014 +0200 +++ /tmp/Makefile.host 2018-06-06 15:19:01.200009936 +0200 @@ -583,6 +583,11 @@ $(MINIPERL) make_ext.pl $@ $(MAKE_EXT_ARGS) MAKE="$(MAKE)" LIBPERL_A=$(LIBPERL) LINKTYPE=static $(STATIC_LDFLAGS) n_dummy $(nonxs_ext): $(MINIPERL_EXE) lib/buildcustomize.pl preplibrary FORCE + @echo "------- makefile -$@-" + @if [ "$@" = "cpan/Getopt-Long/pm_to_blib" ]; then \ + echo "---- getopt" ; \ + sleep 60 ; \ + fi $(MINIPERL) make_ext.pl $@ $(MAKE_EXT_ARGS) MAKE="$(MAKE)" LIBPERL_A=$(LIBPERL) ext/Pod-Functions/pm_to_blib: cpan/Pod-Simple/pm_to_blib cpan/Pod-Escapes/pm_to_blib pod/perlfunc.pod
Using the patch provided by Dan Dedrick, the problem is solved.
Could this patch please be applied to official releases?
Thomas De Schampheleire via RT <email@example.com> wrote:
Thanks, applied (with an extra comment, and Dan Dedrick added to the
Welcome to the roster of Perl authors, Dan!
Thank you for filing this report. You have helped make Perl better.
With the release today of Perl 5.30.0, this and 160 other issues have been
Perl 5.30.0 may be downloaded via:
If you find that the problem persists, feel free to reopen this ticket.