Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PATCH] fix parallel ALL_STATIC Win32 build #16492

Open
p5pRT opened this issue Apr 6, 2018 · 9 comments
Open

[PATCH] fix parallel ALL_STATIC Win32 build #16492

p5pRT opened this issue Apr 6, 2018 · 9 comments

Comments

@p5pRT
Copy link

@p5pRT p5pRT commented Apr 6, 2018

Migrated from rt.perl.org#133070 (status was 'open')

Searchable as RT133070$

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Apr 6, 2018

From @bulk88

Created by @bulk88

See attached patch. This is intended for 5.28. Its a follow on to
problems I discussed in
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132992 . It fixes not being
able to build ALL_STATIC Extensions_static target individually and
relying on a race/adjacent targets to randomly finish first to make
Extensions_static not fail.

example of failed static Normalize compile because the mktables data
wasn't generated yet
----------------------------
cl -c -nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32 -D_CONSOLE
-DNO_STRICT
-DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS
-DWIN32_NO_R
EGISTRY -DUSE_PERLIO -D_USE_32BIT_TIME_T -DPERLDLL -O1 -MD -Zi -DNDEBUG
-GL -D
VERSION=\"1.25\" -DXS_VERSION=\"1.25\" "-I..\..\lib\CORE" Collate.c
Collate.c
C​:\perl521\srcnew\miniperl.exe "-I..\..\lib" -MExtUtils​::Command -e rm_f
-- "..\
..\lib\auto\Unicode\Collate\Collate.lib"
lib -ltcg -out​:..\..\lib\auto\Unicode\Collate\Collate.lib Collate.obj
Microsoft (R) Library Manager Version 7.10.6030
Copyright (C) Microsoft Corporation. All rights reserved.

C​:\perl521\srcnew\miniperl.exe "-I..\..\lib" -MExtUtils​::Command -e
chmod -- 755
  ..\..\lib\auto\Unicode\Collate\Collate.lib
gmake[1]​: Leaving directory 'C​:/perl521/srcnew/cpan/Unicode-Collate'
Making header files for XS...
Unicode​::Normalize, mkheader​: CombiningClass.pl not found at ./mkheader
line 67.

  require ./mkheader called at Makefile.PL line 11
Unsuccessful Makefile.PL(dist/Unicode-Normalize)​: code=512 at
..\make_ext.pl lin
e 518.
GNUmakefile​:1584​: recipe for target 'Extensions_static' failed
gmake​: *** [Extensions_static] Error 2
----------------------------

example of static extension failing because lib.pm not finished yet by a
race

----------------------------
gmake[1]​: Leaving directory 'C​:/perl521/srcnew/ext/B'
Can't locate lib.pm in @​INC (you may need to install the lib module)
(@​INC conta
ins​: C​:\perl521\srcnew\lib C​:\perl521\srcnew\cpan\AutoLoader\lib
C​:\perl521\srcn
ew\dist\Carp\lib C​:\perl521\srcnew\dist\PathTools
C​:\perl521\srcnew\dist\PathToo
ls\lib C​:\perl521\srcnew\cpan\ExtUtils-Install\lib
C​:\perl521\srcnew\cpan\ExtUti
ls-MakeMaker\lib C​:\perl521\srcnew\cpan\ExtUtils-Manifest\lib
C​:\perl521\srcnew\
cpan\File-Path\lib C​:\perl521\srcnew\ext\re
C​:\perl521\srcnew\dist\Term-ReadLine
\lib C​:\perl521\srcnew\dist\Exporter\lib
C​:\perl521\srcnew\ext\File-Find\lib C​:\
perl521\srcnew\cpan\Text-Tabs\lib C​:\perl521\srcnew\dist\constant\lib
C​:\perl521
\srcnew\cpan\version\lib C​:\perl521\srcnew\cpan\Text-ParseWords\lib
C​:\perl521\s
rcnew\dist\ExtUtils-ParseXS\lib C​:\perl521\srcnew\cpan\Getopt-Long\lib
C​:\perl52
1\srcnew\cpan\parent\lib C​:\perl521\srcnew\cpan\ExtUtils-Constant\lib .)
at Make
file.PL line 6.
BEGIN failed--compilation aborted at Makefile.PL line 6.
Unsuccessful Makefile.PL(cpan/Compress-Raw-Bzip2)​: code=512 at
..\make_ext.pl li
ne 518.
GNUmakefile​:1584​: recipe for target 'Extensions_static' failed
gmake​: *** [Extensions_static] Error 2
----------------------------

Perl Info
---
Flags:
             category=core
             severity=low
---
Site configuration information for perl 5.27.9:

Configured by Administrator at Tue Jan 30 20:34:30 2018.

Summary of my perl5 (revision 5 version 27 subversion 9) configuration:

           Platform:
             osname=MSWin32
             osvers=5.2.3790
             archname=MSWin32-x86-multi-thread
             uname=''
             config_args='undef'
             hint=recommended
             useposix=true
             d_sigaction=undef
             useithreads=define
             usemultiplicity=define
             use64bitint=undef
             use64bitall=undef
             uselongdouble=undef
             usemymalloc=n
             default_inc_excludes_dot=define
             bincompat5005=undef
           Compiler:
             cc='cl'
             ccflags ='-nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32
-D_CONSOLE -DNO_STRICT -D_CRT_SECURE_NO_DEPRECATE
-D_CRT_NONSTDC_NO_DEPRECATE  -DPERL_TEXTMODE_SCRIPTS
-DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DWIN32_NO_REGISTRY'
             optimize='-O1 -MD -Zi -DNDEBUG -GL'
             cppflags='-DWIN32'
             ccversion='15.00.30729.01'
             gccversion=''
             gccosandvers=''
             intsize=4
             longsize=4
             ptrsize=4
             doublesize=8
             byteorder=1234
             doublekind=3
             d_longlong=undef
             longlongsize=8
             d_longdbl=define
             longdblsize=8
             longdblkind=0
             ivtype='long'
             ivsize=4
             nvtype='double'
             nvsize=8
             Off_t='__int64'
             lseeksize=8
             alignbytes=8
             prototype=define
           Linker and Libraries:
             ld='link'
             ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -ltcg
-libpath:"c:\perl\lib\CORE"        -machine:x86'
             libpth="C:\Program Files (x86)\Microsoft Visual Studio
9.0\VC\lib"
             libs=oldnames.lib kernel32.lib user32.lib gdi32.lib 
winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib
odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib
             perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib
winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib
odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib
             libc=msvcrt.lib
             so=dll
             useshrplib=true
             libperl=perl527.lib
             gnulibc_version=''
           Dynamic Linking:
             dlsrc=dl_win32.xs
             dlext=dll
             d_dlsymun=undef
             ccdlflags=' '
             cccdlflags=' '
             lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -ltcg
             -libpath:"c:\perl\lib\CORE"        -machine:x86'


---
@INC for perl 5.27.9:
             lib
             C:/p527/srcnew/lib

---
Environment for perl 5.27.9:
             CYGWIN=tty
             HOME (unset)
             LANG (unset)
             LANGUAGE (unset)
             LD_LIBRARY_PATH=/usr/lib/x86:/usr/X11R6/lib
             LOGDIR (unset)
             PATH=C:\WINDOWS\system32;C:\Program Files (x86)\Microsoft 
Visual
Studio 9.0\VC\BIN;C:\Program Files\Microsoft
SDKs\Windows\v6.0A\bin;C:\Perl\bin;C:\WINDOWS;C:\Program Files
(x86)\Microsoft Visual Studio 9.0\Common7\IDE;C:\Program Files
(x86)\Git\bin;C:\sp3220\c\bin;
             PERL_BADLANG (unset)
             SHELL (unset)

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Apr 6, 2018

From @bulk88

0001-fix-parallel-ALL_STATIC-Win32-build.patch
From 6259ddde8c36041fc830030571c8ad7abe524b78 Mon Sep 17 00:00:00 2001
From: Daniel Dragan <bulk88@hotmail.com>
Date: Wed, 4 Apr 2018 09:20:38 -0400
Subject: [PATCH] fix parallel ALL_STATIC Win32 build

Normalize depends on mktables's output. Mktables is a very long serial CPU
intensive build product. No other modules except Normalize depend on
mktables. Normalize XS module must be split out from all other XS modules
to fill CPU cores with parallel work, so that the 2 longest running
tagets, Mktables and all XS modules, run in parallel instead of 1 after
another. Since Normalize was hardcoded as a dynamic module, in an
ALL_STATIC build, Normalize was build twice, once as a static in the
big Extensions_static target, and once as a dynamic module, a race
happened if a parallel build was done, and if the Extensions_static target
got around to trying to build Normalize static, before by chance Normalize
dyn was build, Normalize static failed. This can be demostrated by cleaning
the src tree and building each target individually explicitly. Fix the race
by creating a Normalize static and dynamic target and dropout the deps
depending on if its ALL_STATIC or not. Also the now "big" Extensions_static
target when doing an ALL_STATIC build also failed because the general XS
modules that need lib.pm to build didn't have lib.pm available when their
Makefile.PL'es were run. For example cpan/Compress-Raw-Bzip2 needs lib.pm
to build. So separate out lib.pm from Dynaloader target so both dyn
extensions and static exts have lib.pm available. Technically, static exts
dont need lib.pm unless its ALL_STATIC, since for dyn win32 perl Win32CORE
is the only static module, but lib.pm is relativly fast and easy and low
dep to build so build it anyways for static ext target in non-ALL_STATIC
mode rather than special case the dep between ALL_STATIC and
non-ALL_STATIC.

-re.pm doesn't need lib.pm
-normalize doesn't need lib.pm

Reverts "Fix static builds with MinGW" commit 9999704e7ac and adds fixes.
This is a follow on to [perl #132992].

To reduce the diff-ness between the dmake and gmake mkfs. Make the
distclean identical. For dmkf, dont delete Storable.pm twice. And for dmfk
remove Amiga dir deletion since the gmkf never had that line since day 1
of gmkf in commit 342634f3c8 "kmx's original GNUmakefile". In gmfk move
Storable.pm line to match dmfk which is more psuedo-alpha sorted. Also fix
the Test vs Test2 typo.

Some more text is available in the ticket associated with this patch.
---
 win32/GNUmakefile | 30 ++++++++++++++++++++----------
 win32/makefile.mk | 28 ++++++++++++++++++----------
 2 files changed, 38 insertions(+), 20 deletions(-)

diff --git a/win32/GNUmakefile b/win32/GNUmakefile
index 5e70d04..bcb2e85 100644
--- a/win32/GNUmakefile
+++ b/win32/GNUmakefile
@@ -1109,11 +1109,13 @@ ifeq ($(ALL_STATIC),define)
 # some exclusions, unfortunately, until fixed:
 #  - MakeMaker isn't capable enough for SDBM_File (small bug)
 STATIC_EXT	= * !SDBM_File
+NORMALIZE_STATIC = Normalize_static
 else
 # specify static extensions here, for example:
 # (be sure to include Win32CORE to load Win32 on demand)
 #STATIC_EXT	= Win32CORE Cwd Compress/Raw/Zlib
 STATIC_EXT	= Win32CORE
+NORMALIZE_DYN	= Normalize_dyn
 endif
 
 DYNALOADER	= ..\DynaLoader$(o)
@@ -1571,25 +1573,33 @@ endif
 # DynaLoader.pm, so this will have to do
 
 #most of deps of this target are in DYNALOADER and therefore omitted here
-Extensions : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) MakePPPort
+Extensions : $(PERLDEP) $(DYNALOADER) Extension_lib $(GLOBEXE) MakePPPort
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic !Unicode/Normalize
 
-Extensions_normalize : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
+Normalize_static : $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) $(UNIDATAFILES)
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static +Unicode/Normalize
+
+Normalize_dyn : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +Unicode/Normalize
 
 Extensions_reonly : $(PERLDEP) $(DYNALOADER)
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +re
 
-Extensions_static : ..\make_ext.pl list_static_libs.pl $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
-	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static
+Exts_static_general : ..\make_ext.pl $(CONFIGPM) Extension_lib $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static !Unicode/Normalize
+
+Extensions_static : list_static_libs.pl Exts_static_general $(NORMALIZE_STATIC)
 	$(MINIPERL) -I..\lib list_static_libs.pl > Extensions_static
 
 Extensions_nonxs : ..\make_ext.pl ..\pod\perlfunc.pod $(CONFIGPM) $(GLOBEXE)
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --nonxs !libs
 
+Extension_lib : ..\make_ext.pl $(CONFIGPM)
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) lib
+
 #lib must be built, it can't be buildcustomize.pl-ed, and is required for XS building
 $(DYNALOADER) : ..\make_ext.pl $(CONFIGPM) $(HAVE_COREDIR)
-	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader lib
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader
 
 Extensions_clean :
 	-if exist $(MINIPERL) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --all --target=clean
@@ -1611,9 +1621,9 @@ PostExt : ..\lib\Storable\Limit.pm
 # be running in parallel like UNIDATAFILES, this target a placeholder for the
 # future
 ifeq ($(PERLSTATIC),static)
-rebasePE : Extensions_normalize Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
+rebasePE : Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
 else
-rebasePE : Extensions $(PERLDLL) Extensions_normalize $(PERLEXE)
+rebasePE : Extensions $(PERLDLL) $(NORMALIZE_DYN) $(PERLEXE)
 endif
 	$(NOOP)
 
@@ -1669,7 +1679,7 @@ utils: $(HAVEMINIPERL) ..\utils\Makefile
 	$(MINIPERL) -I..\lib ..\autodoc.pl ..
 	$(MINIPERL) -I..\lib ..\pod\perlmodlib.PL -q ..
 
-..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs Extensions_normalize utils
+..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs $(NORMALIZE_DYN) utils
 	$(PERLEXE) -f ..\pod\buildtoc -q
 
 # Note that the pod cleanup in this next section is parsed (and regenerated
@@ -1686,11 +1696,11 @@ distclean: realclean
 	-del /f $(LIBDIR)\SDBM_File.pm $(LIBDIR)\Socket.pm $(LIBDIR)\POSIX.pm
 	-del /f $(LIBDIR)\B.pm $(LIBDIR)\O.pm $(LIBDIR)\re.pm
 	-del /f $(LIBDIR)\File\Glob.pm
-	-del /f $(LIBDIR)\Storable.pm $(LIBDIR)\Storable\Limit.pm
 	-del /f $(LIBDIR)\Sys\Hostname.pm
 	-del /f $(LIBDIR)\Time\HiRes.pm
 	-del /f $(LIBDIR)\Unicode\Normalize.pm
 	-del /f $(LIBDIR)\Math\BigInt\FastCalc.pm
+	-del /f $(LIBDIR)\Storable.pm $(LIBDIR)\Storable\Limit.pm
 	-del /f $(LIBDIR)\Win32.pm
 	-del /f $(LIBDIR)\Win32CORE.pm
 	-del /f $(LIBDIR)\Win32API\File.pm
@@ -1749,7 +1759,7 @@ distclean: realclean
 	-if exist $(LIBDIR)\TAP rmdir /s /q $(LIBDIR)\TAP
 	-if exist $(LIBDIR)\Term rmdir /s /q $(LIBDIR)\Term
 	-if exist $(LIBDIR)\Test rmdir /s /q $(LIBDIR)\Test
-	-if exist $(LIBDIR)\Test2 rmdir /s /q $(LIBDIR)\Test
+	-if exist $(LIBDIR)\Test2 rmdir /s /q $(LIBDIR)\Test2
 	-if exist $(LIBDIR)\Text rmdir /s /q $(LIBDIR)\Text
 	-if exist $(LIBDIR)\Thread rmdir /s /q $(LIBDIR)\Thread
 	-if exist $(LIBDIR)\threads rmdir /s /q $(LIBDIR)\threads
diff --git a/win32/makefile.mk b/win32/makefile.mk
index ff82a18..acb0418 100644
--- a/win32/makefile.mk
+++ b/win32/makefile.mk
@@ -1083,11 +1083,13 @@ SETARGV_OBJ	= setargv$(o)
 # some exclusions, unfortunately, until fixed:
 #  - MakeMaker isn't capable enough for SDBM_File (small bug)
 STATIC_EXT	= * !SDBM_File
+NORMALIZE_STATIC = Normalize_static
 .ELSE
 # specify static extensions here, for example:
 # (be sure to include Win32CORE to load Win32 on demand)
 #STATIC_EXT	= Win32CORE Cwd Compress/Raw/Zlib
 STATIC_EXT	= Win32CORE
+NORMALIZE_DYN	= Normalize_dyn
 .ENDIF
 
 DYNALOADER	= ..\DynaLoader$(o)
@@ -1512,25 +1514,33 @@ $(PERLEXESTATIC): $(PERLSTATICLIB) $(CONFIGPM) $(PERLEXEST_OBJ) $(PERLEXE_RES)
 # DynaLoader.pm, so this will have to do
 
 #most of deps of this target are in DYNALOADER and therefore omitted here
-Extensions : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) MakePPPort
+Extensions : $(PERLDEP) $(DYNALOADER) Extension_lib $(GLOBEXE) MakePPPort
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic !Unicode/Normalize
 
-Extensions_normalize : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
+Normalize_static : $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) $(UNIDATAFILES)
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static +Unicode/Normalize
+
+Normalize_dyn : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +Unicode/Normalize
 
 Extensions_reonly : $(PERLDEP) $(DYNALOADER)
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +re
 
-Extensions_static : ..\make_ext.pl list_static_libs.pl $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
-	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static
+Exts_static_general : ..\make_ext.pl $(CONFIGPM) Extension_lib $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static !Unicode/Normalize
+
+Extensions_static : list_static_libs.pl Exts_static_general $(NORMALIZE_STATIC)
 	$(MINIPERL) -I..\lib list_static_libs.pl > Extensions_static
 
 Extensions_nonxs : ..\make_ext.pl ..\pod\perlfunc.pod $(CONFIGPM) $(GLOBEXE)
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --nonxs !libs
 
+Extension_lib : ..\make_ext.pl $(CONFIGPM)
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) lib
+
 #lib must be built, it can't be buildcustomize.pl-ed, and is required for XS building
 $(DYNALOADER) : ..\make_ext.pl $(CONFIGPM) $(HAVE_COREDIR)
-	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader lib
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader
 
 Extensions_clean :
 	-if exist $(MINIPERL) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --all --target=clean
@@ -1542,9 +1552,9 @@ Extensions_realclean :
 # be running in parallel like UNIDATAFILES, this target a placeholder for the
 # future
 .IF "$(PERLSTATIC)"=="static"
-rebasePE : Extensions_normalize Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
+rebasePE : Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
 .ELSE
-rebasePE : Extensions $(PERLDLL) Extensions_normalize $(PERLEXE)
+rebasePE : Extensions $(PERLDLL) $(NORMALIZE_DYN) $(PERLEXE)
 .ENDIF
 	$(NOOP)
 
@@ -1612,7 +1622,7 @@ utils: $(HAVEMINIPERL) ..\utils\Makefile
 	$(MINIPERL) -I..\lib ..\autodoc.pl ..
 	$(MINIPERL) -I..\lib ..\pod\perlmodlib.PL -q ..
 
-..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs Extensions_normalize utils
+..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs $(NORMALIZE_DYN) utils
 	$(PERLEXE) -f ..\pod\buildtoc -q
 
 # Note that the pod cleanup in this next section is parsed (and regenerated
@@ -1629,7 +1639,6 @@ distclean: realclean
 	-del /f $(LIBDIR)\SDBM_File.pm $(LIBDIR)\Socket.pm $(LIBDIR)\POSIX.pm
 	-del /f $(LIBDIR)\B.pm $(LIBDIR)\O.pm $(LIBDIR)\re.pm
 	-del /f $(LIBDIR)\File\Glob.pm
-	-del /f $(LIBDIR)\Storable.pm
 	-del /f $(LIBDIR)\Sys\Hostname.pm
 	-del /f $(LIBDIR)\Time\HiRes.pm
 	-del /f $(LIBDIR)\Unicode\Normalize.pm
@@ -1642,7 +1651,6 @@ distclean: realclean
 	-del /f $(LIBDIR)\buildcustomize.pl
 	-del /f $(DISTDIR)\XSLoader\XSLoader.pm
 	-del /f *.def *.map
-	-if exist $(LIBDIR)\Amiga rmdir /s /q $(LIBDIR)\Amiga
 	-if exist $(LIBDIR)\App rmdir /s /q $(LIBDIR)\App
 	-if exist $(LIBDIR)\Archive rmdir /s /q $(LIBDIR)\Archive
 	-if exist $(LIBDIR)\Attribute rmdir /s /q $(LIBDIR)\Attribute
-- 
2.5.0.windows.1

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Apr 6, 2018

From @bulk88

On Thu, 05 Apr 2018 20​:34​:20 -0700, bulk88 wrote​:

This is a bug report for perl from bulk88@​hotmail.com,
generated with the help of perlbug 1.41 running under perl 5.27.9.

-----------------------------------------------------------------
[Please describe your issue here]

See attached patch. This is intended for 5.28. Its a follow on to
problems I discussed in
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132992 . It fixes not being
able to build ALL_STATIC Extensions_static target individually and
relying on a race/adjacent targets to randomly finish first to make
Extensions_static not fail.

Revised patch attach, the above one failed a regen.t test because I removed the Amiga dir from dmake mkf, thinking that will never be built on Win32 bc I was thinking the original author of gmake mkf also thought that and thats why he left it out. It turns out there a lib_cleanup.pl that was checking the dir clean list in the win32 makefiles but it didnt know about gmake makefile. So the test failed. To make things look more proper, add the amiga dir to the gmake makefile and start having that file be chked by lib_cleanup.pl like the other 2 mkfs are checked.

--
bulk88 ~ bulk88 at hotmail.com

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Apr 6, 2018

From @bulk88

0001-fix-parallel-ALL_STATIC-Win32-build.patch
From 63a433af1aa5ea2c6f9f7df33d7278ba15683bee Mon Sep 17 00:00:00 2001
From: Daniel Dragan <bulk88@hotmail.com>
Date: Fri, 6 Apr 2018 09:12:48 -0400
Subject: [PATCH] fix parallel ALL_STATIC Win32 build

Normalize depends on mktables's output. Mktables is a very long serial CPU
intensive build product. No other modules except Normalize depend on
mktables. Normalize XS module must be split out from all other XS modules
to fill CPU cores with parallel work, so that the 2 longest running
tagets, Mktables and all XS modules, run in parallel instead of 1 after
another. Since Normalize was hardcoded as a dynamic module, in an
ALL_STATIC build, Normalize was build twice, once as a static in the
big Extensions_static target, and once as a dynamic module, a race
happened if a parallel build was done, and if the Extensions_static target
got around to trying to build Normalize static, before by chance Normalize
dyn was build, Normalize static failed. This can be demostrated by cleaning
the src tree and building each target individually explicitly. Fix the race
by creating a Normalize static and dynamic target and dropout the deps
depending on if its ALL_STATIC or not. Also the now "big" Extensions_static
target when doing an ALL_STATIC build also failed because the general XS
modules that need lib.pm to build didn't have lib.pm available when their
Makefile.PL'es were run. For example cpan/Compress-Raw-Bzip2 needs lib.pm
to build. So separate out lib.pm from Dynaloader target so both dyn
extensions and static exts have lib.pm available. Technically, static exts
dont need lib.pm unless its ALL_STATIC, since for dyn win32 perl Win32CORE
is the only static module, but lib.pm is relativly fast and easy and low
dep to build so build it anyways for static ext target in non-ALL_STATIC
mode rather than special case the dep between ALL_STATIC and
non-ALL_STATIC.

-re.pm doesn't need lib.pm
-normalize doesn't need lib.pm

Reverts "Fix static builds with MinGW" commit 9999704e7ac and adds fixes.
This is a follow on to [perl #132992].

To reduce the diff-ness between the dmake and gmake mkfs. Make the
distclean identical. For dmkf, dont delete Storable.pm twice. And for gmfk
add Amiga dir deletion since lib_cleanup.pl requires it for dmkf but the
original day 1 of gmkf in commit 342634f3c8 "kmx's original GNUmakefile"
never had the Amiga line, but that day 1 commit also never added
GNUmakefile to lib_cleanup.pl the way makefile.mk was already added and
chked for its dir clean list. So add GNUmakefile to be chked by
lib_cleanup.pl the way the other 2 win32 mkfs are checked. In gmfk move
Storable.pm line to match dmfk which is more psuedo-alpha sorted. Also
fix the Test vs Test2 typo.

Some more text is available in the ticket associated with this patch.
---
 regen/lib_cleanup.pl |  2 +-
 t/porting/regen.t    |  2 +-
 win32/GNUmakefile    | 31 +++++++++++++++++++++----------
 win32/makefile.mk    | 27 ++++++++++++++++++---------
 4 files changed, 41 insertions(+), 21 deletions(-)

diff --git a/regen/lib_cleanup.pl b/regen/lib_cleanup.pl
index d381269..5c5c4e2 100644
--- a/regen/lib_cleanup.pl
+++ b/regen/lib_cleanup.pl
@@ -154,7 +154,7 @@ sub edit_win32_makefile {
 }
 
 process('Makefile.SH', 'Makefile.SH', \&edit_makefile_SH, $TAP && '', $Verbose);
-foreach ('win32/Makefile', 'win32/makefile.mk') {
+foreach ('win32/Makefile', 'win32/makefile.mk', 'win32/GNUmakefile') {
     process($_, $_, \&edit_win32_makefile, $TAP && '', $Verbose);
 }
 
diff --git a/t/porting/regen.t b/t/porting/regen.t
index f296626..26639a9 100644
--- a/t/porting/regen.t
+++ b/t/porting/regen.t
@@ -26,7 +26,7 @@ if ( $Config{usecrosscompile} ) {
   skip_all( "Not all files are available during cross-compilation" );
 }
 
-my $tests = 26; # I can't see a clean way to calculate this automatically.
+my $tests = 27; # I can't see a clean way to calculate this automatically.
 
 my %skip = ("regen_perly.pl"    => [qw(perly.act perly.h perly.tab)],
             "regen/keywords.pl" => [qw(keywords.c keywords.h)],
diff --git a/win32/GNUmakefile b/win32/GNUmakefile
index 877c16b..82bf593 100644
--- a/win32/GNUmakefile
+++ b/win32/GNUmakefile
@@ -1110,11 +1110,13 @@ ifeq ($(ALL_STATIC),define)
 # some exclusions, unfortunately, until fixed:
 #  - MakeMaker isn't capable enough for SDBM_File (small bug)
 STATIC_EXT	= * !SDBM_File
+NORMALIZE_STATIC = Normalize_static
 else
 # specify static extensions here, for example:
 # (be sure to include Win32CORE to load Win32 on demand)
 #STATIC_EXT	= Win32CORE Cwd Compress/Raw/Zlib
 STATIC_EXT	= Win32CORE
+NORMALIZE_DYN	= Normalize_dyn
 endif
 
 DYNALOADER	= ..\DynaLoader$(o)
@@ -1572,25 +1574,33 @@ endif
 # DynaLoader.pm, so this will have to do
 
 #most of deps of this target are in DYNALOADER and therefore omitted here
-Extensions : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) MakePPPort
+Extensions : $(PERLDEP) $(DYNALOADER) Extension_lib $(GLOBEXE) MakePPPort
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic !Unicode/Normalize
 
-Extensions_normalize : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
+Normalize_static : $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) $(UNIDATAFILES)
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static +Unicode/Normalize
+
+Normalize_dyn : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +Unicode/Normalize
 
 Extensions_reonly : $(PERLDEP) $(DYNALOADER)
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +re
 
-Extensions_static : ..\make_ext.pl list_static_libs.pl $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
-	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static
+Exts_static_general : ..\make_ext.pl $(CONFIGPM) Extension_lib $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static !Unicode/Normalize
+
+Extensions_static : list_static_libs.pl Exts_static_general $(NORMALIZE_STATIC)
 	$(MINIPERL) -I..\lib list_static_libs.pl > Extensions_static
 
 Extensions_nonxs : ..\make_ext.pl ..\pod\perlfunc.pod $(CONFIGPM) $(GLOBEXE)
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --nonxs !libs
 
+Extension_lib : ..\make_ext.pl $(CONFIGPM)
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) lib
+
 #lib must be built, it can't be buildcustomize.pl-ed, and is required for XS building
 $(DYNALOADER) : ..\make_ext.pl $(CONFIGPM) $(HAVE_COREDIR)
-	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader lib
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader
 
 Extensions_clean :
 	-if exist $(MINIPERL) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --all --target=clean
@@ -1612,9 +1622,9 @@ PostExt : ..\lib\Storable\Limit.pm
 # be running in parallel like UNIDATAFILES, this target a placeholder for the
 # future
 ifeq ($(PERLSTATIC),static)
-rebasePE : Extensions_normalize Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
+rebasePE : Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
 else
-rebasePE : Extensions $(PERLDLL) Extensions_normalize $(PERLEXE)
+rebasePE : Extensions $(PERLDLL) $(NORMALIZE_DYN) $(PERLEXE)
 endif
 	$(NOOP)
 
@@ -1670,7 +1680,7 @@ utils: $(HAVEMINIPERL) ..\utils\Makefile
 	$(MINIPERL) -I..\lib ..\autodoc.pl ..
 	$(MINIPERL) -I..\lib ..\pod\perlmodlib.PL -q ..
 
-..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs Extensions_normalize utils
+..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs $(NORMALIZE_DYN) utils
 	$(PERLEXE) -f ..\pod\buildtoc -q
 
 # Note that the pod cleanup in this next section is parsed (and regenerated
@@ -1687,11 +1697,11 @@ distclean: realclean
 	-del /f $(LIBDIR)\SDBM_File.pm $(LIBDIR)\Socket.pm $(LIBDIR)\POSIX.pm
 	-del /f $(LIBDIR)\B.pm $(LIBDIR)\O.pm $(LIBDIR)\re.pm
 	-del /f $(LIBDIR)\File\Glob.pm
-	-del /f $(LIBDIR)\Storable.pm $(LIBDIR)\Storable\Limit.pm
 	-del /f $(LIBDIR)\Sys\Hostname.pm
 	-del /f $(LIBDIR)\Time\HiRes.pm
 	-del /f $(LIBDIR)\Unicode\Normalize.pm
 	-del /f $(LIBDIR)\Math\BigInt\FastCalc.pm
+	-del /f $(LIBDIR)\Storable.pm $(LIBDIR)\Storable\Limit.pm
 	-del /f $(LIBDIR)\Win32.pm
 	-del /f $(LIBDIR)\Win32CORE.pm
 	-del /f $(LIBDIR)\Win32API\File.pm
@@ -1699,6 +1709,7 @@ distclean: realclean
 	-del /f $(LIBDIR)\buildcustomize.pl
 	-del /f $(DISTDIR)\XSLoader\XSLoader.pm
 	-del /f *.def *.map
+	-if exist $(LIBDIR)\Amiga rmdir /s /q $(LIBDIR)\Amiga
 	-if exist $(LIBDIR)\App rmdir /s /q $(LIBDIR)\App
 	-if exist $(LIBDIR)\Archive rmdir /s /q $(LIBDIR)\Archive
 	-if exist $(LIBDIR)\Attribute rmdir /s /q $(LIBDIR)\Attribute
@@ -1750,7 +1761,7 @@ distclean: realclean
 	-if exist $(LIBDIR)\TAP rmdir /s /q $(LIBDIR)\TAP
 	-if exist $(LIBDIR)\Term rmdir /s /q $(LIBDIR)\Term
 	-if exist $(LIBDIR)\Test rmdir /s /q $(LIBDIR)\Test
-	-if exist $(LIBDIR)\Test2 rmdir /s /q $(LIBDIR)\Test
+	-if exist $(LIBDIR)\Test2 rmdir /s /q $(LIBDIR)\Test2
 	-if exist $(LIBDIR)\Text rmdir /s /q $(LIBDIR)\Text
 	-if exist $(LIBDIR)\Thread rmdir /s /q $(LIBDIR)\Thread
 	-if exist $(LIBDIR)\threads rmdir /s /q $(LIBDIR)\threads
diff --git a/win32/makefile.mk b/win32/makefile.mk
index 2797b8a..265e93b 100644
--- a/win32/makefile.mk
+++ b/win32/makefile.mk
@@ -1084,11 +1084,13 @@ SETARGV_OBJ	= setargv$(o)
 # some exclusions, unfortunately, until fixed:
 #  - MakeMaker isn't capable enough for SDBM_File (small bug)
 STATIC_EXT	= * !SDBM_File
+NORMALIZE_STATIC = Normalize_static
 .ELSE
 # specify static extensions here, for example:
 # (be sure to include Win32CORE to load Win32 on demand)
 #STATIC_EXT	= Win32CORE Cwd Compress/Raw/Zlib
 STATIC_EXT	= Win32CORE
+NORMALIZE_DYN	= Normalize_dyn
 .ENDIF
 
 DYNALOADER	= ..\DynaLoader$(o)
@@ -1513,25 +1515,33 @@ $(PERLEXESTATIC): $(PERLSTATICLIB) $(CONFIGPM) $(PERLEXEST_OBJ) $(PERLEXE_RES)
 # DynaLoader.pm, so this will have to do
 
 #most of deps of this target are in DYNALOADER and therefore omitted here
-Extensions : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) MakePPPort
+Extensions : $(PERLDEP) $(DYNALOADER) Extension_lib $(GLOBEXE) MakePPPort
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic !Unicode/Normalize
 
-Extensions_normalize : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
+Normalize_static : $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) $(UNIDATAFILES)
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static +Unicode/Normalize
+
+Normalize_dyn : $(PERLDEP) $(DYNALOADER) $(GLOBEXE) $(UNIDATAFILES)
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +Unicode/Normalize
 
 Extensions_reonly : $(PERLDEP) $(DYNALOADER)
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --dynamic +re
 
-Extensions_static : ..\make_ext.pl list_static_libs.pl $(CONFIGPM) $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
-	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static
+Exts_static_general : ..\make_ext.pl $(CONFIGPM) Extension_lib $(GLOBEXE) $(HAVE_COREDIR) MakePPPort
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --static !Unicode/Normalize
+
+Extensions_static : list_static_libs.pl Exts_static_general $(NORMALIZE_STATIC)
 	$(MINIPERL) -I..\lib list_static_libs.pl > Extensions_static
 
 Extensions_nonxs : ..\make_ext.pl ..\pod\perlfunc.pod $(CONFIGPM) $(GLOBEXE)
 	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --nonxs !libs
 
+Extension_lib : ..\make_ext.pl $(CONFIGPM)
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) lib
+
 #lib must be built, it can't be buildcustomize.pl-ed, and is required for XS building
 $(DYNALOADER) : ..\make_ext.pl $(CONFIGPM) $(HAVE_COREDIR)
-	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader lib
+	$(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(EXTDIR) --dir=$(DISTDIR) --dynaloader
 
 Extensions_clean :
 	-if exist $(MINIPERL) $(MINIPERL) -I..\lib ..\make_ext.pl "MAKE=$(PLMAKE)" --dir=$(CPANDIR) --dir=$(DISTDIR) --dir=$(EXTDIR) --all --target=clean
@@ -1543,9 +1553,9 @@ Extensions_realclean :
 # be running in parallel like UNIDATAFILES, this target a placeholder for the
 # future
 .IF "$(PERLSTATIC)"=="static"
-rebasePE : Extensions_normalize Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
+rebasePE : Extensions $(PERLDLL) $(PERLEXE) $(PERLEXESTATIC)
 .ELSE
-rebasePE : Extensions $(PERLDLL) Extensions_normalize $(PERLEXE)
+rebasePE : Extensions $(PERLDLL) $(NORMALIZE_DYN) $(PERLEXE)
 .ENDIF
 	$(NOOP)
 
@@ -1613,7 +1623,7 @@ utils: $(HAVEMINIPERL) ..\utils\Makefile
 	$(MINIPERL) -I..\lib ..\autodoc.pl ..
 	$(MINIPERL) -I..\lib ..\pod\perlmodlib.PL -q ..
 
-..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs Extensions_normalize utils
+..\pod\perltoc.pod: $(PERLEXE) $(PERLDLL) Extensions Extensions_nonxs $(NORMALIZE_DYN) utils
 	$(PERLEXE) -f ..\pod\buildtoc -q
 
 # Note that the pod cleanup in this next section is parsed (and regenerated
@@ -1630,7 +1640,6 @@ distclean: realclean
 	-del /f $(LIBDIR)\SDBM_File.pm $(LIBDIR)\Socket.pm $(LIBDIR)\POSIX.pm
 	-del /f $(LIBDIR)\B.pm $(LIBDIR)\O.pm $(LIBDIR)\re.pm
 	-del /f $(LIBDIR)\File\Glob.pm
-	-del /f $(LIBDIR)\Storable.pm
 	-del /f $(LIBDIR)\Sys\Hostname.pm
 	-del /f $(LIBDIR)\Time\HiRes.pm
 	-del /f $(LIBDIR)\Unicode\Normalize.pm
-- 
2.5.0.windows.1

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Apr 6, 2018

From Vadim.Konovalov@dell.com

Hi Daniel

On Thu, 05 Apr 2018 20​:34​:20 -0700, bulk88 wrote​:

This is
See
attached patch. This is intended for 5.28. Its a follow on to
problems I discussed in

...

Given that you so much dived into details of this problem, maybe you can
also have an idea, why all extensions happen to be stuffed into the libperl527s.a for GCC build??

I've questioned this question here​:
https://www.nntp.perl.org/group/perl.perl5.porters/2018/03/msg249982.html

TIA

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Apr 6, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Apr 9, 2018

From @tonycoz

On Fri, 06 Apr 2018 06​:28​:22 -0700, bulk88 wrote​:

Revised patch attach, the above one failed a regen.t test because I
removed the Amiga dir from dmake mkf, thinking that will never be
built on Win32 bc I was thinking the original author of gmake mkf also
thought that and thats why he left it out. It turns out there a
lib_cleanup.pl that was checking the dir clean list in the win32
makefiles but it didnt know about gmake makefile. So the test failed.
To make things look more proper, add the amiga dir to the gmake
makefile and start having that file be chked by lib_cleanup.pl like
the other 2 mkfs are checked.

Thanks, applied as 908f2cb.

There's still a problem if you build with STATIC_EXT=Unicode/Normalize but that might be more difficult to fix.

I do wonder if the Win32 build should build miniperl, then use that to build the final config.sh/config.h and generate a Makefile for the full perl from there.

Tony

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Apr 16, 2018

From @bulk88

On Sun, 08 Apr 2018 21​:03​:16 -0700, tonyc wrote​:

Thanks, applied as 908f2cb.

There's still a problem if you build with STATIC_EXT=Unicode/Normalize
but that might be more difficult to fix.

I do wonder if the Win32 build should build miniperl, then use that to
build the final config.sh/config.h and generate a Makefile for the
full perl from there.

Tony

What will generate the makefile? Makefile.SH would have to be rewritten in PP.

IMO its pointless because Windows doesn't come with a "Unix shell" compiler and M4 compiler out of the box and switching to Visual C with Autoconf+Bash (I've built such FOSS projects before) is insane since not even Unix perl uses M4 or autoconf. I know you wrote one patch 2 yrs ago that generates an auxiliary extensions only makefile for parallel extension building so each extension can be parallelzed to infinity cores instead of the 4 "categories" of extensions building side by side they way they do now and doesn't saturate all cores.

--
bulk88 ~ bulk88 at hotmail.com

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Apr 17, 2018

From @tonycoz

On Mon, Apr 16, 2018 at 10​:54​:02AM -0700, bulk88 via RT wrote​:

On Sun, 08 Apr 2018 21​:03​:16 -0700, tonyc wrote​:

Thanks, applied as 908f2cb.

There's still a problem if you build with STATIC_EXT=Unicode/Normalize
but that might be more difficult to fix.

I do wonder if the Win32 build should build miniperl, then use that to
build the final config.sh/config.h and generate a Makefile for the
full perl from there.

Tony

What will generate the makefile? Makefile.SH would have to be rewritten in PP.

Well, it wouldn't match Makefile.SH anyway, since it doesn't need to
build miniperl.

IMO its pointless because Windows doesn't come with a "Unix shell" compiler and M4 compiler out of the box and switching to Visual C with Autoconf+Bash (I've built such FOSS projects before) is insane since not even Unix perl uses M4 or autoconf. I know you wrote one patch 2 yrs ago that generates an auxiliary extensions only makefile for parallel extension building so each extension can be parallelzed to infinity cores instead of the 4 "categories" of extensions building side by side they way they do now and doesn't saturate all cores.

It would at least fix building with STATIC_EXT=Unicode/Normalize.

It may be possible to improve parallel building over my extensions.mk,
since with that the main makefile can't depend on individual
extensions, or it will risk the sub-makefile and the main makefile
trying to build the same extension at the same time, while if they're
in the same file a target can depend on the .dll or $(PERL_EXE) as
needed.

At this point it's just brainstorming which has at least inspired a
little discussion.

Tony

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants