Failure testing libapreq2 with perl-5.38.0-RC2 #21179
Comments
|
If you upload debug binaries of everything somewhere, I can look into it. Building apache is a painful and pretty much undocumented process. |
Unfortunately, the mod_perl build is failing with debug builds of everything due to 3 unresolved externals: libaprext.lib(modperl_error.obj) : error LNK2001: unresolved external symbol PL_valid_types_RV It was (obviously) fine with release builds. I haven't seen this before and I currently have no idea what the problem is. I will try it again tomorrow. |
A build with CFG=DebugSymbols should prevent the references to those symbols and be debuggable. If I build blead with CFG=Debug the generated dll exports the above symbols and the import library also includes them. My first though is that perhaps somehow the symbol isn't marked as an import, which for a data reference would generate code referencing __imp_PL_valid_types_RV. I'd expect the errors to use those names, though perhaps Microsoft have made the errors more friendly. If you could get the preprocessed version of modperl_error.c, hopefully with |
Attached. For reference, this is the command that was run during the build process to compile modperl_error.c: cl -ID:/Dev/Temp/mod_perl-2.0.12/src/modules/perl -ID:/Dev/Temp/mod_perl-2.0.12/xs -ID:\Dev\Temp\apache\include -I"D:\Dev\Temp\apache\include" -ID:\Dev\Temp\apache\include -nologo -GF -W3 -MDd -DWIN32 -D_CONSOLE -DNO_STRICT -D_DEBUG -DDEBUGGING -DWIN64 -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -D_WINSOCK_DEPRECATED_NO_WARNINGS -DPERL_TEXTMODE_SCRIPTS -DMULTIPLICITY -DPERL_IMPLICIT_SYS -I"D:\Dev\Temp\perl\lib\CORE" -DMOD_PERL -DMP_COMPAT_1X -DMP_TRACE -Od -Zi -fp:precise -c modperl_error.c |
Using debug builds of PCRE, Expat, apr, apr-util and httpd I find that mod_perl fails to build if perl is built in Debug (or DebugFull) configuration due to the valid_types_* unresolved symbols mentioned above. So I had to use a DebugSymbols (or default (release)) build of perl. mod_perl then builds, but the problem disappears: libapreq2 passes all tests. Reverting to default (release) builds of PCRE, Expat, apr, apr-util and httpd I again find that mod_perl fails to build if perl is built in Debug (or DebugFull) configuration due to the valid_types_* unresolved symbols. So again I had to use a DebugSymbols build of perl. mod_perl then builds (as it did originally with a default (release) build of perl), and fortunately the problem still manifests itself: libapreq2's test suite crashes when starting up the httpd server. I have breakpoints on lines 5304 (the last line of my_CloseHandle(), which calls CloseHandle_orig(h)) and 5327 (the return in win32_hook_closehandle_in_crt() if the attemnpt to get the CloseHandle_orig handle has failed) of perl's win32\win32.c. I also have a breakpoint on line 132 (the _dup2(fd, 1) call in apr_file_dup2(), conditional on fd==3) of apr's file_io\win32\filedup.c. The apr breakpoint is hit first, with this call stack:
This does not result in a call to my_CloseHandle(). Continuing, it then hits the breakpoint on the CloseHandle_orig(h) call ten times, and the lands on the apr breakpoint again, again with this call stack:
However, stepping over the _dup2() call this time results in my_CloseHandle() getting called:
The crash occurs because this time CloseHandle_orig is 0x0000000000000000, which it hadn't been on the ten previous hits on this breakpoint. At no point did the line 5327 breakpoint get hit, which could have explained why CloseHandle_orig is 0x0000000000000000 so I don't know why CloseHandle_orig has become 0x0000000000000000. I also didn't necessarily expect perl's my_CloseHandle() function to be getting called from other (non-perl) code like this. Is that intentional? Repeating the same debugging (same command-line and same breakpoints) with the build in which the problem disappeared (i.e. perl is still the DebugSymbols configuration, but everything else is a debug build rather than default (release) builds), I again get one hit on the apr breakpoint (not followed by a call to my_CloseHandle()), followed by ten hits on the CloseHandle_orig(h) call, followed by another hit on the apr breakpoint and this time that is NOT followed by a call to my_CloseHandle(), which is what caused the crash above. I'm struggling to understand this. Why is code in apr_file_dup2() sometimes resulting in perl's my_CloseHandle() function getting called? And why does that only seem to happen when using release builds of httpd et al? I've uploaded builds of everything in both flavours here: https://people.apache.org/~stevehay/.private/ The libapreq2-release-builds.zip file contains default (release) builds of everything, except perl - which is the DebugSymbols mode. |
In some embedded Perl scenarios, such as mod_perl, the Perl DLL may be unloaded, causing the hook to become invalid. Fixes: Perl#21179
Yes, we patch CRT in-memory to redirect all calls to CloseHandle() inside CRT to my_CloseHandle(). Anyway, I think I have a fix: #21182. Basically, the problem was that Apache does some shenanigans with loading and unloading the Perl DLL and our hooking code wasn't prepared to handle this. |
Excellent! Thanks for this. I can confirm that it fixes the problem for me: With this patch I have perl, mod_perl and libapreq2 all running their tests as usual. |
Thanks, it is importing correctly: The call making all the references here is the call to newRV_noinc() which is now inlined and its call to newSV_type(), which is also inlined. Can you show the failing linker command-line? |
|
Just to check, please ensure |
|
perl538.lib is rebuilt, yes. I've been deleting and rebuilding from scratch the perl folder each time I change configuration to avoid any possible mix-ups with things not getting rebuilt. Here are the compiler and linker commands for the APR::Brigade module, which is the first XS module that the build process fails on: cl -c -ID:/Dev/Temp/mod_perl-2.0.12/src/modules/perl -ID:/Dev/Temp/mod_perl-2.0.12/xs -ID:\Dev\Temp\apache\include -I"D:\Dev\Temp\apache\include" -ID:\Dev\Temp\apache\include -ID:/Dev/Temp/mod_perl-2.0.12/src/modules/perl -ID:/Dev/Temp/mod_perl-2.0.12/xs -ID:\Dev\Temp\apache\include -ID:\Dev\Temp\apache\include -ID:\Dev\Temp\apache\include -ID:/Dev/Temp/mod_perl-2.0.12/src/modules/perl -ID:/Dev/Temp/mod_perl-2.0.12/xs -ID:\Dev\Temp\apache\include -ID:\Dev\Temp\apache\include -ID:\Dev\Temp\apache\include -ID:/Dev/Temp/mod_perl-2.0.12/src/modules/perl -ID:/Dev/Temp/mod_perl-2.0.12/xs -ID:\Dev\Temp\apache\include -I"D:\Dev\Temp\apache\include" -ID:\Dev\Temp\apache\include -nologo -GF -W3 -MD -DWIN32 -D_CONSOLE -DNO_STRICT -DDEBUGGING -DWIN64 -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -D_WINSOCK_DEPRECATED_NO_WARNINGS -DPERL_TEXTMODE_SCRIPTS -DMULTIPLICITY -DPERL_IMPLICIT_SYS -DMOD_PERL -DMP_COMPAT_1X -Od -Zi -fp:precise -DVERSION="0.009000" -DXS_VERSION="0.009000" "-ID:\Dev\Temp\perl\lib\CORE" -DMP_HAVE_APR_LIBS -DMP_HAVE_APR_LIBS -DMP_HAVE_APR_LIBS -DMP_HAVE_APR_LIBS -FdBrigade.pdb Brigade.c That was with a standard CFG=Debug configuration build of perl, for which perl -V is: Summary of my perl5 (revision 5 version 38 subversion 0) configuration: Platform: Characteristics of this binary (from libperl): |
|
Does the installed perl538.dll have the needed exports? Does the installed library include the needed exports? Does the perlldll.def file in the build directory include the needed exports? Given the paths in your build log you probably want: If any of those is missing the symbols, please try building from a fresh tree ( |
|
None of those are missing the symbols: D:\Dev\Temp\mod_perl-2.0.12>dumpbin /exports D:\Dev\Temp\perl\bin\perl538.dll | find "valid_types" D:\Dev\Temp\mod_perl-2.0.12>dumpbin /exports D:\Dev\Temp\perl\lib\CORE\perl538.lib | find "valid_types" D:\Dev\Temp\mod_perl-2.0.12>cd ..\perl-5.38.0-RC2\win32 D:\Dev\Temp\perl-5.38.0-RC2\win32>find "valid_types" perldll.def ---------- PERLDLL.DEF All of the build folders (perl, mod_perl, httpd etc.) are freshly extracted from their respective .tar.gz files, and have been installed into fresh locations (i.e. not over the top of any existing installations). |
In some embedded Perl scenarios, such as mod_perl, the Perl DLL may be unloaded, causing the hook to become invalid. Fixes: #21179
In some embedded Perl scenarios, such as mod_perl, the Perl DLL may be unloaded, causing the hook to become invalid. Fixes: Perl#21179
In the course of testing perl-5.38.0-RC2 I have run into a failure in the test suite of libapreq2:
The Apache httpd server used for running this module's glue\perl\t tests crashes on start up. The server is started with the command: D:\Dev\Temp\apache\bin\httpd.exe -d D:/Dev/Temp/libapreq2-2.17/glue/perl/t -f D:/Dev/Temp/libapreq2-2.17/glue/perl/t/conf/httpd.conf -D APACHE2 -D APACHE2_4 -D PERL_USEITHREADS
Debugging that command, I find that it fails in perl's my_CloseHandle() in win32\win32.c. That function was quite recently added by commit 8552f09 in March this year. The error (from a release build with debug info) is:
Exception thrown at 0x0000000000000000 in httpd.exe: 0xC0000005: Access violation executing location 0x0000000000000000.
The httpd server starts and all of the libapreq2 tests pass if I replace perl-5.38.0-RC2 with perl-5.36.1, keeping everything else the same.
Version numbers of everything used are as follows:
Microsoft Visual Studio Professional 2022 (64-bit) LTSC 17.4 Version 17.4.8 on Windows 10 Enterprise Version 22H2
perl-5.38.0-RC2 (perl -V output is below)
Apache httpd-2.4.57 (with pcre-8.45, expat-2.5.0, apr-1.7.4, and apr-util-1.6.3)
mod_perl-2.0.12 (with Win32-Process-0.17, the latest apxs, and the do_open9() fix from CPAN RT#148451)
libapreq2-2.17 (with Parse-RecDescent-1.967015, Tie-IxHash-1.23, and ExtUtils-XSBuilder-0.28)
All build options (apart from some install locations) are out-of-the-box. I can assist if anyone is willing and able to look at this but has trouble building any of these things.
Similar tests using the Apache httpd server have run and passed during the earlier build of mod_perl, so it is possible that it is something being done by libapreq2 that is at fault, but it's troubling from perl's perspective that no problem is seen when switching to perl-5.3.6.1 with everything else the same.
perl -V output:
Summary of my perl5 (revision 5 version 38 subversion 0) configuration:
Platform:
osname=MSWin32
osvers=10.0.19045.3086
archname=MSWin32-x64-multi-thread
uname=''
config_args='undef'
hint=recommended
useposix=true
d_sigaction=undef
useithreads=define
usemultiplicity=define
use64bitint=define
use64bitall=undef
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
Compiler:
cc='cl'
ccflags ='-nologo -GF -W3 -MD -DWIN32 -D_CONSOLE -DNO_STRICT -DWIN64 -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -D_WINSOCK_DEPRECATED_NO_WARNINGS -DPERL_TEXTMODE_SCRIPTS -DMULTIPLICITY -DPERL_IMPLICIT_SYS'
optimize='-O1 -Zi -GL -fp:precise'
cppflags='-DWIN32'
ccversion='19.34.31946'
gccversion=''
gccosandvers=''
intsize=4
longsize=4
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=undef
longlongsize=8
d_longdbl=define
longdblsize=8
longdblkind=0
ivtype='__int64'
ivsize=8
nvtype='double'
nvsize=8
Off_t='__int64'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries:
ld='link'
ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"D:\Dev\Temp\perl\lib\CORE" -machine:AMD64 -subsystem:console,"5.02"'
libpth="C:\Program Files\Microsoft Visual Studio\2022 17.4\Professional\VC\Tools\MSVC\14.34.31933\lib\x64"
libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib vcruntime.lib ucrt.lib
perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib vcruntime.lib ucrt.lib
libc=ucrt.lib
so=dll
useshrplib=true
libperl=perl538.lib
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_win32.xs
dlext=dll
d_dlsymun=undef
ccdlflags=' '
cccdlflags=' '
lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"D:\Dev\Temp\perl\lib\CORE" -machine:AMD64 -subsystem:console,"5.02"'
Characteristics of this binary (from libperl):
Compile-time options:
HAS_LONG_DOUBLE
HAS_TIMES
HAVE_INTERP_INTERN
MULTIPLICITY
PERLIO_LAYERS
PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_SIPHASH13
PERL_HASH_USE_SBOX32
PERL_IMPLICIT_SYS
PERL_MALLOC_WRAP
PERL_OP_PARENT
PERL_PRESERVE_IVUV
PERL_USE_SAFE_PUTENV
USE_64_BIT_INT
USE_ITHREADS
USE_LARGE_FILES
USE_LOCALE
USE_LOCALE_COLLATE
USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC
USE_LOCALE_TIME
USE_PERLIO
USE_PERL_ATOF
USE_THREAD_SAFE_LOCALE
Locally applied patches:
RC2
Built under MSWin32
Compiled at Jun 27 2023 08:07:13
@inc:
D:/Dev/Temp/perl/site/lib
D:/Dev/Temp/perl/lib
The text was updated successfully, but these errors were encountered: