5.22.0 documentation request: How to debug "handshake key" errors #14714
From smoker report
pegex_parser_t_c4f8477.c: loadable library and perl binaries are mismatched (got handshake key 0xd800000, needed 0xd700000)
In total my smokers produced 13 fail reports with this error between
It is not a showstopper but a documentation request for 5.22.1.
The following description in perldiag is not deemed helpful enough,
%s: loadable library and perl binaries are mismatched (got handshake key
Can some guidance be added how to proceed if it happens?
P.S. the above perldiag comes from
Summary of my perl5 (revision 5 version 22 subversion 0) configuration:
Characteristics of this binary (from libperl):
On Fri May 22 08:45:38 2015, andreas.koenig.7os6VVqR@franz.ak.mind.de wrote:
The XS module reports the interp struct (or a fake unallocated one if it is no threads perl) is 0xd80 bytes long, the perl core says the interp struct is 0xd70 bytes long. The low 16 bits on the handshake are a bitfield of misc things, there is no problem here since both sides are 0x0000.
The -D'es on the cmd line to the CC dont match the -D'es that the perl core was compiled with, or somehow EUMM/MB is using the wrong CORE dir and the wrong config.h file, or your @INC patch or arch dirs are messed up. A wild guess is you have -DDEBUGGING on the cmd line to the CC in your module. DEBUGGING can only be set at perl interp build time, not randomly, per module, afterwards. If you want C asserts for a single XS module, eghh, I forgot how to turn on C asserts without turning on DEBUGGING. Perl, assert.h, and -DNDEBUG have a complicated sketchy interaction.
The current code added in 5.21.7 in perl.h is
Before 5.21.7 is was
#if defined(DEBUGGING) && defined(I_ASSERT)
Worst case scenario for debugging interp size differences is producing TWO .i files. One from a working XS module, one from a failing (wrong interp size) module. Send both .i files through gnu indent to normalize whitespace. Then copy paste the interp structs alone into separate files, then run diff on the 2 files. Now you know what members are different. If the interp structs are text identical, then probably the PL_sys_intern and struct interp_intern are different. If that doesn't reveal the different, produce the 2 .i files again, do a diff of the full size .i files and look for the difference. Stripping all ^#.+$ lines from both .i files and gnu indent may or may not help reduce the noise in the diff.
On Fri May 22 10:20:25 2015, bulk88 wrote:
About that warning message. I think "reinstall the XS module" is the limit of what advice we can give, other than contact the author. You can't seriously tell a [pure] Perl users to debug XS/C/.i files/headers/drink boiling macro soup/rebuild with symbols then use gdb stacktraces and C variable watching.
The handshake feature stops people from copy-paste installing XS modules across different perl interp versions/builds. It also stops segvs from upgrading blead perl to blead, but not reinstalling any /site XS modules (im lazy). If the blead perls are ABI compatible (enough, I guess, handshake api wont catch changing the integers behind perl SV flags for example, but will catch threads, DEBUGGING, DEBUG_LEAKING_SCALARS, 64 bit IVs on 32 bit), you dont need to recompile, if they are ABI not compatible, you get the handshake message. Also some .t files do "push @INC, 'lib', '../lib','../../lib' and pick up another Perl's /lib on my system, and previously, trying to do the god damn croak() to tell the user that module was compiled with 5.12, but I am running on 5.19 (or any other version other than 5.12), would SEGV inside 5.12 interp, when passed a my_perl ptr from 5.19. None of that is possible now (assuming the XS module was compiled against a handshake enabled perl version, hence the call stack below will still crash to this day) See the below call stack for hilarity.
perl512.dll!Perl_safesysmalloc(unsigned int size=12) Line 94 + 0xd bytes C
Finally, handshake stops with a user message, a subtle case, lets say 64 bit IVs on 32, where the BOOT XSUB registers all the XSUBs correctly, and basic XSUBs work, since PL_stack_sp and PL_stack_base and PL_markstack are at the start of the interp struct and their positions never change, but if you use MY_CXT, where PL_my_cxt_list is at the bottom of the list/struct, chances are very high you will SEGV since the offsets are different by the end of the interp struct. Then to a pure perl user, or a novice XS programing who set CCFLAGS to an empty string or set to his own randomly created constant string, some XSUBs work, some crash, "IDK it crashes and IDK how to use C debuggers".
> The guide
Thanks, this helped a lot in finding the culprit.
> The XS module reports the interp struct (or a fake unallocated one
> The -D'es on the cmd line to the CC dont match the -D'es that the
This sentence alone contained all I needed (as I realized a bit later).
perl Makefile.PL DEFINE='-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -DDOUBLE_SLASHES_SPECIAL=0'
MakeMaker added those in one project (Cwd) but not in the other
> [...] Worst case scenario for debugging interp size differences is
I assume you mean PerlHandShakeInterpreter struct.
> alone into separate files, then run diff on the 2 files. Now you
They were identical.
> then probably the PL_sys_intern and struct interp_intern are
They did not exist.
> If that doesn't reveal the different, produce the 2 .i files again,
The diff was in things like
+typedef __ino_t ino_t;
and that was where I decided I had to go back to your sentence about the
On Sat May 23 02:48:43 2015, andreas.koenig.7os6VVqR@franz.ak.mind.de wrote:
Unconditional assignment, instead of catting onto CCFLAGS, in a Makefile.PL will strip most or all of the -Ds cause breakage. Remember, adding your own -Ds to CCFLAGS or the cmd line to the CC is fine, but never remove Perl's -Ds or the -Ds that were there by default (unless you were the author of that define in the perl core).
PerlHandShakeInterpreter is the fake never allocated interp struct for no threads perl. On threaded perl it will be PerlInterpreter.
They do on Windows. They are different sizes depending on ithread without psuedofork and ithreads with psuedofork. Since its unix, that struct not existing is fine.
off_t would be a similar case. Is your "int" 32 or 64 (a performance question if 64 isnt native to the CPU)? is your size_t/pointer size 32 or 64 (what is your virtual memory space? what pointer do you use?)? is your maximum file on your os size 32 or 64 bits (a property of OS FS driver API and libc)?
All those numbers can be different. Plain C, unlike C++, offers no protection against declaring a function with a 32 bit arg vs a 64 bit arg between declaration and definition, and the segv/uninit data crash that results when you mix the 2 up.
Late response in this ticket is because I wrote the response then forgot about this tab for a month.