panic: top_env from embedded perl using 'do' with unused XS calls

[p5pRT]

Migrated from rt.perl.org#44051 (status was 'open')

Searchable as RT44051$

[p5pRT]

From perlbug@daveola.com

This is a bug report for perl from perlbug@​daveola.com,
generated with the help of perlbug 1.35 running under perl v5.8.8.

---

[Please enter your report here]

I'm getting a panic​: top_env

I've managed to reduce the test case to the following​:

  1. From a C program, create/start a perl interpreter object
  2. Load a perl script (using eval_pv "do ...")
  3. Setup an XS routine (which, incidentally, never gets used)
  4. Call a perl routine in the perl script
  5. From that perl routine, execute another perl script (using 'do file')
  6. That perl script needs to define four subs, two which must call the XS routine at least four times, and two which must call it at least once - though again note that none of the routines are called.

The attempt to 'do file' in step 5 causes the panic.

Here's the testcase and Makefile

[p5pRT]

From perlbug@daveola.com

bug.tar.gz

[p5pRT]

From @doy

I can't reproduce this with perl 5.16.0. Is this still an issue?

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From perlbug@daveola.com

On Fri Jun 22 10​:01​:49 2012, doy wrote​:

I can't reproduce this with perl 5.16.0. Is this still an issue?

I (Ubuntu) has finally updated to perl 5.14.2

The original issue is gone with 5.14.2, but it reveals new problems.

If we actually try to call a routine in the eval/do/require'd script
that calls any of the XS functions, then we have a problem.

I've made a simpler example. We have embedded perl that 'eval's another
script that has a function that calls an XS call. If we eval/do, then
when we try to call that function, perl complains about an 'Undefined
subroutine' even though the eval didn't have any errors.

If we change it to a require, then the require complains when it hits
the XS call saying that we've got too many arguments for the call, even
though it's the correct number. If we remove the arguments from the XS
call, then the require works and the XS call is made, but it will
complain about not getting enough arguments.

It seems the XS information is lost across the eval.

I've attached a new example tar.

--
David Ljung Madison
http​://GetDave.com/ http​://MarginalHacks.com/
http​://DaveSource.com/

[p5pRT]

From perlbug@daveola.com

bug.tar.gz

[p5pRT]

From perlbug@daveola.com

On Fri Jun 22 10​:01​:49 2012, doy wrote​:

I can't reproduce this with perl 5.16.0. Is this still an issue?

I (Ubuntu) has finally updated to perl 5.14.2

The original issue is gone with 5.14.2, but it reveals new problems.

If we actually try to call a routine in the eval/do/require'd script
that calls any of the XS functions, then we have a problem.

I've made a simpler example. We have embedded perl that 'eval's another
script that has a function that calls an XS call. If we eval/do, then
when we try to call that function, perl complains about an 'Undefined
subroutine' even though the eval didn't have any errors.

If we change it to a require, then the require complains when it hits
the XS call saying that we've got too many arguments for the call, even
though it's the correct number. If we remove the arguments from the XS
call, then the require works and the XS call is made, but it will
complain about not getting enough arguments.

It seems the XS information is lost across the eval.

I've attached a new example tar.

--
David Ljung Madison
http​://GetDave.com/ http​://MarginalHacks.com/
http​://DaveSource.com/

[p5pRT]

From perlbug@daveola.com

bug.tar.gz

[p5pRT]

From @bulk88

On Sat Nov 03 00​:27​:16 2012, daveola wrote​:

On Fri Jun 22 10​:01​:49 2012, doy wrote​:

I can't reproduce this with perl 5.16.0. Is this still an issue?

I (Ubuntu) has finally updated to perl 5.14.2

The original issue is gone with 5.14.2, but it reveals new problems.

If we actually try to call a routine in the eval/do/require'd script
that calls any of the XS functions, then we have a problem.

I've made a simpler example. We have embedded perl that 'eval's another
script that has a function that calls an XS call. If we eval/do, then
when we try to call that function, perl complains about an 'Undefined
subroutine' even though the eval didn't have any errors.

If we change it to a require, then the require complains when it hits
the XS call saying that we've got too many arguments for the call, even
though it's the correct number. If we remove the arguments from the XS
call, then the require works and the XS call is made, but it will
complain about not getting enough arguments.

It seems the XS information is lost across the eval.

I've attached a new example tar.

--
David Ljung Madison
http​://GetDave.com/ http​://MarginalHacks.com/
http​://DaveSource.com/

I got something else from the code you supplied. On Win32 Perl I got a
crash.
___________________________________________________
  ntdll.dll!_RtlpWaitForCriticalSection@​4() + 0x5b
  ntdll.dll!_RtlEnterCriticalSection@​4() + 0x46

perl517.dll!PerlIOUnix_refcnt_inc(int fd=0) Line 2374 C
  perl517.dll!PerlIOUnix_setfd(interpreter * my_perl=0x008647bc, _PerlIO
* * f=0x00913254, int fd=0, int imode=0) Line 2654 + 0x9 C
  perl517.dll!PerlIOUnix_open(interpreter * my_perl=0x008647bc,
_PerlIO_funcs * self=0x2819ee20, PerlIO_list_s * layers=0x00913104, long
n=0, const char * mode=0x281921d5, int fd=0, int imode=0, int perm=0,
_PerlIO * * f=0x00913254, int narg=0, sv * * args=0x00000000) Line
2735 + 0x15 C
  perl517.dll!PerlIOBuf_open(interpreter * my_perl=0x008647bc,
_PerlIO_funcs * self=0x2819efe0, PerlIO_list_s * layers=0x00913104, long
n=1, const char * mode=0x281921d4, int fd=0, int imode=0, int perm=0,
_PerlIO * * f=0x00000000, int narg=0, sv * * args=0x00000000) Line
3860 + 0x35 C
  perl517.dll!PerlIO_openn(interpreter * my_perl=0x008647bc, const char
* layers=0x00000000, const char * mode=0x281921d4, int fd=0, int
imode=0, int perm=0, _PerlIO * * f=0x00000000, int narg=0, sv * *
args=0x00000000) Line 1649 + 0x32 C
  perl517.dll!PerlIO_fdopen(int fd=0, const char * mode=0x281921d4)
Line 4884 + 0x1d C
  perl517.dll!PerlIO_stdstreams(interpreter * my_perl=0x008647bc) Line
1221 + 0xc C
  perl517.dll!PerlIO_resolve_layers(interpreter * my_perl=0x008647bc,
const char * layers=0x00000000, const char * mode=0x28189294, int
narg=1, sv * * args=0x0012fde4) Line 1539 + 0x9 C
  perl517.dll!PerlIO_openn(interpreter * my_perl=0x008647bc, const char
* layers=0x00000000, const char * mode=0x28189294, int fd=-1, int
imode=0, int perm=0, _PerlIO * * f=0x00000000, int narg=1, sv * *
args=0x0012fde4) Line 1620 + 0x19 C
  perl517.dll!PerlIO_open(const char * path=0x2818a0b0, const char *
mode=0x28189294) Line 4893 + 0x1d C
  perl517.dll!S_open_script(interpreter * my_perl=0x008647bc, const char
* scriptname=0x2818a0b0, char dosearch=0, char * suidscript=0x0012fe53)
Line 3705 + 0xe C
  perl517.dll!S_parse_body(interpreter * my_perl=0x008647bc, char * *
env=0x00000000, void (interpreter *)* xsinit=0x00000000) Line 2062 + 0x15 C
  perl517.dll!perl_parse(interpreter * my_perl=0x008647bc, void
(interpreter *)* xsinit=0x00000000, int argc=3, char * *
argv=0x0012ff50, char * * env=0x00000000) Line 1621 + 0x11 C
  main.exe!main() Line 28 + 0x14 C
  main.exe!mainCRTStartup() Line 398 + 0xe C
  kernel32.dll!_BaseProcessStart@​4() + 0x23
___________________________________________________
PL_perlio_mutex is not set up correctly since DebugInfo should be a
pointer not null. DebugInfo is part of a link list debugging thing
Windows does in critical section structs.
___________________________________________________
- PL_perlio_mutex {DebugInfo=0x00000000 {Type=???
CreatorBackTraceIndex=??? CriticalSection=??? ...} LockCount=1
RecursionCount=0 ...} _RTL_CRITICAL_SECTION
+ DebugInfo 0x00000000 {Type=??? CreatorBackTraceIndex=???
CriticalSection=??? ...} _RTL_CRITICAL_SECTION_DEBUG *
  LockCount 1 long
  RecursionCount 0 long
  OwningThread 0x00000000 void *
  LockSemaphore 0x000007bc void *
  SpinCount 0 unsigned long
___________________________________________________

[p5pRT]

From @bulk88

On Wed Nov 07 21​:40​:01 2012, bulk88 wrote​:

I've attached a new example tar.

--
David Ljung Madison
http​://GetDave.com/ http​://MarginalHacks.com/
http​://DaveSource.com/

I got something else from the code you supplied. On Win32 Perl I got a
crash.
___________________________________________________
ntdll.dll!_RtlpWaitForCriticalSection@​4() + 0x5b
ntdll.dll!_RtlEnterCriticalSection@​4() + 0x46

perl517.dll!PerlIOUnix_refcnt_inc(int fd=0) Line 2374 C
perl517.dll!PerlIOUnix_setfd(interpreter * my_perl=0x008647bc, _PerlIO
* * f=0x00913254, int fd=0, int imode=0) Line 2654 + 0x9 C
perl517.dll!PerlIOUnix_open(interpreter * my_perl=0x008647bc,
_PerlIO_funcs * self=0x2819ee20, PerlIO_list_s * layers=0x00913104, long
n=0, const char * mode=0x281921d5, int fd=0, int imode=0, int perm=0,
_PerlIO * * f=0x00913254, int narg=0, sv * * args=0x00000000) Line
2735 + 0x15 C
perl517.dll!PerlIOBuf_open(interpreter * my_perl=0x008647bc,
_PerlIO_funcs * self=0x2819efe0, PerlIO_list_s * layers=0x00913104, long
n=1, const char * mode=0x281921d4, int fd=0, int imode=0, int perm=0,
_PerlIO * * f=0x00000000, int narg=0, sv * * args=0x00000000) Line
3860 + 0x35 C
perl517.dll!PerlIO_openn(interpreter * my_perl=0x008647bc, const char
* layers=0x00000000, const char * mode=0x281921d4, int fd=0, int
imode=0, int perm=0, _PerlIO * * f=0x00000000, int narg=0, sv * *
args=0x00000000) Line 1649 + 0x32 C
perl517.dll!PerlIO_fdopen(int fd=0, const char * mode=0x281921d4)
Line 4884 + 0x1d C
perl517.dll!PerlIO_stdstreams(interpreter * my_perl=0x008647bc) Line
1221 + 0xc C
perl517.dll!PerlIO_resolve_layers(interpreter * my_perl=0x008647bc,
const char * layers=0x00000000, const char * mode=0x28189294, int
narg=1, sv * * args=0x0012fde4) Line 1539 + 0x9 C
perl517.dll!PerlIO_openn(interpreter * my_perl=0x008647bc, const char
* layers=0x00000000, const char * mode=0x28189294, int fd=-1, int
imode=0, int perm=0, _PerlIO * * f=0x00000000, int narg=1, sv * *
args=0x0012fde4) Line 1620 + 0x19 C
perl517.dll!PerlIO_open(const char * path=0x2818a0b0, const char *
mode=0x28189294) Line 4893 + 0x1d C
perl517.dll!S_open_script(interpreter * my_perl=0x008647bc, const char
* scriptname=0x2818a0b0, char dosearch=0, char * suidscript=0x0012fe53)
Line 3705 + 0xe C
perl517.dll!S_parse_body(interpreter * my_perl=0x008647bc, char * *
env=0x00000000, void (interpreter *)* xsinit=0x00000000) Line 2062
+ 0x15 C
perl517.dll!perl_parse(interpreter * my_perl=0x008647bc, void
(interpreter *)* xsinit=0x00000000, int argc=3, char * *
argv=0x0012ff50, char * * env=0x00000000) Line 1621 + 0x11 C
main.exe!main() Line 28 + 0x14 C
main.exe!mainCRTStartup() Line 398 + 0xe C
kernel32.dll!_BaseProcessStart@​4() + 0x23
___________________________________________________
PL_perlio_mutex is not set up correctly since DebugInfo should be a
pointer not null. DebugInfo is part of a link list debugging thing
Windows does in critical section structs.
___________________________________________________
- PL_perlio_mutex {DebugInfo=0x00000000 {Type=???
CreatorBackTraceIndex=??? CriticalSection=??? ...} LockCount=1
RecursionCount=0 ...} _RTL_CRITICAL_SECTION
+ DebugInfo 0x00000000 {Type=??? CreatorBackTraceIndex=???
CriticalSection=??? ...} _RTL_CRITICAL_SECTION_DEBUG *
LockCount 1 long
RecursionCount 0 long
OwningThread 0x00000000 void *
LockSemaphore 0x000007bc void *
SpinCount 0 unsigned long
___________________________________________________

The crash on windows is caused by a missing (see perlembed )
"PERL_SYS_INIT3(&argc,&argv,&env);" and "PERL_SYS_TERM();" in main.c .
On threaded Win32 Perl, File IO definitely won't work without those 2
calls. I dont know what those 2 do under the hood on Unix, so that might
explain why there is more subtle behavior on Unix. Try adding those 2 as
described in perlembed and see if your problem goes away. I deleted the
files I used to do the above a while ago and forgot about this post so I
can't test the suggestion to see if it stops crashing on Win32 Perl.

--
bulk88 ~ bulk88 at hotmail.com

[p5pRT]

From perlbug@daveola.com

The crash on windows is caused by a missing (see perlembed )
"PERL_SYS_INIT3(&argc,&argv,&env);" and "PERL_SYS_TERM();" in main.c .

I've added those to the second example (the one showing a missing
subroutine) and I still see the problem.

A third tar is attached.

[p5pRT]

From perlbug@daveola.com

bug3.tar.z

[p5pRT]

From perlbug@daveola.com

The crash on windows is caused by a missing (see perlembed )
"PERL_SYS_INIT3(&argc,&argv,&env);" and "PERL_SYS_TERM();" in main.c

It doesn't seem that the docs really explain what PERL_SYS_INIT3 does.
If I don't have the env, can I supply null or perhaps a pointer to an
empty array of char pointers? This code is being called from another
language, so we don't have a main or env in the C code.

[p5pRT]

From [Unknown Contact. See original ticket]

The crash on windows is caused by a missing (see perlembed )
"PERL_SYS_INIT3(&argc,&argv,&env);" and "PERL_SYS_TERM();" in main.c

It doesn't seem that the docs really explain what PERL_SYS_INIT3 does.
If I don't have the env, can I supply null or perhaps a pointer to an
empty array of char pointers? This code is being called from another
language, so we don't have a main or env in the C code.

[p5pRT]

From @bulk88

On Mon Mar 11 21​:08​:22 2013, daveola wrote​:

The crash on windows is caused by a missing (see perlembed )
"PERL_SYS_INIT3(&argc,&argv,&env);" and "PERL_SYS_TERM();" in main.c

It doesn't seem that the docs really explain what PERL_SYS_INIT3 does.
If I don't have the env, can I supply null or perhaps a pointer to an
empty array of char pointers? This code is being called from another
language, so we don't have a main or env in the C code.

Tried your new version. No crashes on 5.17.7. Used (win32 platform)
__________________________________________________________________

  cl -c -nologo -GF -W3 -MD -Zi -DNDEBUG -O1 -GL -G7 -DWIN32
-D_CONSOLE -DNO_STRICT -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT
-DPERL_IMPLICIT_SYS -DUSE_PERLIO -D_USE_32BIT_TIME_T -MD -Zi -DNDEBUG
-O1 -GL -G7 "-IC​:\perl517\lib\CORE" main.c
  link -out​:main.exe -nologo -nodefaultlib -debug -opt​:ref,icf
-ltcg -libpath​:"c​:\perl517\lib\CORE" -machine​:x86 main.obj
C​:\perl517\lib\CORE\perl517.lib oldnames.lib kernel32.lib user32.lib
gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib
oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib
version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib
__________________________________________________________________
to compile.

Running your code unmodified gave
__________________________________________________________________
C​:\Documents and Settings\Owner\Desktop\cpan libs\prtbug\bug3>main
Evaluate builtin.pl
Setup the XS routine
Call builtinInit
do script.pl
Eval returns​: scriptpl and
Undefined subroutine &main​::fubar0 called at builtin.pl line 5.

C​:\Documents and Settings\Owner\Desktop\cpan libs\prtbug\bug3>
___________________________________________________________________

In builtin.pl you wrote " my $ret = eval 'script.pl';" That doesn't do
what I think you think it does. You probably want a do() or a require().
The error is appropriate since script.pl was never read.

Changing the eval to a require so the line reads, " my $ret = require
'script.pl';", gives
____________________________________________________________________
C​:\Documents and Settings\Owner\Desktop\cpan libs\prtbug\bug3>main
Evaluate builtin.pl
Setup the XS routine
Call builtinInit
do script.pl
Too many arguments for main​::iout at script.pl line 5, near ""Hi")"
Compilation failed in require at builtin.pl line 3.

C​:\Documents and Settings\Owner\Desktop\cpan libs\prtbug\bug3>
____________________________________________________________________
"Too many arguments for " is an error from perl prototypes system.
Looking in main.c I see " newXSproto("iout",XS_iout,__FILE__,"");", if I
change the line to " newXSproto("iout",XS_iout,__FILE__,NULL);" and
recompile, I get
____________________________________________________________________
C​:\Documents and Settings\Owner\Desktop\cpan libs\prtbug\bug3>main
Evaluate builtin.pl
Setup the XS routine
Call builtinInit
do script.pl
Eval returns​: 1 and
in fubar0
HiDone with script.pl
Done

C​:\Documents and Settings\Owner\Desktop\cpan libs\prtbug\bug3>
_____________________________________________________________________

I dont see a C bug in Perl here. You used "newXSproto" and gave a
prototype. If there is a bug, it is the POD for newXSproto at
http​://perldoc.perl.org/perlapi.html isnt listing a C prototype nor
describes the params that func/macro takes (I think that should be fixed).
--
bulk88 ~ bulk88 at hotmail.com