smoke-me/leont/perlio-win32 branch

[p5pRT]

Migrated from rt.perl.org#122224 (status was 'resolved')

Searchable as RT122224$

[p5pRT]

From @bulk88

Created by @bulk88

After some offline discussion with leont, he created
http​://perl5.git.perl.org/perl.git/shortlog/refs/heads/smoke-me/leont/perlio-win32
. The change

-------------------------------------------------------------
--- a/win32/win32io.c
+++ b/win32/win32io.c
@​@​ -323,7 +323,7 @​@​ PerlIOWin32_dup(pTHX_ PerlIO *f, PerlIO *o,
CLONE_PARAMS *params, int flags)
  PerlIOWin32 *os = PerlIOSelf(f,PerlIOWin32);
  HANDLE proc = GetCurrentProcess();
  HANDLE new_h;
- if (DuplicateHandle(proc, os->h, proc, &new_h, 0, FALSE,
DUPLICATE_SAME_ACCESS))
+ if (DuplicateHandle(proc, os->h, proc, &new_h, 0, TRUE,
DUPLICATE_SAME_ACCESS))
  {
  char mode[8];
  int fd = win32_open_osfhandle((intptr_t) new_h,
PerlIOUnix_oflags(PerlIO_modestr(o,mode)));

-------------------------------------------------------------

reflects correcting PerlIOWin32_dup to MS CRT's implementation of dup.
Obviously we want the handle to exist in a child process
(system/backticks/open).

But this branch SEGVs. Callstack

-------------------------------------------------------------

  perl521.dll!PerlIOWin32_dup(interpreter * my_perl=0x00364dbc, _PerlIO * * f=0x008f62cc, _PerlIO * * o=0x008f677c, clone_params * params=0x00000000, int flags=2) Line 326 + 0xb C
  perl521.dll!PerlIOBase_dup(interpreter * my_perl=0x00364dbc, _PerlIO * * f=0x008f62cc, _PerlIO * * o=0x008f627c, clone_params * param=0x00000000, int flags=2) Line 2207 C
  perl521.dll!PerlIO_fdupopen(interpreter * my_perl=0x0012fbb0, _PerlIO * * f=0x00070023, clone_params * param=0x00000000, int flags=2) Line 506 + 0x13 C
  perl521.dll!Perl_do_open6(interpreter * my_perl=0x00364dbc, gv * gv=0x00e6de6c, const char * oname=0x00ebc87c, unsigned int len=3, _PerlIO * * supplied_fp=0x00000000, sv * * svp=0x00f9d4f4, unsigned long num_svs=0) Line 443 + 0x10 C
  perl521.dll!Perl_pp_open(interpreter * my_perl=0x00364dbc) Line 638 C
  perl521.dll!Perl_runops_standard(interpreter * my_perl=0x00364dbc) Line 41 + 0x4 C
  perl521.dll!S_run_body(interpreter * my_perl=0x008f62cc, long oldscope=1) Line 2408 + 0xa C
  perl521.dll!perl_run(interpreter * my_perl=0x00364dbc) Line 2331 + 0x8 C
  perl521.dll!RunPerl(int argc=3, char * * argv=0x01364d58, char * * env=0x00363380) Line 258 + 0x6 C
  perl.exe!mainCRTStartup() Line 398 + 0xe C
  kernel32.dll!_BaseProcessStart@​4() + 0x23
-----------------------------------------------

at

-----------------------------------------------
PerlIO *
PerlIOWin32_dup(pTHX_ PerlIO *f, PerlIO *o, CLONE_PARAMS *params, int flags)
{
PerlIOWin32 *os = PerlIOSelf(f,PerlIOWin32);
HANDLE proc = GetCurrentProcess();
HANDLE new_h;
if (DuplicateHandle(proc, os->h, proc, &new_h, 0, TRUE, DUPLICATE_SAME_ACCESS)) <<<<<<<<<<<<<<<<<<<<<
  {
  char mode[8];
  int fd = win32_open_osfhandle((intptr_t) new_h, PerlIOUnix_oflags(PerlIO_modestr(o,mode)));
------------------------------------------------

because var os of type PerlIOWin32 * is null.

dumping vars f and o shows f is a ptr to NULL, o is a ptr to something
valid.

-----------------------------------------------------
- f 0x008f6cb4 _PerlIO * *
+ 0x00000000 {next=??? tab=??? flags=??? ...} _PerlIO *
- o 0x00365cfc _PerlIO * *
+ 0x008f709c {next=0x00000000 {next=??? tab=??? flags=??? ...}
tab=0x282bd478 _PerlIO_win32 flags=2114560 ...} _PerlIO *
-----------------------------------------------------

crash line in PP is

http​://perl5.git.perl.org/perl.git/blob/smoke-me/leont/perlio-win32​:/dist/IO/lib/IO/Handle.pm#l379

----------------------------------------------------
sub fdopen {
  @​_ == 3 or croak 'usage​: $io->fdopen(FD, MODE)';
  my ($io, $fd, $mode) = @​_;
  local(*GLOB);

  if (ref($fd) && "".$fd =~ /GLOB\(/o) {
  # It's a glob reference; Alias it as we cannot get name of anon GLOBs
  my $n = qualify(*GLOB);
  *GLOB = *{*$fd};
  $fd = $n;
  } elsif ($fd =~ m#^\d+$#) {
  # It's an FD number; prefix with "=".
  $fd = "=$fd";
  }

  open($io, _open_mode_string($mode) . '&' . $fd)<<<<<<<<<<<<<<<<<<<<
  ? $io : undef;
}
-----------------------------------------------------------

The process that crashed was "..\perl.exe -I..\lib harness".

I dont have any background in perlio on PP or C level. What are vars
PerlIO *f, PerlIO *o in Dup? what does "f" and "o" mean?
http​://perldoc.perl.org/perliol.html explains nothing.

----------------------------------------------------------
PerlIO *
PerlIOUnix_dup(pTHX_ PerlIO *f, PerlIO *o, CLONE_PARAMS *param, int flags)
{
  const PerlIOUnix * const os = PerlIOSelf(o, PerlIOUnix);
  int fd = os->fd;
  if (flags & PERLIO_DUP_FD) {
  fd = PerlLIO_dup(fd);
  }
  if (fd >= 0) {
  f = PerlIOBase_dup(aTHX_ f, o, param, flags);
  if (f) {
--------------------------------------------------------
PerlIOWin32_dup(pTHX_ PerlIO *f, PerlIO *o, CLONE_PARAMS *params, int flags)
{
PerlIOWin32 *os = PerlIOSelf(f,PerlIOWin32);
HANDLE proc = GetCurrentProcess();
HANDLE new_h;
if (DuplicateHandle(proc, os->h, proc, &new_h, 0, TRUE,
DUPLICATE_SAME_ACCESS))
  {
  char mode[8];
  int fd = win32_open_osfhandle((intptr_t) new_h,
PerlIOUnix_oflags(PerlIO_modestr(o,mode)));
--------------------------------------------------------
Unix_dup uses var o first. Win32_dup uses var f first. Should Win32_dup
be using var o first? its not a ptr to NULL unlike var f. IDK.

Perl Info

Flags:
        category=core
        severity=low

Site configuration information for perl 5.21.1:

Configured by Owner at Wed May 28 03:57:28 2014.

Summary of my perl5 (revision 5 version 21 subversion 1) configuration:
      Local Commit: 1abbcfa06576bf8a6937c566bb4d18ba803b59d8
      Ancestor: 234105dd8a732de5fb48ccb1838c99281f89f669
      Platform:
        osname=MSWin32, osvers=5.1, archname=MSWin32-x86-multi-thread
        uname=''
        config_args='undef'
        hint=recommended, useposix=true, d_sigaction=undef
        useithreads=define, usemultiplicity=define
        use64bitint=undef, use64bitall=undef, uselongdouble=undef
        usemymalloc=n, bincompat5005=undef
      Compiler:
        cc='cl', ccflags ='-nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -G7 -GL
-DWIN32 -D_CONSOLE -DNO_STRICT  -DPERL_TEXTMODE_SCRIPTS
-DPERL_HASH_FUNC_ONE_AT_A_TIME -DPERL_IMPLICIT_CONTEXT
-DPERL_IMPLICIT_SYS -DUSE_PERLIO -D_USE_32BIT_TIME_T',
        optimize='-O1 -MD -Zi -DNDEBUG -G7 -GL',
        cppflags='-DWIN32'
        ccversion='13.10.6030', gccversion='', gccosandvers=''
        intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
        d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=8
        ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64',
lseeksize=8
        alignbytes=8, prototype=define
      Linker and Libraries:
        ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf
-ltcg  -libpath:"c:\perl521\lib\CORE"  -machine:x86'
        libpth="C:\Program Files\Microsoft Visual Studio .NET 2003\VC7\lib"
        libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib
comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib
netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib  version.lib
odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib
        perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib
winspool.lib  comdlg32.lib advapi32.lib shell32.lib ole32.lib
oleaut32.lib  netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib
version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib
        libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl521.lib
        gnulibc_version=''
      Dynamic Linking:
        dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
        cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug
-opt:ref,icf -ltcg  -libpath:"c:\perl521\lib\CORE"  -machine:x86'

Locally applied patches:
        7494266ea378a3cdc4bfd51725012c1e432db0f1
        61961437d9453dd0d4053ad100e97a029a24edbb
        cd30b936fc5177ce169d776445d09c9898c15da1
        1abbcfa06576bf8a6937c566bb4d18ba803b59d8


@INC for perl 5.21.1:
        C:/perl521/site/lib
        C:/perl521/lib
        .


Environment for perl 5.21.1:
        HOME (unset)
        LANG (unset)
        LANGUAGE (unset)
        LD_LIBRARY_PATH (unset)
        LOGDIR (unset)
        PATH=C:\perl521\bin;C:\Program Files\Microsoft Visual Studio .NET
2003\Common7\IDE;C:\Program Files\Microsoft Visual Studio .NET
2003\VC7\BIN;C:\Program Files\Microsoft Visual Studio .NET
2003\Common7\Tools;C:\Program Files\Microsoft Visual Studio .NET
2003\Common7\Tools\bin\prerelease;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\system32\wbem;
        PERL_BADLANG (unset)
        SHELL (unset)

[p5pRT]

From @Leont

On Fri, Jul 4, 2014 at 2​:21 AM, bulk88 <perlbug-followup@​perl.org> wrote​:

reflects correcting PerlIOWin32_dup to MS CRT's implementation of dup.
Obviously we want the handle to exist in a child process
(system/backticks/open).

But this branch SEGVs because var os of type PerlIOWin32 * is null.

dumping vars f and o shows f is a ptr to NULL, o is a ptr to something
valid.

I dont have any background in perlio on PP or C level. What are vars
PerlIO *f, PerlIO *o in Dup? what does "f" and "o" mean?
http​://perldoc.perl.org/perliol.html explains nothing.

Unix_dup uses var o first. Win32_dup uses var f first. Should Win32_dup
be using var o first? its not a ptr to NULL unlike var f. IDK.

Yes, it should use o instead of f there. f is the new PerlIO handle that is
being constructed, however at that stage it's still pointing at thing air.
Only after the PerlIOBase_dup it's something remotely usable. o (old?
other?) is pointing to the handle that is being duplicated, so it's
definitely the thing to use to get to the handle that must be duplicated.

I pushed a new patch to that branch, I hope that will get a little further.

Leon

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @bulk88

On Thu Jul 03 19​:04​:24 2014, LeonT wrote​:

I pushed a new patch to that branch, I hope that will get a little further.

Leon

Slightly better, now an assert fails.

---

  set PERL_STATIC_EXT=Win32CORE
  cd ..\t
  ..\perl.exe -I..\lib harness
PerlIO layer '​:win32' is experimental.
PerlIO layer '​:win32' is experimental at harness line 13.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Harness.pm line 3.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Harness.pm line 4.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Harness.pm line 5.
PerlIO layer '​:win32' is experimental at ../lib/Carp.pm line 99.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Harness.pm line 7.
PerlIO layer '​:win32' is experimental at ../lib/File/Spec.pm line 4.
PerlIO layer '​:win32' is experimental at ../lib/vars.pm line 7.
PerlIO layer '​:win32' is experimental at ../lib/File/Spec.pm line 22.
PerlIO layer '​:win32' is experimental at ../lib/File/Spec/Win32.pm line 6.
PerlIO layer '​:win32' is experimental at ../lib/File/Spec/Unix.pm line 127.
PerlIO layer '​:win32' is experimental at ../lib/File/Spec/Unix.pm line 13.
PerlIO layer '​:win32' is experimental at ../lib/XSLoader.pm line 96.
PerlIO layer '​:win32' is experimental at ../lib/DynaLoader.pm line 22.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Harness.pm line 8.
PerlIO layer '​:win32' is experimental at ../lib/File/Path.pm line 6.
PerlIO layer '​:win32' is experimental at ../lib/File/Path.pm line 7.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Harness.pm line 9.
PerlIO layer '​:win32' is experimental at ../lib/IO/Handle.pm line 267.
PerlIO layer '​:win32' is experimental at ../lib/IO/Handle.pm line 268.
PerlIO layer '​:win32' is experimental at ../lib/IO/Handle.pm line 269.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Harness.pm line 11.
PerlIO layer '​:win32' is experimental at ../lib/base.pm line 99.
PerlIO layer '​:win32' is experimental at ../lib/base.pm line 99.
PerlIO layer '​:win32' is experimental at (eval 5) line 1.
PerlIO layer '​:win32' is experimental at ../lib/Exporter.pm line 16.
PerlIO layer '​:win32' is experimental at harness line 78.
PerlIO layer '​:win32' is experimental at harness line 18.
PerlIO layer '​:win32' is experimental at ../lib/Config.pm line 80.
PerlIO layer '​:win32' is experimental at ../lib/Config_heavy.pl line 1244.
PerlIO layer '​:win32' is experimental at ./TEST line 399.
PerlIO layer '​:win32' is experimental at harness line 215.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser.pm line 6.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/Grammar.pm line 6.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/ResultFactory.pm line
6.
PerlIO layer '​:win32' is experimental at ../lib/base.pm line 99.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/ResultFactory.pm line
7.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/ResultFactory.pm line
8.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/ResultFactory.pm line
9.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/ResultFactory.pm line
10.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/ResultFactory.pm line
11.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/ResultFactory.pm line
12.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/ResultFactory.pm line
13.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/Grammar.pm line 7.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser.pm line 9.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser.pm line 10.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser.pm line 11.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser.pm line 12.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/SourceHandler/Executa
ble.pm line 7.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/Iterator/Process.pm l
ine 84.
PerlIO layer '​:win32' is experimental at ../lib/POSIX.pm line 17.
PerlIO layer '​:win32' is experimental at ../lib/POSIX.pm line 422.
PerlIO layer '​:win32' is experimental at ../lib/base.pm line 99.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser.pm line 13.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/SourceHandler/Perl.pm
line 12.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser.pm line 14.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/SourceHandler/File.pm
line 7.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser.pm line 15.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/SourceHandler/RawTAP.
pm line 7.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser.pm line 16.
PerlIO layer '​:win32' is experimental at (eval 8) line 2.
PerlIO layer '​:win32' is experimental at ../lib/base.pm line 99.
PerlIO layer '​:win32' is experimental at (eval 9) line 2.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/Aggregator.pm line 5.

PerlIO layer '​:win32' is experimental at (eval 11) line 2.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/Scheduler.pm line 7.
PerlIO layer '​:win32' is experimental at ../lib/TAP/Parser/Scheduler.pm line 8.
PerlIO layer '​:win32' is experimental at ./TEST line 192.
PerlIO layer '​:win32' is experimental at (eval 12) line 2.
PerlIO layer '​:win32' is experimental at (eval 13) line 2.
PerlIO layer '​:win32' is experimental at ../lib/IPC/Open3.pm line 154.
PerlIO layer '​:win32' is experimental at ../lib/IPC/Open3.pm line 154.
PerlIO layer '​:win32' is experimental at ../lib/IPC/Open3.pm line 334.
PerlIO layer '​:win32' is experimental at ../lib/IO/Handle.pm line 379.
PerlIO layer '​:win32' is experimental at ../lib/IO/Handle.pm line 379.
Assertion failed​: seen, file ..\perlio.c, line 444

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
NMAKE : fatal error U1077​: '..\perl.exe' : return code '0x3'
Stop.

C​:\perl521\src\win32>

---

---

#ifdef DEBUGGING
# define VERIFY_HEAD(f) PerlIO_verify_head(aTHX_ f)
static void
PerlIO_verify_head(pTHX_ PerlIO *f)
{
  PerlIOl *head, *p;
  int seen = 0;
#ifndef PERL_IMPLICIT_SYS
  PERL_UNUSED_CONTEXT;
#endif
  if (!PerlIOValid(f))
  return;
  p = head = PerlIOBase(f)->head;
  assert(p);
  do {
  assert(p->head == head);
  if (p == (PerlIOl*)f)
  seen = 1;
  p = p->next;
  } while (p);
  assert(seen);<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
}
#else
# define VERIFY_HEAD(f)
#endif

---

  ntdll.dll!_DbgBreakPoint@​0()
  perl521.dll!PerlIO_verify_head(interpreter * my_perl=0x00364dd4, _PerlIO * * f=0x008f6e2c) Line 445 C
  perl521.dll!PerlIO_push(interpreter * my_perl=0x00364dd4, _PerlIO * * f=0x008f6e2c, _PerlIO_funcs * tab=0x282bbf00, const char * mode=0x0012fc00, sv * arg=0x00364e74) Line 1150 + 0xd C
  perl521.dll!PerlIOBuf_open(interpreter * my_perl=0x00364dd4, _PerlIO_funcs * self=0x282bbf00, PerlIO_list_s * layers=0x008f6c2c, long n=1, const char * mode=0x0012fc00, int fd=9, int imode=0, int perm=0, _PerlIO * * f=0x008f6e2c, int narg=0, sv * * args=0x00000000) Line 3830 + 0x26 C
  perl521.dll!PerlIO_openn(interpreter * my_perl=0x00364dd4, const char * layers=0x00000000, const char * mode=0x0012fc00, int fd=9, int imode=0, int perm=0, _PerlIO * * f=0x00000000, int narg=0, sv * * args=0x00000000) Line 1565 + 0x32 C
  perl521.dll!S_openn_cleanup(interpreter * my_perl=0x00364dd4, gv * gv=0x00ee189c, io * io=0x00dba14c, _PerlIO * * fp=0x008f6e1c, char * mode=0x0012fc00, const char * oname=0x00f3a614, _PerlIO * * saveifp=0x00000000, _PerlIO * * saveofp=0x00000000, int savefd=-1, char savetype=' ', int writing=1, char was_fdopen=0, const char * type=0x00000000) Line 783 + 0x22 C
  perl521.dll!Perl_do_open6(interpreter * my_perl=0x00364dd4, gv * gv=0x00ee189c, const char * oname=0x00f3a614, unsigned int len=19, _PerlIO * * supplied_fp=0x00000000, sv * * svp=0x010330c4, unsigned long num_svs=0) Line 601 + 0x39 C
  perl521.dll!Perl_pp_open(interpreter * my_perl=0x00364dd4) Line 637 + 0x28 C
  perl521.dll!Perl_runops_debug(interpreter * my_perl=0x00364dd4) Line 2360 + 0xd C
  perl521.dll!S_run_body(interpreter * my_perl=0x00364dd4, long oldscope=1) Line 2408 + 0xd C
  perl521.dll!perl_run(interpreter * my_perl=0x00364dd4) Line 2334 C
  perl521.dll!RunPerl(int argc=3, char * * argv=0x00364d70, char * * env=0x00365ed8) Line 258 + 0x9 C++
  perl.exe!main(int argc=3, char * * argv=0x00364d70, char * * env=0x00363390) Line 23 + 0x12 C
  perl.exe!mainCRTStartup() Line 398 + 0xe C
  kernel32.dll!_BaseProcessStart@​4() + 0x23

---

PP location is same as before, http​://perl5.git.perl.org/perl.git/blob/smoke-me/leont/perlio-win32​:/dist/IO/lib/IO/Handle.pm#l379

I attached a dump of var f from PerlIO_verify_head's frame.

--
bulk88 ~ bulk88 at hotmail.com

[p5pRT]

From @bulk88

var_f_dump.PNG

[p5pRT]

From @bulk88

On Thu Jul 03 19​:37​:28 2014, bulk88 wrote​:

I attached a dump of var f from PerlIO_verify_head's frame.

Inside PerlIO_verify_head, there is

---

  if (p == (PerlIOl*)f)
  seen = 1;

---

but the cast makes things strange. var f is a PerlIO **, p is a PerlIO *, var f is the realloc permitting indirection layer (the PerlIO **), therefore we shouldn't ever find a PerlIO * and a PerlIO ** that match. Notice 0x00f3a7a4, which is a pointer to the :win32 _PerlIO/PerlIOWin32 struct, does appear in the chain going down from head. Unless the bug is really that f->head should be 0x008f6e2c and not 0x008f6e1c as in the pic.

--
bulk88 ~ bulk88 at hotmail.com

[p5pRT]

From @bulk88

On Thu Jul 03 19​:49​:56 2014, bulk88 wrote​:

On Thu Jul 03 19​:37​:28 2014, bulk88 wrote​:

I attached a dump of var f from PerlIO_verify_head's frame.

Inside PerlIO_verify_head, there is

-------------------------------------
if (p == (PerlIOl*)f)
seen = 1;
-------------------------------------

but the cast makes things strange. var f is a PerlIO **, p is a PerlIO
*, var f is the realloc permitting indirection layer (the PerlIO **),
therefore we shouldn't ever find a PerlIO * and a PerlIO ** that
match. Notice 0x00f3a7a4, which is a pointer to the :win32
_PerlIO/PerlIOWin32 struct, does appear in the chain going down from
head. Unless the bug is really that f->head should be 0x008f6e2c and
not 0x008f6e1c as in the pic.

Maybe the bug is PerlIOWin32_dup will return non-NULL/var f if DuplicateHandle fails (returns 0)??? Unix_dup returns PerlIO * NULL if PerlLIO_dup returns a negative number. The error handling logic in PerlIOWin32_dup is to return a non-NULL ptr by default, in Unix_dup the logic is to return NULL by default.

--
bulk88 ~ bulk88 at hotmail.com

[p5pRT]

From @bulk88

Maybe the bug is "new" one after head was introduced in http​://perl5.git.perl.org/perl.git/commit/16865ff7e97c2532fd2001d68cf18909acb0d838 in 2010. PerlIOUnix_open uses
PerlIO_push to fill in var f so f isn't a ptr to null

---

  if (!f) {
  f = PerlIO_allocate(aTHX);
  }
  if (!PerlIOValid(f)) {
  if (!(f = PerlIO_push(aTHX_ f, self, mode, PerlIOArg))) {<<<<<<<<<<<<<<<<<
  PerlLIO_close(fd);
  return NULL;
  }
  }

---

, PerlIOWin32_open uses http​://perl5.git.perl.org/perl.git/blob/smoke-me/leont/perlio-win32​:/win32/win32io.c#l169 " *f = &s->base;"
--
bulk88 ~ bulk88 at hotmail.com

[p5pRT]

From @bulk88

On Fri Jul 04 01​:14​:37 2014, bulk88 wrote​:

Maybe the bug is "new" one after head was introduced in
http​://perl5.git.perl.org/perl.git/commit/16865ff7e97c2532fd2001d68cf18909acb0d838
in 2010. PerlIOUnix_open uses
PerlIO_push to fill in var f so f isn't a ptr to null

------------------------------------------
if (!f) {
f = PerlIO_allocate(aTHX);
}
if (!PerlIOValid(f)) {
if (!(f = PerlIO_push(aTHX_ f, self, mode, PerlIOArg)))
{<<<<<<<<<<<<<<<<<
PerlLIO_close(fd);
return NULL;
}
}
------------------------------------------

, PerlIOWin32_open uses http​://perl5.git.perl.org/perl.git/blob/smoke-
me/leont/perlio-win32​:/win32/win32io.c#l169 " *f = &s->base;"

I am not sure that this can be solved without reverting http​://perl5.git.perl.org/perl.git/commit/16865ff7e97c2532fd2001d68cf18909acb0d838 . If all PerlIOl* structs must have a head member, then they (a layer instance) can never be shared between 2 different head layer instances.

--
bulk88 ~ bulk88 at hotmail.com

[p5pRT]

From @iabyn

On Sat, Jul 05, 2014 at 03​:47​:57PM -0700, bulk88 via RT wrote​:

On Fri Jul 04 01​:14​:37 2014, bulk88 wrote​:

Maybe the bug is "new" one after head was introduced in
http​://perl5.git.perl.org/perl.git/commit/16865ff7e97c2532fd2001d68cf18909acb0d838
in 2010. PerlIOUnix_open uses
PerlIO_push to fill in var f so f isn't a ptr to null

------------------------------------------
if (!f) {
f = PerlIO_allocate(aTHX);
}
if (!PerlIOValid(f)) {
if (!(f = PerlIO_push(aTHX_ f, self, mode, PerlIOArg)))
{<<<<<<<<<<<<<<<<<
PerlLIO_close(fd);
return NULL;
}
}
------------------------------------------

, PerlIOWin32_open uses http​://perl5.git.perl.org/perl.git/blob/smoke-
me/leont/perlio-win32​:/win32/win32io.c#l169 " *f = &s->base;"

I am not sure that this can be solved without reverting
http​://perl5.git.perl.org/perl.git/commit/16865ff7e97c2532fd2001d68cf18909acb0d838
. If all PerlIOl* structs must have a head member, then they (a layer
instance) can never be shared between 2 different head layer instances.

They are not supposed to be shared. If anything is shared, something has
gone wrong.

A bit of general background explanation - hopefully this will make it
easier for you to decode what' going wrong (I haven't looked hard enough
to determine that).

The basic data type is a layer​:

  typedef struct _PerlIO PerlIOl;

  struct _PerlIO {
  PerlIOl *next; /* Lower layer */
  PerlIO_funcs *tab; /* Functions for this layer */
  U32 flags; /* Various flags for state */
  PerlIOl *head; /* our ultimate parent pointer */
  };

where the layers form a chain of layer->next's.

The perl file handle table PL_perlio is a big array of PerlIOl's, although
only the 'next' field is directly used​: i.e. the first layer isn't stored
as PL_perlio[N]; instead PL_perlio[N].next points to the first layer.

PerlIO is a sort of fake data structure that just contains the address of
the 'next' pointer, i.e. for a file handle f in slot N, you would do

  f = (PerlIO *) &(PL_perlio[N].next);

Maybe some ascii diagrams will help. Consider a file handle f in slot N
which has two layers​:

  PL_perlio

  | |
  N-1 | |
  +-------+ +-------+ +-------+
f ---->| next | --> | next |----> | next | --> NULL
  |(head )| | head | | head |
  N |(tab )| | tab | | tab |
  |(flags)| | flags | | flags |
  +-------+ +-------+ +-------+
  | |
  N+1 | |

Not shown are the two 'head' pointers pointing back to the same thing f is
pointing to.

When a file handle is being duped (e.g. open my $newfh, ">&STDOUT"),
a new empty file handle is created​:

  +-------+
f2---->| next | --> NULL
  |(head )|
  N+1|(tab )|
  |(flags)|
  +-------+

then PerlIOBase_dup() recursively copies each layer, from the bottom up,
using PerlIO_push() to push each new layer. Each push inserts a new layer
between the head and the current top layer. By the end of duplication, it
should look like​:

  PL_perlio

  | |
  N-1 | | +-------+ +-------+
  +-------+ | next |----> | next | --> NULL
f ---->| next | --> | head | | head |
  |(head )| | tab | | tab |
  N |(tab )| | flags | | flags |
  |(flags)| +-------+ +-------+
  +-------+
f2---->| next | --> +-------+ +-------+
  |(head )| | next |----> | next | --> NULL
  N+1 |(tab )| | head | | head |
  |(flags)| | tab | | tab |
  +-------+ | flags | | flags |
  | | +-------+ +-------+
  N+2 | |

PerlIOUnix_dup (and presumably PerlIOWin32_dup) gets called with f being
the new partially constructed filehandle, and o being the current layer
within the original filehandle being duped.

PerlIOUnix_dup() calls the OS-level function to duplicate the physical
file descriptor, then calls PerlIOBase_dup(), which will​: recursively
duplicate any further-down layers (which there aren't in this case), then
return with a new layer pushed at the head of the new handle f.
PerlIOBase_dup() then sets the new fd in this new layer, and returns f.
(I think earlier you mistakenly concluded that PerlIOUnix_dup returns
NULL on success; it in fact returns NULL on failure.)

PerlIO_verify_head() for a handle or layer just checks that all the head
pointers in a layer chain point to the head, i.e.​:

  | | ----<-------------<---------
  | | / \ \
  N-1 | |L \ \
  +-------+ +-------+ / +-------+ /
f ---->| next | | next | / | next | /
  |(head )| | head |- | head |-
  N |(tab )| | tab | | tab |
  |(flags)| | flags | | flags |
  +-------+ +-------+ +-------+
  | |
  N+1 | |
--
My get-up-and-go just got up and went.

[p5pRT]

From @Leont

On Mon, Jul 7, 2014 at 2​:29 PM, Dave Mitchell <davem@​iabyn.com> wrote​:

They are not supposed to be shared. If anything is shared, something has
gone wrong.

A bit of general background explanation - hopefully this will make it
easier for you to decode what' going wrong (I haven't looked hard enough
to determine that).

The basic data type is a layer​:

typedef struct \_PerlIO PerlIOl;

struct \_PerlIO \{
    PerlIOl \*next;          /\* Lower layer \*/
    PerlIO\_funcs \*tab;              /\* Functions for this layer \*/
    U32 flags;                      /\* Various flags for state \*/
    PerlIOl \*head;          /\* our ultimate parent pointer \*/
\};

where the layers form a chain of layer->next's.

The perl file handle table PL_perlio is a big array of PerlIOl's, although
only the 'next' field is directly used​: i.e. the first layer isn't stored
as PL_perlio[N]; instead PL_perlio[N].next points to the first layer.

PerlIO is a sort of fake data structure that just contains the address of
the 'next' pointer, i.e. for a file handle f in slot N, you would do

f = \(PerlIO \*\) &\(PL\_perlio\[N\]\.next\);

I must admit I'm not fond of this design, it's rather confusing IME, a
special purpose struct that has exactly the right fields would be easier to
grok.

Leon

[p5pRT]

From @bulk88

After removing the warning which stops the segv during fork, I got a make test to run, results look bad. And there are 2 different behaviors based on whether I'm redirecting stdio fds to a file using cmd prompt shell or letting them goto console by default. Doing shell redirect to a file gives

---

  xcopy /f /r /i /d /y ..\perlglob.exe ..\t\
0 File(s) copied
  set PERL_STATIC_EXT=Win32CORE
  cd ..\t
  ..\perl.exe -I..\lib harness
base/cond.t .......................................................
No subtests run
base/if.t .........................................................
No subtests run
base/lex.t ........................................................
No subtests run
base/num.t ........................................................
No subtests run
base/pat.t ........................................................
No subtests run
base/rs.t .........................................................
No subtests run
base/term.t .......................................................
Dubious, test returned 9 (wstat 2304, 0x900)
No subtests run
base/while.t ......................................................
No subtests run
comp/bproto.t .....................................................
No subtests run
comp/cmdopt.t .....................................................
No subtests run
comp/colon.t ......................................................
No subtests run
comp/decl.t .......................................................
No subtests run
comp/final_line_num.t .............................................
No subtests run
comp/fold.t .......................................................
No subtests run
comp/form_scope.t .................................................
No subtests run
comp/hints.t ......................................................
No subtests run
comp/line_debug.t .................................................
No subtests run
comp/multiline.t ..................................................
CUTTTTTTTTTTTTTTTTTTTTTTT
  Parse errors​: No plan found in TAP output
../lib/utf8.t (Wstat​: 0 Tests​: 0 Failed​: 0)
  Parse errors​: No plan found in TAP output
../lib/vars.t (Wstat​: 0 Tests​: 0 Failed​: 0)
  Parse errors​: No plan found in TAP output
../lib/vars_carp.t (Wstat​: 0 Tests​: 0 Failed​: 0)
  Parse errors​: No plan found in TAP output
../lib/vmsish.t (Wstat​: 0 Tests​: 0 Failed​: 0)
  Parse errors​: No plan found in TAP output
../lib/warnings.t (Wstat​: 0 Tests​: 0 Failed​: 0)
  Parse errors​: No plan found in TAP output
Files=2318, Tests=0, 2277 wallclock secs (16.22 usr + 2.58 sys = 18.80 CPU)
Result​: FAIL

---

The test are running CPU wise.

If I dont redirect IO, the ok/not ok lines are printing in the console. Many not oks are flashing by.

--
bulk88 ~ bulk88 at hotmail.com

[p5pRT]

From @bulk88

Flushing a segv fix out of my src tree.

--
bulk88 ~ bulk88 at hotmail.com

[p5pRT]

From @bulk88

0001-fix-crash-in-PerlIOWin32_open.patch

From ecafd7654a8d1bc705e8fe0cd3c89044374db045 Mon Sep 17 00:00:00 2001
From: Daniel Dragan <bulk88@hotmail.com>
Date: Fri, 11 Jul 2014 00:10:49 -0400
Subject: [PATCH] fix crash in PerlIOWin32_open

checking if(f) isn't enough, *f also must be checked or it will crash,
use the correct PerlIOValid macro instead of just checking f
---
 win32/win32io.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/win32/win32io.c b/win32/win32io.c
index 5e357b5..9ac2621 100644
--- a/win32/win32io.c
+++ b/win32/win32io.c
@@ -69,7 +69,7 @@ PerlIOWin32_open(pTHX_ PerlIO_funcs *self, PerlIO_list_t *layers, IV n, const ch
 {
  const char *tmode = mode;
  HANDLE h = INVALID_HANDLE_VALUE;
- if (f)
+ if (PerlIOValid(f))
   {
    /* Close if already open */
    if (PerlIOBase(f)->flags & PERLIO_F_OPEN)
-- 
1.7.9.msysgit.0

[p5pRT]

From @bulk88

Leont's revert the warning commit has a problem. Turning off the :win32 is default line to get back a regular perl, shows this failing.

---

C​:\perl521\src\t>.\perl -I../lib harness -v op/caller.t
op/caller.t .. # Failed test 2 - all bits off via "no warnings" at op/caller.t l
ine 114
# got​: \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
# expected​: \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
# Failed test 26 - warnings match caller (no bits) at op/caller.t line 115
# got​: \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00
# expected​: \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00

1..95
# Tests with caller(0)
ok 1 - [perl \#68712] filenames after require in a BEGIN block
not ok 2 - all bits off via "no warnings"
ok 3 - default bits on via "use warnings"
ok 4 - warnings match caller (all)
ok 5 - warning bits on via "use warnings​::register"
ok 6 - caller(0) in main program
ok 7 - subroutine name in an eval {}
ok 8 - hasargs false in an eval {}
ok 9 - subroutine name in an eval ''
ok 10 - hasargs false in an eval ''
ok 11 - anonymous subroutine name
ok 12 - hasargs true with anon sub
ok 13 - deleted subroutine name
ok 14 - hasargs true with deleted sub
# Tests with caller(1)
ok 15 - subroutine name
ok 16 - hasargs true with callf()
ok 17 - hasargs false with &callf
ok 18 - subroutine name in an eval {}
ok 19 - hasargs false in an eval {}
ok 20 - subroutine name in an eval ''
ok 21 - hasargs false in an eval ''
ok 22 - anonymous subroutine name
ok 23 - hasargs true with anon sub
ok 24 - deleted subroutine name
ok 25 - hasargs true with deleted sub
not ok 26 - warnings match caller (no bits)
ok 27 - warnings match caller (ahead of w​::r)
ok 28 - warnings match caller (following w​::r)
ok 29 - do not skip over eval (and caller returns 10 elements)
ok 30 - actually return the right function name
ok 31 - do not skip over eval even if $^P had been on at some point
ok 32 - actually return the right function name even if $^P had been on at some
point
# caller can now return the compile time state of %^H
ok 33 - @​DB​::args doesn't leak with $^P = 0
ok 34 - @​DB​::args doesn't leak with $^P = 1
ok 35 - caller does not leak @​DB​::args elems when AvREAL
ok 36 - No crash when @​DB​::args is freed between caller calls
ok 37 - caller dies with tie @​DB​::args
ok 38 - @​DB​::args is still tied
ok 39 - No crash when freed stash is reused for PV with offset hack
ok 40 - eval text returned by caller does not include \n;
ok 41 - [perl \#115768] caller gets line numbers from nulled cops
ok 42 - [perl \#115768] caller gets line numbers from nulled cops (2)
ok 43 - no assertion failure after modifying eval text via caller
ok 44 - here-docs do not gut eval text
ok 45 - here-docs in quote-like ops do not gut eval text
ok 46 - value from (caller 0)[9] (bitmask) works in ${^WARNING_BITS}
ok 47 - caller should not SEGV when the current package is undefined
ok 48 - caller should not SEGV for eval '' stack frames
ok 49
ok 50
ok 51
ok 52
ok 53
ok 54
ok 55
ok 56
ok 57
ok 58
ok 59
ok 60
ok 61
ok 62
ok 63
ok 64
ok 65
ok 66
ok 67
ok 68
ok 69
ok 70
ok 71
ok 72
ok 73
# which now works inside evals
ok 74
ok 75
ok 76
ok 77
ok 78
ok 79
ok 80
ok 81
ok 82 - Can do embedded 0 bytes
ok 83 - Can do Unicode
ok 84 - Can do IVs
ok 85 - Can do UVs
ok 86 - UTF-8 or not, it's the same
ok 87 - UTF-8 or not, it's the same
ok 88 - Octect sequences and UTF-8 are distinct
ok 89 - Octect sequences and UTF-8 are distinct
ok 90 - Keys with the same hash value don't clash
ok 91 - Keys with the same hash value don't clash
ok 92 - Keys with the same hash value don't clash
ok 93 - Keys with the same hash value don't clash
ok 94 - Keys with the same hash value don't clash
ok 95 - Keys with the same hash value don't clash
Failed 2/95 subtests

Test Summary Report

---

op/caller.t (Wstat​: 0 Tests​: 95 Failed​: 2)
  Failed tests​: 2, 26
Files=1, Tests=95, 0 wallclock secs ( 0.08 usr + 0.00 sys = 0.08 CPU)
Result​: FAIL

C​:\perl521\src\t>

---

Patch attached.

--
bulk88 ~ bulk88 at hotmail.com