Bleadperl v5.27.1-114-g9449f0d6ae breaks MLEHMANN/Glib-EV-2.02.tar.gz

[p5pRT]

Migrated from rt.perl.org#131942 (status was 'resolved')

Searchable as RT131942$

[p5pRT]

From @andk

Once again, affected are -DDEBUGGING builds only.

bisect

---

commit 9449f0d
Author​: David Mitchell <davem@​iabyn.com>
Date​: Mon Jul 3 13​:43​:27 2017 +0100

  save si_stack_hwm across JMPENV_PUSH

other distros also affected

---

MARKSMITH/HTTP-HeaderParser-XS-0.20.tar.gz
FRACTAL/IO-Vectored-0.110.tar.gz
SDPRICE/Linux-DVB-DVBT-Advert-0.04.tar.gz
SDPRICE/Linux-DVB-DVBT-TS-0.08.tar.gz

cpantesters report

---

http​://www.cpantesters.org/cpan/report/72a6d462-7362-11e7-a0cc-5c58e11ca5c1

perl -V

---

Summary of my perl5 (revision 5 version 27 subversion 2) configuration​:
  Commit id​: 9449f0d
  Platform​:
  osname=linux
  osvers=4.9.0-2-amd64
  archname=x86_64-linux
  uname='linux k93msid 4.9.0-2-amd64 #1 smp debian 4.9.18-1 (2017-03-30) x86_64 gnulinux '
  config_args='-Dprefix=/home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.27.1-114-g9449f0d6ae/d2d1 -Dmyhostname=k93msid -Dinstallusrbinperl=n -Uversiononly -Dusedevel -des -Ui_db -Dlibswanted=cl pthread socket inet nsl gdbm dbm malloc dl ld sun m crypt sec util c cposix posix ucb BSD gdbm_compat -Uuseithreads -Uuselongdouble -DDEBUGGING=both'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=undef
  usemultiplicity=undef
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
  optimize='-O2 -g'
  cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='6.3.0 20170406'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.24.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.24'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​:
  DEBUGGING
  HAS_TIMES
  PERLIO_LAYERS
  PERL_COPY_ON_WRITE
  PERL_DONT_CREATE_GVSV
  PERL_MALLOC_WRAP
  PERL_OP_PARENT
  PERL_PRESERVE_IVUV
  PERL_USE_DEVEL
  USE_64_BIT_ALL
  USE_64_BIT_INT
  USE_LARGE_FILES
  USE_LOCALE
  USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC
  USE_LOCALE_TIME
  USE_PERLIO
  USE_PERL_ATOF
  Built under linux
  Compiled at Aug 20 2017 19​:39​:36
  @​INC​:
  /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.27.1-114-g9449f0d6ae/d2d1/lib/site_perl/5.27.2/x86_64-linux
  /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.27.1-114-g9449f0d6ae/d2d1/lib/site_perl/5.27.2
  /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.27.1-114-g9449f0d6ae/d2d1/lib/5.27.2/x86_64-linux
  /home/sand/src/perl/repoperls/installed-perls/host/k93msid/v5.27.1-114-g9449f0d6ae/d2d1/lib/5.27.2

--
andreas

[p5pRT]

From @jkeenan

On Mon, 21 Aug 2017 20​:39​:56 GMT, andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

Once again, affected are -DDEBUGGING builds only.

bisect
------
commit 9449f0d
Author​: David Mitchell <davem@​iabyn.com>
Date​: Mon Jul 3 13​:43​:27 2017 +0100

save si_stack_hwm across JMPENV_PUSH

other distros also affected
---------------------------
MARKSMITH/HTTP-HeaderParser-XS-0.20.tar.gz
FRACTAL/IO-Vectored-0.110.tar.gz
SDPRICE/Linux-DVB-DVBT-Advert-0.04.tar.gz
SDPRICE/Linux-DVB-DVBT-TS-0.08.tar.gz

cpantesters report
------------------
http​://www.cpantesters.org/cpan/report/72a6d462-7362-11e7-a0cc-
5c58e11ca5c1

Examples of failures​:

#####
[HTTP-HeaderParser-XS-0.20] 547 $ /testing/blead/bin/prove -I/testing/blead/lib -vb t/10-basic.t
t/10-basic.t ..
1..39
HTTPHeaders.c​: loadable library and perl binaries are mismatched (got handshake key 0xd100000, needed 0xd180000)
Dubious, test returned 1 (wstat 256, 0x100)
Failed 39/39 subtests

Test Summary Report

---

t/10-basic.t (Wstat​: 256 Tests​: 0 Failed​: 0)
  Non-zero exit status​: 1
  Parse errors​: Bad plan. You planned 39 tests but ran 0.
Files=1, Tests=0, 0 wallclock secs ( 0.02 usr 0.00 sys + 0.03 cusr 0.00 csys = 0.05 CPU)
Result​: FAIL
#####
[IO-Vectored-0.110] 515 $ /testing/blead/bin/prove -I/testing/blead/lib -vb t/*.t
t/exceptions.t ..
1..6
Vectored.c​: loadable library and perl binaries are mismatched (got handshake key 0xd100000, needed 0xd180000)
Dubious, test returned 1 (wstat 256, 0x100)
Failed 6/6 subtests
t/normal.t ......
1..4
Vectored.c​: loadable library and perl binaries are mismatched (got handshake key 0xd100000, needed 0xd180000)
Dubious, test returned 1 (wstat 256, 0x100)
Failed 4/4 subtests
t/numeric.t .....
1..3
Vectored.c​: loadable library and perl binaries are mismatched (got handshake key 0xd100000, needed 0xd180000)
Dubious, test returned 1 (wstat 256, 0x100)
Failed 3/3 subtests

Test Summary Report

---

t/exceptions.t (Wstat​: 256 Tests​: 0 Failed​: 0)
  Non-zero exit status​: 1
  Parse errors​: Bad plan. You planned 6 tests but ran 0.
t/normal.t (Wstat​: 256 Tests​: 0 Failed​: 0)
  Non-zero exit status​: 1
  Parse errors​: Bad plan. You planned 4 tests but ran 0.
t/numeric.t (Wstat​: 256 Tests​: 0 Failed​: 0)
  Non-zero exit status​: 1
  Parse errors​: Bad plan. You planned 3 tests but ran 0.
Files=3, Tests=0, 0 wallclock secs ( 0.02 usr 0.01 sys + 0.10 cusr 0.00 csys = 0.13 CPU)
Result​: FAIL
#####

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @iabyn

On Tue, Aug 22, 2017 at 06​:35​:32PM -0700, James E Keenan via RT wrote​:

On Mon, 21 Aug 2017 20​:39​:56 GMT, andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

Once again, affected are -DDEBUGGING builds only.

bisect
------
commit 9449f0d
Author​: David Mitchell <davem@​iabyn.com>
Date​: Mon Jul 3 13​:43​:27 2017 +0100

save si_stack_hwm across JMPENV_PUSH

other distros also affected
---------------------------
MARKSMITH/HTTP-HeaderParser-XS-0.20.tar.gz
FRACTAL/IO-Vectored-0.110.tar.gz
SDPRICE/Linux-DVB-DVBT-Advert-0.04.tar.gz
SDPRICE/Linux-DVB-DVBT-TS-0.08.tar.gz

Examples of failures​:

#####
[HTTP-HeaderParser-XS-0.20] 547 $ /testing/blead/bin/prove -I/testing/blead/lib -vb t/10-basic.t
t/10-basic.t ..
1..39
HTTPHeaders.c​: loadable library and perl binaries are mismatched (got handshake key 0xd100000, needed 0xd180000)

They should hopefully be fixed with the commit below which I've just
pushed. I've only tested IO-Vectored, but I'm hoping/assuming they're all
the same cause and fix.

commit d2dccc0
Author​: David Mitchell <davem@​iabyn.com>
AuthorDate​: Wed Aug 23 10​:18​:53 2017 +0100
Commit​: David Mitchell <davem@​iabyn.com>
CommitDate​: Wed Aug 23 10​:18​:53 2017 +0100

  make 'struct jmpenv' size independent of DEBUGGING
 
  RT #131942
 
  My commit v5.27.1-114-g9449f0d added a field je_old_stack_hwm to
  the jmpenv structure on debugging builds.
 
  However, one field in the interpreter structure is a JMPENV (rather than
  JMPENV*), so this can make the interpreter structure size vary on
  debugging and non-debugging builds.
 
  This is usually a no-no, since it breaks re.pm, which recompiles
  the various re functions with DEBUGGING enabled, even on non-debugging
  fields.
 
  This was supposed to be handled by the extra '!defined DEBUGGING_RE_ONLY'
  condition, and indeed all core XS modules including re.xs work ok.
 
  However, it seems to have broken several CPAN modules. I still don't
  understand why, but the easiest fix is to just always include the
  debugging je_old_stack_hwm field in all builds.

Affected files ...
  M cop.h

Differences ...

Inline Patch

diff --git a/cop.h b/cop.h
index f621cd2..34ebbc0 100644
--- a/cop.h
+++ b/cop.h
@@ -35,9 +35,7 @@ struct jmpenv {
     int			je_ret;		/* last exception thrown */
     bool		je_mustcatch;	/* need to call longjmp()? */
     U16                 je_old_delaymagic; /* saved PL_delaymagic */
-#if defined DEBUGGING && !defined DEBUGGING_RE_ONLY
     SSize_t             je_old_stack_hwm;
-#endif
 };
 
 typedef struct jmpenv JMPENV;


-- 

Art is anything that has a label (especially if the label is "untitled 1")

[p5pRT]

From @jkeenan

On Wed, 23 Aug 2017 09​:30​:10 GMT, davem wrote​:

On Tue, Aug 22, 2017 at 06​:35​:32PM -0700, James E Keenan via RT wrote​:

On Mon, 21 Aug 2017 20​:39​:56 GMT,
andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

Once again, affected are -DDEBUGGING builds only.

bisect
------
commit 9449f0d
Author​: David Mitchell <davem@​iabyn.com>
Date​: Mon Jul 3 13​:43​:27 2017 +0100

save si_stack_hwm across JMPENV_PUSH

other distros also affected
---------------------------
MARKSMITH/HTTP-HeaderParser-XS-0.20.tar.gz
FRACTAL/IO-Vectored-0.110.tar.gz
SDPRICE/Linux-DVB-DVBT-Advert-0.04.tar.gz
SDPRICE/Linux-DVB-DVBT-TS-0.08.tar.gz

Examples of failures​:

#####
[HTTP-HeaderParser-XS-0.20] 547 $ /testing/blead/bin/prove
-I/testing/blead/lib -vb t/10-basic.t
t/10-basic.t ..
1..39
HTTPHeaders.c​: loadable library and perl binaries are mismatched (got
handshake key 0xd100000, needed 0xd180000)

They should hopefully be fixed with the commit below which I've just
pushed. I've only tested IO-Vectored, but I'm hoping/assuming they're
all
the same cause and fix.

commit d2dccc0
Author​: David Mitchell <davem@​iabyn.com>
AuthorDate​: Wed Aug 23 10​:18​:53 2017 +0100
Commit​: David Mitchell <davem@​iabyn.com>
CommitDate​: Wed Aug 23 10​:18​:53 2017 +0100

make 'struct jmpenv' size independent of DEBUGGING

RT #131942

My commit v5.27.1-114-g9449f0d added a field je_old_stack_hwm to
the jmpenv structure on debugging builds.

However, one field in the interpreter structure is a JMPENV (rather
than
JMPENV*), so this can make the interpreter structure size vary on
debugging and non-debugging builds.

This is usually a no-no, since it breaks re.pm, which recompiles
the various re functions with DEBUGGING enabled, even on non-debugging
fields.

This was supposed to be handled by the extra '!defined
DEBUGGING_RE_ONLY'
condition, and indeed all core XS modules including re.xs work ok.

However, it seems to have broken several CPAN modules. I still don't
understand why, but the easiest fix is to just always include the
debugging je_old_stack_hwm field in all builds.

Affected files ...
M cop.h

Differences ...

diff --git a/cop.h b/cop.h
index f621cd2..34ebbc0 100644
--- a/cop.h
+++ b/cop.h
@​@​ -35,9 +35,7 @​@​ struct jmpenv {
int je_ret; /* last exception
thrown */
bool je_mustcatch; /* need to call longjmp()? */
U16 je_old_delaymagic; /* saved PL_delaymagic */
-#if defined DEBUGGING && !defined DEBUGGING_RE_ONLY
SSize_t je_old_stack_hwm;
-#endif
};

typedef struct jmpenv JMPENV;

These all appear to be fixed.

#####
$ ./bin/perl -v | head -2 | tail -1
This is perl 5, version 27, subversion 4 (v5.27.4 (v5.27.3-14-gd2dccc0)) built for x86_64-linux

$ ./bin/perl -Ilib -MHTTP​::HeaderParser​::XS -E 'say q|hello world|;'
hello world

$ ./bin/perl -Ilib -MIO​::Vectored -E 'say q|hello world|;'
hello world

$ ./bin/perl -Ilib -MLinux​::DVB​::DVBT​::Advert -E 'say q|hello world|;'
hello world

$ ./bin/perl -Ilib -MLinux​::DVB​::DVBT​::TS -E 'say q|hello world|;'
hello world

$ ./bin/perl -Ilib -MEV -E 'say q|hello world|;'
hello world
#####

The Glib​::EV case was more complicated because the author has a Makefile.PL which requires responding to prompts. So when I first tried to install this via './bin/cpanm', I got a FAIL at the Configure stage. When I went into the .cpanm build directories, manually ran Makefile.PL and responded to the prompts, I was then able to call make, make test and make install.

Unless other problems appear, I believe this ticket can be closed.

Thank you very much.

--
James E Keenan (jkeenan@​cpan.org)

[p5pRT]

From @kentfredric

On 24 August 2017 at 05​:09, James E Keenan via RT
<perlbug-followup@​perl.org> wrote​:

The Glib​::EV case was more complicated because the author has a Makefile.PL which requires responding to prompts. So when I first tried to install this via './bin/cpanm', I got a FAIL at the Configure stage. When I went into the .cpanm build directories, manually ran Makefile.PL and responded to the prompts, I was then able to call make, make test and make install.

cpanm --help | grep inter
  --interactive Turns on interactive configure (required
for Task​:: modules)

This is *usually* sufficient to get around interactive prompting.

--
Kent

KENTNL - https://metacpan.org/author/KENTNL

[p5pRT]

@iabyn - Status changed from 'open' to 'resolved'