Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bleadperl v5.21.4-88-ga51d618 breaks DCONWAY/Regexp-Grammars-1.036.tar.gz #14129

Closed
p5pRT opened this issue Oct 3, 2014 · 11 comments

Comments

@p5pRT
Copy link

commented Oct 3, 2014

Migrated from rt.perl.org#122890 (status was 'resolved')

Searchable as RT122890$

@p5pRT

This comment has been minimized.

Copy link
Author

commented Oct 3, 2014

From @andk

git bisect


commit a51d618
Author​: Yves Orton <demerphq@​gmail.com>
Date​: Fri Sep 19 19​:57​:34 2014 +0200

  rt 122283 - do not recurse into GOSUB/GOSTART when not SCF_DO_SUBSTR

sample fail report


http​://www.cpantesters.org/cpan/report/98605662-4578-11e4-966f-9b262c6f0924

perl -V


Summary of my perl5 (revision 5 version 21 subversion 5) configuration​:
  Commit id​: a83034f
  Platform​:
  osname=linux, osvers=3.16-1-amd64, archname=x86_64-linux-thread-multi-ld
  uname='linux k83 3.16-1-amd64 #1 smp debian 3.16.2-2 (2014-09-08) x86_64 gnulinux '
  config_args='-Dprefix=/home/sand/src/perl/repoperls/installed-perls/perl/v5.21.4-101-ga83034f/a2da -Dmyhostname=k83 -Dinstallusrbinperl=n -Uversiononly -Dusedevel -des -Ui_db -Duseithreads -Duselongdouble -DDEBUGGING=-g'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  use64bitint=define, use64bitall=define, uselongdouble=define
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2',
  optimize='-O2 -g',
  cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion='', gccversion='4.9.1', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
  ivtype='long', ivsize=8, nvtype='long double', nvsize=16, Off_t='off_t', lseeksize=8
  alignbytes=16, prototype=define
  Linker and Libraries​:
  ld='cc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc -lgdbm_compat
  perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
  libc=libc-2.19.so, so=so, useshrplib=false, libperl=libperl.a
  gnulibc_version='2.19'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
  cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​: HAS_TIMES MULTIPLICITY PERLIO_LAYERS
  PERL_DONT_CREATE_GVSV
  PERL_HASH_FUNC_ONE_AT_A_TIME_HARD
  PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
  PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
  PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT
  USE_ITHREADS USE_LARGE_FILES USE_LOCALE
  USE_LOCALE_COLLATE USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_LONG_DOUBLE
  USE_PERLIO USE_PERL_ATOF USE_REENTRANT_API
  Built under linux
  Compiled at Sep 25 2014 22​:31​:36
  %ENV​:
  PERL5LIB=""
  PERL5OPT=""
  PERL5_CPANPLUS_IS_RUNNING="21904"
  PERL5_CPAN_IS_RUNNING="21904"
  PERL_MM_USE_DEFAULT="1"
  @​INC​:
  /home/sand/src/perl/repoperls/installed-perls/perl/v5.21.4-101-ga83034f/a2da/lib/site_perl/5.21.5/x86_64-linux-thread-multi-ld
  /home/sand/src/perl/repoperls/installed-perls/perl/v5.21.4-101-ga83034f/a2da/lib/site_perl/5.21.5
  /home/sand/src/perl/repoperls/installed-perls/perl/v5.21.4-101-ga83034f/a2da/lib/5.21.5/x86_64-linux-thread-multi-ld
  /home/sand/src/perl/repoperls/installed-perls/perl/v5.21.4-101-ga83034f/a2da/lib/5.21.5
  .
--
andreas

@p5pRT

This comment has been minimized.

Copy link
Author

commented Oct 4, 2014

From @cpansprout

On Thu Oct 02 23​:19​:01 2014, andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

git bisect
----------
commit a51d618
Author​: Yves Orton <demerphq@​gmail.com>
Date​: Fri Sep 19 19​:57​:34 2014 +0200

rt 122283 - do not recurse into GOSUB/GOSTART when not SCF_DO_SUBSTR

sample fail report
------------------
http​://www.cpantesters.org/cpan/report/98605662-4578-11e4-966f-
9b262c6f0924

Reduced case​:

print scalar '7% solution' =~
  m/(?​: (?&solution) | % ) \Z
  (?(DEFINE) (?<solution>7\%\ solution) )
  /msx, "\n";

Output in 5.21.4​:
1

Output in blead is an empty line.

--

Father Chrysostomos

@p5pRT

This comment has been minimized.

Copy link
Author

commented Oct 4, 2014

The RT System itself - Status changed from 'new' to 'open'

@p5pRT

This comment has been minimized.

Copy link
Author

commented Oct 5, 2014

From @cpansprout

On Fri Oct 03 22​:03​:48 2014, sprout wrote​:

On Thu Oct 02 23​:19​:01 2014, andreas.koenig.7os6VVqR@​franz.ak.mind.de wrote​:

git bisect
----------
commit a51d618
Author​: Yves Orton <demerphq@​gmail.com>
Date​: Fri Sep 19 19​:57​:34 2014 +0200

rt 122283 - do not recurse into GOSUB/GOSTART when not SCF_DO_SUBSTR

sample fail report
------------------
http​://www.cpantesters.org/cpan/report/98605662-4578-11e4-966f-
9b262c6f0924

Reduced case​:

print scalar '7% solution' =~
m/(?​: (?&solution) | % ) \Z
(?(DEFINE) (?<solution>7\%\ solution) )
/msx, "\n";

Output in 5.21.4​:
1

Output in blead is an empty line.

More info​: The anchored-end optimisation determines that the pattern cannot start earlier than position 10 (1 character before the end of the string. Apparently it sees (?&solution) as having zero length, instead of potentially infinite length. If study_chunk (or whatever is responsible) is not going to follow (?&...) recursively, it at least needs to note that max-length optimisations need to be skipped. How does one go about that?

Compiling REx "(?​: (?&solution) | %% ) \Z%n (?(DEFINE) (?<solution>7\%%\"...
rarest char
at 0
Final program​:
  1​: BRANCH (5)
  2​: GOSUB1[+12] (9)
  5​: BRANCH (FAIL)
  6​: EXACT <%> (9)
  8​: TAIL (9)
  9​: SEOL (10)
  10​: DEFINEP (12)
  12​: IFTHEN (25)
  14​: OPEN1 'solution' (16)
  16​: EXACT <7% solution> (20)
  20​: CLOSE1 'solution' (25)
  22​: LONGJMP (24)
  24​: TAIL (25)
  25​: END (0)
floating ""$ at 0..1 (checking floating) minlen 0
Enabling $` $& $' support (0x7).

EXECUTING...

Matching REx "(?​: (?&solution) | %% ) \Z%n (?(DEFINE) (?<solution>7\%%\"... against "7%% solution"
Intuit​: trying to determine minimum start position...
  Found floating substr ""$ at offset 11...
  (multiline anchor test skipped)
  try at offset...
Intuit​: Successfully guessed​: match at offset 10
  10 <7%% solutio> <n> | 1​:BRANCH(5)

etc.

--

Father Chrysostomos

@p5pRT

This comment has been minimized.

Copy link
Author

commented Oct 5, 2014

From @demerphq

On 5 October 2014 03​:58, Father Chrysostomos via RT <
perlbug-followup@​perl.org> wrote​:

On Fri Oct 03 22​:03​:48 2014, sprout wrote​:

On Thu Oct 02 23​:19​:01 2014, andreas.koenig.7os6VVqR@​franz.ak.mind.de
wrote​:

git bisect
----------
commit a51d618
Author​: Yves Orton <demerphq@​gmail.com>
Date​: Fri Sep 19 19​:57​:34 2014 +0200

rt 122283 - do not recurse into GOSUB/GOSTART when not SCF_DO_SUBSTR

sample fail report
------------------
http​://www.cpantesters.org/cpan/report/98605662-4578-11e4-966f-
9b262c6f0924

Reduced case​:

print scalar '7% solution' =~
m/(?​: (?&solution) | % ) \Z
(?(DEFINE) (?<solution>7\%\ solution) )
/msx, "\n";

Output in 5.21.4​:
1

Output in blead is an empty line.

More info​: The anchored-end optimisation determines that the pattern
cannot start earlier than position 10 (1 character before the end of the
string. Apparently it sees (?&solution) as having zero length, instead of
potentially infinite length. If study_chunk (or whatever is responsible)
is not going to follow (?&...) recursively, it at least needs to note that
max-length optimisations need to be skipped. How does one go about that?

I don't think it is that simple. There are a bunch of subtle behaviors
interacting here in a suboptimal way.

I will try to find time for this this week. Your reduction to simpler case
makes my life a lot easier, thanks.

For now please leave this to me.

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT

This comment has been minimized.

Copy link
Author

commented Oct 13, 2014

From @demerphq

On 5 October 2014 11​:47, demerphq <demerphq@​gmail.com> wrote​:

On 5 October 2014 03​:58, Father Chrysostomos via RT <
perlbug-followup@​perl.org> wrote​:

On Fri Oct 03 22​:03​:48 2014, sprout wrote​:

On Thu Oct 02 23​:19​:01 2014, andreas.koenig.7os6VVqR@​franz.ak.mind.de
wrote​:

git bisect
----------
commit a51d618
Author​: Yves Orton <demerphq@​gmail.com>
Date​: Fri Sep 19 19​:57​:34 2014 +0200

rt 122283 - do not recurse into GOSUB/GOSTART when not SCF_DO_SUBSTR

sample fail report
------------------
http​://www.cpantesters.org/cpan/report/98605662-4578-11e4-966f-
9b262c6f0924

Reduced case​:

print scalar '7% solution' =~
m/(?​: (?&solution) | % ) \Z
(?(DEFINE) (?<solution>7\%\ solution) )
/msx, "\n";

Output in 5.21.4​:
1

Output in blead is an empty line.

More info​: The anchored-end optimisation determines that the pattern
cannot start earlier than position 10 (1 character before the end of the
string. Apparently it sees (?&solution) as having zero length, instead of
potentially infinite length. If study_chunk (or whatever is responsible)
is not going to follow (?&...) recursively, it at least needs to note that
max-length optimisations need to be skipped. How does one go about that?

I don't think it is that simple. There are a bunch of subtle behaviors
interacting here in a suboptimal way.

I will try to find time for this this week. Your reduction to simpler case
makes my life a lot easier, thanks.

For now please leave this to me.

For the record I put some hours into this over the weekend. No final result
yet, but I have fixed the memory leaks. The rt 122283 pattern now runs to
finish in 372 minutes. :-)

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT

This comment has been minimized.

Copy link
Author

commented Oct 19, 2014

From @andk

also affected​:
SPROUT/JE-0.065.tar.gz
http​://www.cpantesters.org/cpan/report/785cae16-54f9-11e4-b101-b4d49ed67017
--
andreas

@p5pRT

This comment has been minimized.

Copy link
Author

commented Oct 20, 2014

From @demerphq

On 19 October 2014 09​:51, Andreas Koenig <
andreas.koenig.7os6VVqR@​franz.ak.mind.de> wrote​:

also affected​:
SPROUT/JE-0.065.tar.gz
http​://www.cpantesters.org/cpan/report/785cae16-54f9-11e4-b101-b4d49ed67017

Last night I pushed a patch sequence which hopefully fixes this and RT #
122283.

If that fixes these modules as well then I consider this issue finally
properly closed.

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT

This comment has been minimized.

Copy link
Author

commented Oct 20, 2014

From @andk

On Mon, 20 Oct 2014 08​:45​:15 +0200, demerphq <demerphq@​gmail.com> said​:

  > On 19 October 2014 09​:51, Andreas Koenig <
  > andreas.koenig.7os6VVqR@​franz.ak.mind.de> wrote​:

  > also affected​:
  > SPROUT/JE-0.065.tar.gz
  > http​://www.cpantesters.org/cpan/report/785cae16-54f9-11e4-b101-b4d49ed67017

  > Last night I pushed a patch sequence which hopefully fixes this and RT #
  > 122283.

  > If that fixes these modules as well then I consider this issue finally properly
  > closed.

With v5.21.4-553-gf6f22b1​:

  DCONWAY/Regexp-Grammars-1.036.tar.gz
  ./Build test -- OK

  SPROUT/JE-0.065.tar.gz
  /usr/bin/make test -- OK

Thank You && Cheers!
--
andreas

@p5pRT

This comment has been minimized.

Copy link
Author

commented Oct 20, 2014

From @demerphq

On 20 October 2014 20​:16, Andreas Koenig <
andreas.koenig.7os6VVqR@​franz.ak.mind.de> wrote​:

On Mon, 20 Oct 2014 08​:45​:15 +0200, demerphq <demerphq@​gmail.com>
said​:

On 19 October 2014 09​:51, Andreas Koenig <
andreas.koenig.7os6VVqR@​franz.ak.mind.de> wrote​:

also affected&#8203;:
SPROUT/JE\-0\.065\.tar\.gz

http​://www.cpantesters.org/cpan/report/785cae16-54f9-11e4-b101-b4d49ed67017

Last night I pushed a patch sequence which hopefully fixes this and RT
#
122283.

If that fixes these modules as well then I consider this issue finally
properly
closed.

With v5.21.4-553-gf6f22b1​:

DCONWAY/Regexp-Grammars-1.036.tar.gz
./Build test -- OK

SPROUT/JE-0.065.tar.gz
/usr/bin/make test -- OK

Thank You && Cheers!

And thank you for your CPAN reports. :-)

cheers,
Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT

This comment has been minimized.

Copy link
Author

commented Oct 20, 2014

@cpansprout - Status changed from 'open' to 'resolved'

@p5pRT p5pRT closed this Oct 20, 2014
@p5pRT p5pRT added the Severity Low label Oct 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.