5.19.4 - 5.25.2: -x line offset error #15413

p5pRT · 2016-07-01T05:45:38Z

Migrated from rt.perl.org#128508 (status was 'resolved')

Searchable as RT128508$

p5pRT · 2016-07-01T05:45:38Z

From ted@lyncon.se

Created by ted@lyncon.se

In 5.19.4 it seems an line number offset error when using "-x" sneeked in and it's still present in 5.25.2.

Running the below script with "perl -x" reports the erroneous line to be line 7, but it is line 6.

-- cut --
#!/usr/bin/perl

use strict;
use warnings;

my this_is_line_six
-- cut --

In 5.19.3 and earlier the line number is reported correctly.

Best regards,
Ted Lyngmo

Perl Info


Flags:
    category=core
    severity=low

Site configuration information for perl 5.19.4:

Configured by qtedlyn at Thu Jun 30 12:01:53 CEST 2016.

Summary of my perl5 (revision 5 version 19 subversion 4) configuration:
   
  Platform:
    osname=linux, osvers=2.6.32-504.30.3.el6.x86_64, archname=x86_64-linux
    uname='linux eselnlx1796 2.6.32-504.30.3.el6.x86_64 #1 smp thu jul 9 15:20:47 edt 2015 x86_64 x86_64 x86_64 gnulinux '
    config_args='-de -Dprefix=/home/qtedlyn/perl5/perlbrew/perls/perl-5.19.4 -Dusedevel -Aeval:scriptdir=/home/qtedlyn/perl5/perlbrew/perls/perl-5.19.4/bin'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2',
    cppflags='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.4.7 20120313 (Red Hat 4.4.7-11)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib/../lib64 /usr/lib/../lib64 /lib /usr/lib /lib64 /usr/lib64 /usr/local/lib64
    libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.12.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.12'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector'

Locally applied patches:
    Devel::PatchPerl 1.38


@INC for perl 5.19.4:
    /home/qtedlyn/perl5/perlbrew/perls/perl-5.19.4/lib/site_perl/5.19.4/x86_64-linux
    /home/qtedlyn/perl5/perlbrew/perls/perl-5.19.4/lib/site_perl/5.19.4
    /home/qtedlyn/perl5/perlbrew/perls/perl-5.19.4/lib/5.19.4/x86_64-linux
    /home/qtedlyn/perl5/perlbrew/perls/perl-5.19.4/lib/5.19.4
    .


Environment for perl 5.19.4:
    HOME=/home/qtedlyn
    LANG=en_US.utf8
    LANGUAGE (unset)
    LC_COLLATE=C
    LC_MESSAGES=C
    LC_TIME=C
    LD_LIBRARY_PATH=/app/expect/5.43/LMWP3/lib:/proj/sgsn-tools/wh/tools/RHE64-6.6/gcc/4.6.2/lib64
    LOGDIR (unset)
    PATH=/home/qtedlyn/perl5/perlbrew/bin:/home/qtedlyn/perl5/perlbrew/perls/perl-5.19.4/bin:/app/frame/5.5.6/bin:/app/expect/5.43/LMWP3/bin:/vobs/gsn_rel_otp/built/c_linux_gpb_lws/bin:/vobs/gsn_rel_otp/sles10_32/bin:/vobs/otp/otp_delivery/suse_x86/bin:/vobs/otp/otp_delivery/sles10_32/bin:/app/tig/1.1/LMWP3/bin:/app/python/2.7.1/RHEL64/bin:/app/git_tools/0.8/bin:/app/git/1.8.4.1/RHEL6/bin:/app/git/1.8.4.1/RHEL6/libexec/git-core:/app/kdiff3/0.9.96/RHEL64:/usr/lsf/bin:/usr/lsf/etc:/opt/quest/bin:/home/qtedlyn/.afs/3/rbin:/home/qtedlyn/.afs/3/pbin:/env/seln/bin:/home/qtedlyn/.afs/3/ibin:/usr/atria/bin:/usr/afsws/bin:/usr/lib64/qt-3.3/bin:/proj/sgsn-tools/wh/tools/RHE64-6.6/kdiff3/0.9.96/bin:/proj/sgsn-tools/wh/tools/RHE64-6.6/git/1.8.4/bin:/proj/sgsn-tools/wh/tools/RHE64-6.6/python/2.7.1/bin:/opt/tools/tools/vulcanbase/0.74/bin:/bin:/usr/bin:/usr/sbin:/sbin:/app/arc/0/bin:/proj/gsn_tools/bin:/proj/sgsn-tools/gsnws/LATEST:/proj/sgsn-tools/autogsn/LATEST:/proj/eXmachina/publish/bi
 n:/vobs/gsn_ccstart/bin:/vobs/gsn_autogsn/bin:/vobs/gsn/product/test/gtt/core/bin:/vobs/gsn_tecsas/bin:/vobs/gsn/tools/bin:/vobs/gsn_im/tools/dxet
    PERLBREW_CSHRC_VERSION=0.75
    PERLBREW_HOME=/home/qtedlyn/.perlbrew
    PERLBREW_MANPATH=/home/qtedlyn/perl5/perlbrew/perls/perl-5.19.4/man
    PERLBREW_PATH=/home/qtedlyn/perl5/perlbrew/bin:/home/qtedlyn/perl5/perlbrew/perls/perl-5.19.4/bin
    PERLBREW_PERL=perl-5.19.4
    PERLBREW_ROOT=/home/qtedlyn/perl5/perlbrew
    PERLBREW_VERSION=0.75
    PERL_BADLANG (unset)
    SHELL=/bin/tcsh

p5pRT · 2016-07-02T00:36:50Z

From @dcollinsn

Bisects to:

bf1b738 is the first bad commit
commit bf1b738
Author: Father Chrysostomos <sprout@cpan.org>
Date: Thu Aug 22 10:01:58 2013 -0700

[perl #118931] Fix line number bug

Commit 2179133 (in 5.19.2) introduced this line number bug when it
modified the parser to look past newlines when searching for => after
a keyword:

$ perl5.19.3
1 unless
1;
warn;
^D
Warning: something's wrong at - line 2.

The warning should say line 3, not 2.

Before 2179133, the parser’s line-reading algorithm work strictly in
two modes: From a file, one line would be read into PL_linestr at a
time. In a string eval, PL_linestr would contain all the code.

To be able to look past a newline to find => after a keyword, the
parser’s former mode had to be adjusted:

• It has to be possible to append more lines of input to the buffer
without incrementing the line number.
• When reading from a filehandle, the lexer should not assume when it
sees "\n" that it has reached the end of the buffer.

Commit 2179133 did those two things.

It did not, however, make the lexer increment the line number when
encountering a newline in the middle of it (when reading from a han-
dle; string eval follows a different code path).

Fixing it to do that requires that lex_start be adjusted, too. When
lexing begins, PL_linestr is set to "\n;" when there is no string to
eval. This worked for file handles because the lexer, on seeing the
\n, would ignore the semicolon.

Now that it no longer ignores the semicolon, it will end up incre-
mented the line number erroneously at the outset, so set the buffer
to just "\n" instead.

In one place (skipspace2), the mad-specific code was setting PL_bufptr
to point at its previous location after calling skipspace. skipspace
sets it to point after the newline to prevent incline() from being
called a second time for the same newline. But this is exactly what
happens if skipspace2 resets PL_bufptr. The callers of skipspace2
apparently don’t depend on this, so we can remove it.

:040000 040000 f48a7e36a96074f78be759dfeeffa6f1f26f039b 94d0f19069694ed4b68f332b6b2d707cfa6b3bce M t
:100644 100644 4f3eee9f20a66b836ca95e4e090f1ef78fc1d022 362aa71d5e47b16bd67a552299f7ac8653ad69f0 M toke.c
bisect run success

Why this only happens under -x, however, is beyond me.

p5pRT · 2016-07-02T00:36:50Z

The RT System itself - Status changed from 'new' to 'open'

p5pRT · 2016-07-02T07:11:50Z

From @cpansprout

On Fri Jul 01 17:36:50 2016, dcollinsn@gmail.com wrote:

Why this only happens under -x, however, is beyond me.

It’s...complicated. I wrote a barely comprehensible explanation in the commit message for b3dd0ab, which fixes the bug.

I think the patch should be a candidate for backporting to 5.22 and 5.24 (and 5.20 if we are still doing that).

--

Father Chrysostomos

p5pRT · 2016-07-02T07:11:50Z

@cpansprout - Status changed from 'open' to 'pending release'

p5pRT · 2016-07-03T20:57:36Z

From @xsawyerx

On Sat Jul 02 00:11:50 2016, sprout wrote:

On Fri Jul 01 17:36:50 2016, dcollinsn@gmail.com wrote:

Why this only happens under -x, however, is beyond me.

It’s...complicated. I wrote a barely comprehensible explanation in
the commit message for b3dd0ab, which fixes the bug.

I think the patch should be a candidate for backporting to 5.22 and
5.24 (and 5.20 if we are still doing that).

5.20 is not supported. We only support two versions from recent stable.

If we make another release of 5.20 (only in case of important security issues), we can try and include this. I'll make a note of it. Thank you.

p5pRT · 2017-05-30T20:14:10Z

From @khwilliamson

Thank you for filing this report. You have helped make Perl better.

With the release today of Perl 5.26.0, this and 210 other issues have been
resolved.

Perl 5.26.0 may be downloaded via:
https://metacpan.org/release/XSAWYERX/perl-5.26.0

If you find that the problem persists, feel free to reopen this ticket.

p5pRT · 2017-05-30T20:14:11Z

@khwilliamson - Status changed from 'pending release' to 'resolved'

p5pRT closed this as completed May 30, 2017

p5pRT added Severity Low distro-Linux type-core labels Oct 19, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

5.19.4 - 5.25.2: -x line offset error #15413

5.19.4 - 5.25.2: -x line offset error #15413

p5pRT commented Jul 1, 2016

p5pRT commented Jul 1, 2016

p5pRT commented Jul 2, 2016

p5pRT commented Jul 2, 2016

p5pRT commented Jul 2, 2016

p5pRT commented Jul 2, 2016

p5pRT commented Jul 3, 2016

p5pRT commented May 30, 2017

p5pRT commented May 30, 2017

5.19.4 - 5.25.2: -x line offset error #15413

5.19.4 - 5.25.2: -x line offset error #15413

Comments

p5pRT commented Jul 1, 2016

p5pRT commented Jul 1, 2016

From ted@lyncon.se

Created by ted@lyncon.se

p5pRT commented Jul 2, 2016

From @dcollinsn

p5pRT commented Jul 2, 2016

p5pRT commented Jul 2, 2016

From @cpansprout

p5pRT commented Jul 2, 2016

p5pRT commented Jul 3, 2016

From @xsawyerx

p5pRT commented May 30, 2017

From @khwilliamson

p5pRT commented May 30, 2017