Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blead Breaks CPAN: HMBRAND/DBD-CSV-0.53.tgz #16582

Closed
p5pRT opened this issue Jun 9, 2018 · 16 comments
Closed

Blead Breaks CPAN: HMBRAND/DBD-CSV-0.53.tgz #16582

p5pRT opened this issue Jun 9, 2018 · 16 comments

Comments

@p5pRT
Copy link

@p5pRT p5pRT commented Jun 9, 2018

Migrated from rt.perl.org#133270 (status was 'rejected')

Searchable as RT133270$

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 9, 2018

From @eserte

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.28.0.


t/82_free_unref_scalar.t from DBD-CSV-0.53 fails on some newer
perl installations. The fail log looks like this​:

...
# Failed test 'there was an attempt to free unreferenced scalar'
# at /opt/perl-5.28.0-RC2t/lib/site_perl/5.28.0/Test/Builder.pm line 158.
# Attempt to free unreferenced scalar​: SV 0x3f6a0f8, Perl interpreter​: 0x2353010 during global destruction.
...
t/82_free_unref_scalar.t ..
All 405 subtests passed

The earliest perl with this failure is 5.27.9, according to CPAN Testers​:
http​://matrix.cpantesters.org/?dist=DBD-CSV%200.53;reports=1#sl=0,0

Also, there's a single failure with perl 5.26.2.



Flags​:
  category=core
  severity=low


Site configuration information for perl 5.28.0​:

Configured by eserte at Wed Jun 6 20​:01​:53 CEST 2018.

Summary of my perl5 (revision 5 version 28 subversion 0) configuration​:
 
  Platform​:
  osname=linux
  osvers=3.16.0-4-amd64
  archname=x86_64-linux
  uname='linux cabulja 3.16.0-4-amd64 #1 smp debian 3.16.51-3 (2017-12-13) x86_64 gnulinux '
  config_args='-ds -e -Dprefix=/opt/perl-5.28.0-RC2 -Dcf_email=srezic@​cpan.org'
  hint=recommended
  useposix=true
  d_sigaction=define
  useithreads=undef
  usemultiplicity=undef
  use64bitint=define
  use64bitall=define
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='cc'
  ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
  optimize='-O2'
  cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
  ccversion=''
  gccversion='4.9.2'
  gccosandvers=''
  intsize=4
  longsize=8
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=define
  longlongsize=8
  d_longdbl=define
  longdblsize=16
  longdblkind=3
  ivtype='long'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='off_t'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='cc'
  ldflags =' -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
  libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
  perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
  libc=libc-2.19.so
  so=so
  useshrplib=false
  libperl=libperl.a
  gnulibc_version='2.19'
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs
  dlext=so
  d_dlsymun=undef
  ccdlflags='-Wl,-E'
  cccdlflags='-fPIC'
  lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'

Locally applied patches​:
  RC2


@​INC for perl 5.28.0​:
  /opt/perl-5.28.0-RC2/lib/site_perl/5.28.0/x86_64-linux
  /opt/perl-5.28.0-RC2/lib/site_perl/5.28.0
  /opt/perl-5.28.0-RC2/lib/5.28.0/x86_64-linux
  /opt/perl-5.28.0-RC2/lib/5.28.0


Environment for perl 5.28.0​:
  HOME=/home/eserte
  LANG=en_US.UTF-8
  LANGUAGE (unset)
  LD_LIBRARY_PATH (unset)
  LOGDIR (unset)
  PATH=/usr/local/bin​:/usr/bin​:/bin​:/usr/local/sbin​:/usr/sbin​:/sbin​:/home/eserte/bin/linux-gnu​:/home/eserte/bin/sh​:/home/eserte/bin​:/home/eserte/bin/pistachio-perl/bin​:/usr/games​:/home/eserte/devel
  PERLDOC=-MPod​::Perldoc​::ToTextOverstrike
  PERL_BADLANG (unset)
  SHELL=/bin/zsh

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 9, 2018

From @jkeenan

On Sat, 09 Jun 2018 10​:43​:33 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.28.0.

-----------------------------------------------------------------
t/82_free_unref_scalar.t from DBD-CSV-0.53 fails on some newer
perl installations. The fail log looks like this​:

...
# Failed test 'there was an attempt to free unreferenced scalar'
# at /opt/perl-5.28.0-RC2t/lib/site_perl/5.28.0/Test/Builder.pm line
158.
# Attempt to free unreferenced scalar​: SV 0x3f6a0f8, Perl interpreter​:
0x2353010 during global destruction.
...
t/82_free_unref_scalar.t ..
All 405 subtests passed

The earliest perl with this failure is 5.27.9, according to CPAN
Testers​:
http​://matrix.cpantesters.org/?dist=DBD-CSV%200.53;reports=1#sl=0,0

Also, there's a single failure with perl 5.26.2.

I attempted to bisect this, starting at v5.27.8 and ending at v5.27.9. The results were ambiguous; see attachment. I'm puzzled as to why the failure message points to Test​::Builder rather than to DBC​::CSV.

Thank you very much.
Jim Keenan

--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 9, 2018

From @jkeenan

$ perl Porting/bisect.pl --module=DBD​::CSV --start=v5.27.8 --end=v5.27.9
,,,
Running make for H/HM/HMBRAND/DBD-CSV-0.53.tgz
cp lib/Bundle/DBD/CSV.pm blib/lib/Bundle/DBD/CSV.pm
cp lib/DBD/CSV.pm blib/lib/DBD/CSV.pm
cp lib/DBD/CSV/GetInfo.pm blib/lib/DBD/CSV/GetInfo.pm
cp lib/DBD/CSV/TypeInfo.pm blib/lib/DBD/CSV/TypeInfo.pm
Manifying 2 pod documents
  HMBRAND/DBD-CSV-0.53.tgz
  /usr/bin/make -- OK
Running make test
PERL_DL_NONLAZY=1 "/tmp/svUIMFIRKX/bin/perl" "-MExtUtils​::Command​::MM" "-MTest​::Harness" "-e" "undef *Test​::Harness​::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/10_base.t ............... 1/? # Showing relevant versions (DBI_SQL_NANO = not set)
# Using DBI version 1.641
# Using DBD​::File version 0.44
# Using SQL​::Statement version 1.412
# Using Text​::CSV_XS version 1.35
# DBD​::CSV 0.53 using Text​::CSV_XS (1.35)
# DBD​::File 0.44 using IO​::File (1.16)
# DBI​::DBD​::SqlEngine 0.06 using SQL​::Statement 1.412
# DBI 1.641
# OS linux (4.4.0-127-generic)
# Perl 5.027008 (x86_64-linux)
t/10_base.t ............... ok
t/11_dsnlist.t ............ ok
t/20_createdrop.t ......... ok
t/30_insertfetch.t ........ ok
t/31_delete.t ............. ok
t/32_update.t ............. ok
t/40_numrows.t ............ ok
t/41_nulls.t .............. ok
t/42_bindparam.t .......... ok
t/43_blobs.t .............. ok
t/44_listfields.t ......... ok
t/48_utf8.t ............... ok
t/50_chopblanks.t ......... ok
t/51_commit.t ............. ok
t/55_dir_search.t ......... skipped​: No folder scanning during automated tests
t/60_misc.t ............... ok
t/61_meta.t ............... ok
t/70_csv.t ................ ok
t/71_csv-ext.t ............ ok
t/72_csv-schema.t ......... ok
t/73_csv-case.t ........... ok
t/80_rt.t ................. ok
t/82_free_unref_scalar.t .. 254/?
# Failed test 'there was an attempt to free unreferenced scalar'
# at /tmp/svUIMFIRKX/lib/perl5/5.27.8/Test/Builder.pm line 158.
# Attempt to free unreferenced scalar​: SV 0x25fc6a0 during global destruction.

# Failed test 'there was an attempt to free unreferenced scalar'
# at /tmp/svUIMFIRKX/lib/perl5/5.27.8/Test/Builder.pm line 158.
# Attempt to free unreferenced scalar​: SV 0x2580238 during global destruction.

# Failed test 'there was an attempt to free unreferenced scalar'
# at /tmp/svUIMFIRKX/lib/perl5/5.27.8/Test/Builder.pm line 158.
# Attempt to free unreferenced scalar​: SV 0x24c4248 during global destruction.

# Failed test 'there was an attempt to free unreferenced scalar'
# at /tmp/svUIMFIRKX/lib/perl5/5.27.8/Test/Builder.pm line 158.
# Attempt to free unreferenced scalar​: SV 0x23e2a88 during global destruction.

# Failed test 'there was an attempt to free unreferenced scalar'
# at /tmp/svUIMFIRKX/lib/perl5/5.27.8/Test/Builder.pm line 158.
# Attempt to free unreferenced scalar​: SV 0x23c96d8 during global destruction.

# Failed test 'there was an attempt to free unreferenced scalar'
# at /tmp/svUIMFIRKX/lib/perl5/5.27.8/Test/Builder.pm line 158.
# Attempt to free unreferenced scalar​: SV 0x2287100 during global destruction.

# Failed test 'there was an attempt to free unreferenced scalar'
# at /tmp/svUIMFIRKX/lib/perl5/5.27.8/Test/Builder.pm line 158.
# Attempt to free unreferenced scalar​: SV 0x223b4d0 during global destruction.
t/82_free_unref_scalar.t .. All 405 subtests passed
t/85_error.t .............. ok

Test Summary Report


t/82_free_unref_scalar.t (Wstat​: 0 Tests​: 412 Failed​: 7)
  Failed tests​: 406-412
  Parse errors​: Plan (1..405) must be at the beginning or end of the TAP output
  Bad plan. You planned 405 tests but ran 412.
Files=24, Tests=1161, 7 wallclock secs ( 0.27 usr 0.02 sys + 6.81 cusr 0.30 csys = 7.40 CPU)
Result​: FAIL
Failed 1/24 test programs. 7/1161 subtests failed.
Makefile​:865​: recipe for target 'test_dynamic' failed
make​: *** [test_dynamic] Error 255
  HMBRAND/DBD-CSV-0.53.tgz
  /usr/bin/make test -- NOT OK
//hint// to see the cpan-testers results for installing this module, try​:
  reports HMBRAND/DBD-CSV-0.53.tgz
Died at -e line 2.
HEAD is now at 7a783ba $ perl Porting/bisect.pl --module=DBD​::CSV --start=v5.27.8 --end=v5.27.9
Perlhist entry for 5.27.8
bad - non-zero exit from /tmp/svUIMFIRKX/bin/perl -I /home/jkeenan/.cpan -MCPAN​::MyConfig -MCPAN -e $CPAN​::Config->{build_dir}=q{/tmp/UNtfIGKqX6}; -e install('DBD​::CSV'); die unless CPAN​::Shell->expand(Module => 'DBD​::CSV')->uptodate;
Runner returned 256, not 0 for start revision at Porting/bisect.pl line 240.
That took 680 seconds.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 9, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 9, 2018

From @dur-randir

On Sat, 09 Jun 2018 08​:05​:27 -0700, jkeenan wrote​:

The results were ambiguous; see attachment. I'm puzzled as to why the
failure message points to Test​::Builder rather than to DBC​::CSV.

I've tried to bisect this too, but there's some floating/timing issue. The furthest failure I've got is at revision 4bd1355, which is older then yours.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 9, 2018

From @jkeenan

On Sat, 09 Jun 2018 15​:05​:27 GMT, jkeenan wrote​:

On Sat, 09 Jun 2018 10​:43​:33 GMT, slaven@​rezic.de wrote​:

This is a bug report for perl from slaven@​rezic.de,
generated with the help of perlbug 1.41 running under perl 5.28.0.

-----------------------------------------------------------------
t/82_free_unref_scalar.t from DBD-CSV-0.53 fails on some newer
perl installations. The fail log looks like this​:

...
# Failed test 'there was an attempt to free unreferenced scalar'
# at /opt/perl-5.28.0-RC2t/lib/site_perl/5.28.0/Test/Builder.pm
line
158.
# Attempt to free unreferenced scalar​: SV 0x3f6a0f8, Perl
interpreter​:
0x2353010 during global destruction.
...
t/82_free_unref_scalar.t ..
All 405 subtests passed

The earliest perl with this failure is 5.27.9, according to CPAN
Testers​:
http​://matrix.cpantesters.org/?dist=DBD-CSV%200.53;reports=1#sl=0,0

Also, there's a single failure with perl 5.26.2.

I attempted to bisect this, starting at v5.27.8 and ending at v5.27.9.
The results were ambiguous; see attachment. I'm puzzled as to why the
failure message points to Test​::Builder rather than to DBC​::CSV.

Thank you very much.
Jim Keenan

And to further muddy the waters, I have been able to install DBD​::CSV against perl 5 blead using cpanm​:

#####
$ ./bin/perl -Ilib -v | head -2 | tail -1
This is perl 5, version 28, subversion 0 (v5.28.0-RC2-2-g197e798) built for x86_64-linux

$ ./bin/cpanm DBD​::CSV
DBD​::CSV is up to date. (0.53)
#####
--
James E Keenan (jkeenan@​cpan.org)

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 9, 2018

From @iabyn

On Sat, Jun 09, 2018 at 08​:55​:13AM -0700, Sergey Aleynikov via RT wrote​:

On Sat, 09 Jun 2018 08​:05​:27 -0700, jkeenan wrote​:

The results were ambiguous; see attachment. I'm puzzled as to why the
failure message points to Test​::Builder rather than to DBC​::CSV.

I've tried to bisect this too, but there's some floating/timing issue.
The furthest failure I've got is at revision 4bd1355, which is older
then yours.

The scalar which is being double-freed (during global destruction)
is always the tie object returned by

  DBI​::DBD​::SqlEngine​::TieTables​::TIEHASH()

at DBI/DBD/SqlEngine.pm​:1125.

But the triggering of the double-free seems random, and can't even
be reproduced consistently by setting PERL_HASH_SEED to a known value.

I'm still investigating further.

--
If life gives you lemons, you'll probably develop a citric acid allergy.

1 similar comment
@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 9, 2018

From @iabyn

On Sat, Jun 09, 2018 at 08​:55​:13AM -0700, Sergey Aleynikov via RT wrote​:

On Sat, 09 Jun 2018 08​:05​:27 -0700, jkeenan wrote​:

The results were ambiguous; see attachment. I'm puzzled as to why the
failure message points to Test​::Builder rather than to DBC​::CSV.

I've tried to bisect this too, but there's some floating/timing issue.
The furthest failure I've got is at revision 4bd1355, which is older
then yours.

The scalar which is being double-freed (during global destruction)
is always the tie object returned by

  DBI​::DBD​::SqlEngine​::TieTables​::TIEHASH()

at DBI/DBD/SqlEngine.pm​:1125.

But the triggering of the double-free seems random, and can't even
be reproduced consistently by setting PERL_HASH_SEED to a known value.

I'm still investigating further.

--
If life gives you lemons, you'll probably develop a citric acid allergy.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 9, 2018

From @iabyn

On Sat, Jun 09, 2018 at 10​:17​:09PM +0100, Dave Mitchell wrote​:

On Sat, Jun 09, 2018 at 08​:55​:13AM -0700, Sergey Aleynikov via RT wrote​:

On Sat, 09 Jun 2018 08​:05​:27 -0700, jkeenan wrote​:

The results were ambiguous; see attachment. I'm puzzled as to why the
failure message points to Test​::Builder rather than to DBC​::CSV.

I've tried to bisect this too, but there's some floating/timing issue.
The furthest failure I've got is at revision 4bd1355, which is older
then yours.

The scalar which is being double-freed (during global destruction)
is always the tie object returned by

DBI​::DBD​::SqlEngine​::TieTables​::TIEHASH\(\)

at DBI/DBD/SqlEngine.pm​:1125.

But the triggering of the double-free seems random, and can't even
be reproduced consistently by setting PERL_HASH_SEED to a known value.

I'm still investigating further.

I've now seen a failure under 5.26.0 (one failure in about 200 runs of
t/82_free_unref_scalar.t).

--
"Do not dabble in paradox, Edward, it puts you in danger of fortuitous wit."
  -- Lady Croom, "Arcadia"

1 similar comment
@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 9, 2018

From @iabyn

On Sat, Jun 09, 2018 at 10​:17​:09PM +0100, Dave Mitchell wrote​:

On Sat, Jun 09, 2018 at 08​:55​:13AM -0700, Sergey Aleynikov via RT wrote​:

On Sat, 09 Jun 2018 08​:05​:27 -0700, jkeenan wrote​:

The results were ambiguous; see attachment. I'm puzzled as to why the
failure message points to Test​::Builder rather than to DBC​::CSV.

I've tried to bisect this too, but there's some floating/timing issue.
The furthest failure I've got is at revision 4bd1355, which is older
then yours.

The scalar which is being double-freed (during global destruction)
is always the tie object returned by

DBI​::DBD​::SqlEngine​::TieTables​::TIEHASH\(\)

at DBI/DBD/SqlEngine.pm​:1125.

But the triggering of the double-free seems random, and can't even
be reproduced consistently by setting PERL_HASH_SEED to a known value.

I'm still investigating further.

I've now seen a failure under 5.26.0 (one failure in about 200 runs of
t/82_free_unref_scalar.t).

--
"Do not dabble in paradox, Edward, it puts you in danger of fortuitous wit."
  -- Lady Croom, "Arcadia"

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 10, 2018

From @iabyn

On Sat, Jun 09, 2018 at 11​:05​:23PM +0100, Dave Mitchell wrote​:

I've now seen a failure under 5.26.0 (one failure in about 200 runs of
t/82_free_unref_scalar.t).

I've now bisected it (kinda). From at least 5.26.0 until the commit below,
t/82_free_unref_scalar.t would typically trigger a failure once every 100
runs or so, and that failure would typically involve a single double-freed
SV. From the commit through to bleed, around 1 in 4 test runs fails,
typically with several double-freed SVs.

I don't know yet whether blead has introduced or new bug, or whether it
just makes the test more sensitive to an existing bug - a bug which could
be in perl or one or more CPAN modules. :-(

commit e84e428
Author​: David Mitchell <davem@​iabyn.com>
AuthorDate​: Mon Jul 17 16​:33​:38 2017 +0100
Commit​: David Mitchell <davem@​iabyn.com>
CommitDate​: Thu Jul 27 11​:30​:24 2017 +0100

  Give OP_RV2HV a targ
 
  OP_RV2AV already has one; its not clear why OP_RV2HV didn't.
  Having one means that in scalar context it can return an int value
  without having to create a mortal. Ditto when its doing 'keys %h' via
  OPpRV2HV_ISKEYS.

--
All wight. I will give you one more chance. This time, I want to hear
no Wubens. No Weginalds. No Wudolf the wed-nosed weindeers.
  -- Life of Brian

1 similar comment
@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 10, 2018

From @iabyn

On Sat, Jun 09, 2018 at 11​:05​:23PM +0100, Dave Mitchell wrote​:

I've now seen a failure under 5.26.0 (one failure in about 200 runs of
t/82_free_unref_scalar.t).

I've now bisected it (kinda). From at least 5.26.0 until the commit below,
t/82_free_unref_scalar.t would typically trigger a failure once every 100
runs or so, and that failure would typically involve a single double-freed
SV. From the commit through to bleed, around 1 in 4 test runs fails,
typically with several double-freed SVs.

I don't know yet whether blead has introduced or new bug, or whether it
just makes the test more sensitive to an existing bug - a bug which could
be in perl or one or more CPAN modules. :-(

commit e84e428
Author​: David Mitchell <davem@​iabyn.com>
AuthorDate​: Mon Jul 17 16​:33​:38 2017 +0100
Commit​: David Mitchell <davem@​iabyn.com>
CommitDate​: Thu Jul 27 11​:30​:24 2017 +0100

  Give OP_RV2HV a targ
 
  OP_RV2AV already has one; its not clear why OP_RV2HV didn't.
  Having one means that in scalar context it can return an int value
  without having to create a mortal. Ditto when its doing 'keys %h' via
  OPpRV2HV_ISKEYS.

--
All wight. I will give you one more chance. This time, I want to hear
no Wubens. No Weginalds. No Wudolf the wed-nosed weindeers.
  -- Life of Brian

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 15, 2018

From @iabyn

On Sun, Jun 10, 2018 at 09​:36​:22PM +0100, Dave Mitchell wrote​:

On Sat, Jun 09, 2018 at 11​:05​:23PM +0100, Dave Mitchell wrote​:

I've now seen a failure under 5.26.0 (one failure in about 200 runs of
t/82_free_unref_scalar.t).

I've now bisected it (kinda). From at least 5.26.0 until the commit below,
t/82_free_unref_scalar.t would typically trigger a failure once every 100
runs or so, and that failure would typically involve a single double-freed
SV. From the commit through to bleed, around 1 in 4 test runs fails,
typically with several double-freed SVs.

I don't know yet whether blead has introduced or new bug, or whether it
just makes the test more sensitive to an existing bug - a bug which could
be in perl or one or more CPAN modules. :-(

I've now diagnosed the issues. It turns out that its not a perl issue.
but a number of bugs in various DBI / DBD components.

I've raised the following tickets​:


https://rt.cpan.org/Ticket/Display.html?id=125589

  This code in hook() in CSV_XS.xs is problematic in two ways​:

  XPUSHs (newRV_noinc ((SV *)hv));
  XPUSHs (newRV_noinc ((SV *)av));

  First, the newly-created RVs will leak, since they're not mortalised,
  and perl's arg stack isn't reference counted.

  Secondly, because the reference count of hv and av aren't increased,
  they may be prematurely freed and perl may subsequently crash.
  In fact, this is happening in

  https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133270

  In a stripped-down t/82_free_unref_scalar.t from DBD​::CSV, I see

  1) the RV created by newRV_noinc ((SV *)av) lasting until global
  destruction;
  2) in the meantime, the AV it points to is (prematurely) freed, and the
  SV later happens to be reallocated as an HV which happens to survive
  until global destruction;
  3) When the RV is eventually freed during global destruction, it triggers
  a double free of that new HV.

  I suspect you instead want something like

  mXPUSHs (newRV_inc ((SV *)hv));
  mXPUSHs (newRV_inc ((SV *)av));


https://rt.cpan.org/Ticket/Display.html?id=125590

  While debugging an issue with DBD​::CSV and/or perl​:

  https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133270

  I spotted code in DBI​::DBD​::SqlEngine​::TieTables which appears to
  be creating a circular reference, and thus leaking.

  DBI​::DBD​::SqlEngine​::TieTables​::TIEHASH() creates a tie object hash,
  one of whose entries is a reference back to the $dbh handle.

  I don't understand this code well enough to be 100% certain its a leak,
  but it sure looks like a weaken() is in order.

  The following code definitely leaks, although that may be in part due to
  a bug in Text​::CSV_XS which I have reported separately​:

  (it requires editing 'f_dir' to point to an untarred DBD-CSV-0.53
  directory)

  use strict;
  use warnings;

  use DBI;

  while (1) {
  my $dbh = DBI->connect ("dbi​:CSV​:", undef, undef, {
  f_schema => undef,
  f_dir => '/home/davem/tmp-nobackup/dbd-csv-free/DBD-CSV-0
  .53/t',
  f_dir_search => [],
  f_ext => ".csv/r",
  f_lock => 2,
  f_encoding => "utf8",

  csv_auto_diag => 0,

  RaiseError => 1,
  PrintError => 1,
  FetchHashKeyName => "NAME_lc",
  }) or die "$DBI​::errstr\n" || $DBI​::errstr;
  }


--
Any [programming] language that doesn't occasionally surprise the
novice will pay for it by continually surprising the expert.
  -- Larry Wall

1 similar comment
@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 15, 2018

From @iabyn

On Sun, Jun 10, 2018 at 09​:36​:22PM +0100, Dave Mitchell wrote​:

On Sat, Jun 09, 2018 at 11​:05​:23PM +0100, Dave Mitchell wrote​:

I've now seen a failure under 5.26.0 (one failure in about 200 runs of
t/82_free_unref_scalar.t).

I've now bisected it (kinda). From at least 5.26.0 until the commit below,
t/82_free_unref_scalar.t would typically trigger a failure once every 100
runs or so, and that failure would typically involve a single double-freed
SV. From the commit through to bleed, around 1 in 4 test runs fails,
typically with several double-freed SVs.

I don't know yet whether blead has introduced or new bug, or whether it
just makes the test more sensitive to an existing bug - a bug which could
be in perl or one or more CPAN modules. :-(

I've now diagnosed the issues. It turns out that its not a perl issue.
but a number of bugs in various DBI / DBD components.

I've raised the following tickets​:


https://rt.cpan.org/Ticket/Display.html?id=125589

  This code in hook() in CSV_XS.xs is problematic in two ways​:

  XPUSHs (newRV_noinc ((SV *)hv));
  XPUSHs (newRV_noinc ((SV *)av));

  First, the newly-created RVs will leak, since they're not mortalised,
  and perl's arg stack isn't reference counted.

  Secondly, because the reference count of hv and av aren't increased,
  they may be prematurely freed and perl may subsequently crash.
  In fact, this is happening in

  https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133270

  In a stripped-down t/82_free_unref_scalar.t from DBD​::CSV, I see

  1) the RV created by newRV_noinc ((SV *)av) lasting until global
  destruction;
  2) in the meantime, the AV it points to is (prematurely) freed, and the
  SV later happens to be reallocated as an HV which happens to survive
  until global destruction;
  3) When the RV is eventually freed during global destruction, it triggers
  a double free of that new HV.

  I suspect you instead want something like

  mXPUSHs (newRV_inc ((SV *)hv));
  mXPUSHs (newRV_inc ((SV *)av));


https://rt.cpan.org/Ticket/Display.html?id=125590

  While debugging an issue with DBD​::CSV and/or perl​:

  https://rt-archive.perl.org/perl5/Ticket/Display.html?id=133270

  I spotted code in DBI​::DBD​::SqlEngine​::TieTables which appears to
  be creating a circular reference, and thus leaking.

  DBI​::DBD​::SqlEngine​::TieTables​::TIEHASH() creates a tie object hash,
  one of whose entries is a reference back to the $dbh handle.

  I don't understand this code well enough to be 100% certain its a leak,
  but it sure looks like a weaken() is in order.

  The following code definitely leaks, although that may be in part due to
  a bug in Text​::CSV_XS which I have reported separately​:

  (it requires editing 'f_dir' to point to an untarred DBD-CSV-0.53
  directory)

  use strict;
  use warnings;

  use DBI;

  while (1) {
  my $dbh = DBI->connect ("dbi​:CSV​:", undef, undef, {
  f_schema => undef,
  f_dir => '/home/davem/tmp-nobackup/dbd-csv-free/DBD-CSV-0
  .53/t',
  f_dir_search => [],
  f_ext => ".csv/r",
  f_lock => 2,
  f_encoding => "utf8",

  csv_auto_diag => 0,

  RaiseError => 1,
  PrintError => 1,
  FetchHashKeyName => "NAME_lc",
  }) or die "$DBI​::errstr\n" || $DBI​::errstr;
  }


--
Any [programming] language that doesn't occasionally surprise the
novice will pay for it by continually surprising the expert.
  -- Larry Wall

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 15, 2018

@iabyn - Status changed from 'open' to 'rejected'

@p5pRT p5pRT closed this Jun 15, 2018
@p5pRT
Copy link
Author

@p5pRT p5pRT commented Jun 29, 2018

From @Tux

On Sat, 09 Jun 2018 03​:43​:33 -0700, "slaven@​rezic.de \(via RT\)" <perlbug-followup@​perl.org> wrote​:

-----------------------------------------------------------------
t/82_free_unref_scalar.t from DBD-CSV-0.53 fails on some newer
perl installations. The fail log looks like this​:

...
# Failed test 'there was an attempt to free unreferenced scalar'
# at /opt/perl-5.28.0-RC2t/lib/site_perl/5.28.0/Test/Builder.pm line 158.
# Attempt to free unreferenced scalar​: SV 0x3f6a0f8, Perl interpreter​: 0x2353010 during global destruction.
...
t/82_free_unref_scalar.t ..
All 405 subtests passed

The earliest perl with this failure is 5.27.9, according to CPAN Testers​:
http​://matrix.cpantesters.org/?dist=DBD-CSV%200.53;reports=1#sl=0,0

Also, there's a single failure with perl 5.26.2.

Has this been fixed by the recent release of Text​::CSV_XS-1.36 which
implements a fix by Dave M that should address this issue.

I can even change Makefile.PL to require Text​::CSV_XS 1.36 or newer if
perl is 5.28.0 or up

--
H.Merijn Brand http​://tux.nl Perl Monger http​://amsterdam.pm.org/
using perl5.00307 .. 5.27 porting perl5 on HP-UX, AIX, and openSUSE
http​://mirrors.develooper.com/hpux/ http​://www.test-smoke.org/
http​://qa.perl.org http​://www.goldmark.org/jeff/stupid-disclaimers/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant