Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libb-cow-perl: fails to build (test failures) on some architectures #1

Closed
gregoa opened this issue Nov 24, 2019 · 22 comments · Fixed by #4
Closed

libb-cow-perl: fails to build (test failures) on some architectures #1

gregoa opened this issue Nov 24, 2019 · 22 comments · Fixed by #4
Assignees

Comments

@gregoa
Copy link

gregoa commented Nov 24, 2019

We have the following bug reported to the Debian package of B-COW,
c.f. https://bugs.debian.org/945416

It doesn't seem to be a bug in the packaging, so you may want to take
a look. Thanks!

------8<-----------8<-----------8<-----------8<-----------8<-----

Source: libb-cow-perl
Version: 0.001-2
Severity: serious
Tags: upstream ftbfs
Justification: fails to build from source (but built successfully in the past)

As seen on https://buildd.debian.org/status/logs.php?pkg=libb-cow-perl&ver=0.001-2
libb-cow-perl has failures in the test suite on various
architectures.

The error is:

#   Failed test 'hash key a cowrefcnt is 0'
#   at t/01-basic.t line 56.
#          got: '1'
#     expected: '0'
SV = PV(0x2aa3deb4de0) at 0x2aa3ded7d80
  REFCNT = 2
  FLAGS = (POK,IsCOW,pPOK)
  PV = 0x2aa3e12ded0 "a"
  CUR = 1
  LEN = 0

#   Failed test 'hash key c cowrefcnt is 0'
#   at t/01-basic.t line 56.
#          got: '1'
#     expected: '0'
SV = PV(0x2aa3e63ff30) at 0x2aa3e6387f0
  REFCNT = 2
  FLAGS = (POK,IsCOW,pPOK)
  PV = 0x2aa3e63a020 "c"
  CUR = 1
  LEN = 0
# Looks like you failed 2 tests of 23.
t/01-basic.t ........... 
ok 1 - can cow with Perl 5.030000
ok 2 - !is_cow(undef)
ok 3 - is_cow('abcdef')
ok 4 - cowrefcnt is set to 1
ok 5 - is_cow('abcdef')
ok 6 - cowrefcnt for b is set to 2
ok 7 - cowrefcnt for str is set to 2
ok 8 - cowrefcnt_max: might need to adjust...
ok 9 - b is_cow
ok 10 - c !is_cow
ok 11 - cowrefcnt on uncowed SvPV
ok 12
ok 13
ok 14
ok 15
ok 16
ok 17
ok 18 - cowrefcnt decrease to 100
ok 19 - cowrefcnt decrease to 100
ok 20 - key a is cowed
not ok 21 - hash key a cowrefcnt is 0
ok 22 - key c is cowed
not ok 23 - hash key c cowrefcnt is 0
1..23
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/23 subtests 

Test Summary Report
- -------------------
t/01-basic.t         (Wstat: 512 Tests: 23 Failed: 2)
  Failed tests:  21, 23
  Non-zero exit status: 2
Files=3, Tests=25,  1 wallclock secs ( 0.02 usr  0.00 sys +  0.20 cusr  0.01 csys =  0.23 CPU)
Result: FAIL


Cheers,
gregor

------8<-----------8<-----------8<-----------8<-----------8<-----

Thanks for considering,
gregor herrmann,
Debian Perl Group

@atoomic atoomic self-assigned this Nov 25, 2019
@atoomic
Copy link
Owner

atoomic commented Nov 25, 2019

@gregoa could you give me more informations on the Perl version used, would appreciate if you could copy paste here the full output from perl -V
thanks

@gregoa
Copy link
Author

gregoa commented Nov 26, 2019

Here's the output from perl -V on one of the failing architectures (s390x). The others should be similar except for the arch-specific variables:

Summary of my perl5 (revision 5 version 30 subversion 0) configuration:
   
  Platform:
    osname=linux
    osvers=4.19.0
    archname=s390x-linux-gnu-thread-multi
    uname='linux localhost 4.19.0 #1 smp debian 4.19.0 s390x gnulinux '
    config_args='-Dmksymlinks -Dusethreads -Duselargefiles -Dcc=s390x-linux-gnu-gcc -Dcpp=s390x-linux-gnu-cpp -Dld=s390x-linux-gnu-gcc -Dccflags=-DDEBIAN -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/perl-uiMLgJ/perl-5.30.0=. -fstack-protector-strong -Wformat -Werror=format-security -Dldflags= -Wl,-z,relro -Dlddlflags=-shared -Wl,-z,relro -Dcccdlflags=-fPIC -Darchname=s390x-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.30 -Darchlib=/usr/lib/s390x-linux-gnu/perl/5.30 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/s390x-linux-gnu/perl5/5.30 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.30.0 -Dsitearch=/usr/local/lib/s390x-linux-gnu/perl/5.30.0 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -Ui_xlocale -Uversiononly -DDEBUGGING=-g -Doptimize=-O2 -dEs -Duseshrplib -Dlibperl=libperl.so.5.30.0'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='s390x-linux-gnu-gcc'
    ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
    optimize='-O2 -g'
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include'
    ccversion=''
    gccversion='9.2.1 20191025'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=87654321
    doublekind=4
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=2
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='s390x-linux-gnu-gcc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/gcc/s390x-linux-gnu/9/include-fixed /usr/include/s390x-linux-gnu /usr/lib /lib/s390x-linux-gnu /lib/../lib /usr/lib/s390x-linux-gnu /usr/lib/../lib /lib
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=libc-2.29.so
    so=so
    useshrplib=true
    libperl=libperl.so.5.30
    gnulibc_version='2.29'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_TIMES
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_IMPLICIT_CONTEXT
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
    USE_REENTRANT_API
    USE_THREAD_SAFE_LOCALE
  Locally applied patches:
    DEBPKG:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS default for modules installed from CPAN.
    DEBPKG:debian/db_file_ver - https://bugs.debian.org/340047 Remove overly restrictive DB_File version check.
    DEBPKG:debian/doc_info - Replace generic man(1) instructions with Debian-specific information.
    DEBPKG:debian/enc2xs_inc - https://bugs.debian.org/290336 Tweak enc2xs to follow symlinks and ignore missing @INC directories.
    DEBPKG:debian/errno_ver - https://bugs.debian.org/343351 Remove Errno version check due to upgrade problems with long-running processes.
    DEBPKG:debian/libperl_embed_doc - https://bugs.debian.org/186778 Note that libperl-dev package is required for embedded linking
    DEBPKG:fixes/respect_umask - Respect umask during installation
    DEBPKG:debian/writable_site_dirs - Set umask approproately for site install directories
    DEBPKG:debian/extutils_set_libperl_path - EU:MM: set location of libperl.a under /usr/lib
    DEBPKG:debian/no_packlist_perllocal - Don't install .packlist or perllocal.pod for perl or vendor
    DEBPKG:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to the binary targets.
    DEBPKG:debian/instmodsh_doc - Debian policy doesn't install .packlist files for core or vendor.
    DEBPKG:debian/ld_run_path - Remove standard libs from LD_RUN_PATH as per Debian policy.
    DEBPKG:debian/libnet_config_path - Set location of libnet.cfg to /etc/perl/Net as /usr may not be writable.
    DEBPKG:debian/perlivp - https://bugs.debian.org/510895 Make perlivp skip include directories in /usr/local
    DEBPKG:debian/squelch-locale-warnings - https://bugs.debian.org/508764 Squelch locale warnings in Debian package maintainer scripts
    DEBPKG:debian/patchlevel - https://bugs.debian.org/567489 List packaged patches for 5.30.0-9 in patchlevel.h
    DEBPKG:fixes/document_makemaker_ccflags - https://bugs.debian.org/628522 [rt.cpan.org #68613] Document that CCFLAGS should include $Config{ccflags}
    DEBPKG:debian/find_html2text - https://bugs.debian.org/640479 Configure CPAN::Distribution with correct name of html2text
    DEBPKG:debian/perl5db-x-terminal-emulator.patch - https://bugs.debian.org/668490 Invoke x-terminal-emulator rather than xterm in perl5db.pl
    DEBPKG:debian/cpan-missing-site-dirs - https://bugs.debian.org/688842 Fix CPAN::FirstTime defaults with nonexisting site dirs if a parent is writable
    DEBPKG:fixes/memoize_storable_nstore - [rt.cpan.org #77790] https://bugs.debian.org/587650 Memoize::Storable: respect 'nstore' option not respected
    DEBPKG:debian/makemaker-pasthru - https://bugs.debian.org/758471 Pass LD settings through to subdirectories
    DEBPKG:debian/makemaker-manext - https://bugs.debian.org/247370 Make EU::MakeMaker honour MANnEXT settings in generated manpage headers
    DEBPKG:debian/kfreebsd-softupdates - https://bugs.debian.org/796798 Work around Debian Bug#796798
    DEBPKG:fixes/autodie-scope - https://bugs.debian.org/798096 Fix a scoping issue with "no autodie" and the "system" sub
    DEBPKG:fixes/memoize-pod - [rt.cpan.org #89441] Fix POD errors in Memoize
    DEBPKG:debian/hurd-softupdates - https://bugs.debian.org/822735 Fix t/op/stat.t failures on hurd
    DEBPKG:fixes/math_complex_doc_great_circle - https://bugs.debian.org/697567 [rt.cpan.org #114104] Math::Trig: clarify definition of great_circle_midpoint
    DEBPKG:fixes/math_complex_doc_see_also - https://bugs.debian.org/697568 [rt.cpan.org #114105] Math::Trig: add missing SEE ALSO
    DEBPKG:fixes/math_complex_doc_angle_units - https://bugs.debian.org/731505 [rt.cpan.org #114106] Math::Trig: document angle units
    DEBPKG:fixes/cpan_web_link - https://bugs.debian.org/367291 CPAN: Add link to main CPAN web site
    DEBPKG:debian/hppa_op_optimize_workaround - https://bugs.debian.org/838613 Temporarily lower the optimization of op.c on hppa due to gcc-6 problems
    DEBPKG:debian/installman-utf8 - https://bugs.debian.org/840211 Generate man pages with UTF-8 characters
    DEBPKG:fixes/getopt-long-4 - https://bugs.debian.org/864544 [rt.cpan.org #122068] Fix issue #122068.
    DEBPKG:debian/hppa_opmini_optimize_workaround - https://bugs.debian.org/869122 Lower the optimization level of opmini.c on hppa
    DEBPKG:debian/sh4_op_optimize_workaround - https://bugs.debian.org/869373 Also lower the optimization level of op.c and opmini.c on sh4
    DEBPKG:debian/perldoc-pager - https://bugs.debian.org/870340 [rt.cpan.org #120229] Fix perldoc terminal escapes when sensible-pager is less
    DEBPKG:debian/prune_libs - https://bugs.debian.org/128355 Prune the list of libraries wanted to what we actually need.
    DEBPKG:debian/mod_paths - Tweak @INC ordering for Debian
    DEBPKG:debian/configure-regen - https://bugs.debian.org/762638 Regenerate Configure et al. after probe unit changes
    DEBPKG:debian/deprecate-with-apt - https://bugs.debian.org/747628 Point users to Debian packages of deprecated core modules
    DEBPKG:debian/disable-stack-check - https://bugs.debian.org/902779 [perl #133327] Disable debugperl stack extension checks for binary compatibility with perl
    DEBPKG:fixes/eumm-usrmerge - https://bugs.debian.org/913637 Avoid mangling /bin non-perl shebangs on merged-/usr systems
    DEBPKG:debian/perlbug-editor - https://bugs.debian.org/922609 Use "editor" as the default perlbug editor, as per Debian policy
    DEBPKG:fixes/gid-parsing - [79e302e] https://bugs.debian.org/941985 [perl #134169] (perl #134169) mg.c reset endptr after use
  Built under linux
  Compiled at Oct 27 2019 16:52:24
  @INC:
    /etc/perl
    /usr/local/lib/s390x-linux-gnu/perl/5.30.0
    /usr/local/share/perl/5.30.0
    /usr/lib/s390x-linux-gnu/perl5/5.30
    /usr/share/perl5
    /usr/lib/s390x-linux-gnu/perl/5.30
    /usr/share/perl/5.30
    /usr/local/lib/site_perl
    /usr/lib/s390x-linux-gnu/perl-base

@AdrianBunk
Copy link

The pattern of test failure is "fails on big endian".
(Debian/m68k does not run the tests during the build, in case someone wonders why that succeeded)

@plicease
Copy link
Contributor

Given that this is a fail in is_cow on hash keys, and it just exposes CowREFCNT macro which is part of the Perl core, this might be a bug in the Perl COW implementation on big endian cpus?

One (admittedly undesirable) workaround is to build Perl without COW support, but this dist will need to be updated to handle such configurations. #3 is how I would do it.

@pghmcfc
Copy link

pghmcfc commented Apr 22, 2020

Just thinking aloud here, but the tests are failing with is_cow on hash keys and the Perl API has:

SvIsCOW: Returns a U32 value indicating whether the SV is Copy-On-Write (either shared hash key scalars, or full Copy On Write scalars if 5.9.0 is configured for COW).

SvIsCOW_shared_hash: Returns a boolean indicating whether the SV is Copy-On-Write shared hash key scalar.

Maybe the difference between shared hash keys and other scalar values is relevant here?

@plicease
Copy link
Contributor

Just thinking aloud here, but the tests are failing with is_cow on hash keys and the Perl API has:

SvIsCOW: Returns a U32 value indicating whether the SV is Copy-On-Write (either shared hash key scalars, or full Copy On Write scalars if 5.9.0 is configured for COW).

SvIsCOW_shared_hash: Returns a boolean indicating whether the SV is Copy-On-Write shared hash key scalar.

Maybe the difference between shared hash keys and other scalar values is relevant here?

As an experiment, I tried this:

figment% git diff
diff --git a/COW.xs b/COW.xs
index 151a6d9..0607210 100644
--- a/COW.xs
+++ b/COW.xs
@@ -67,6 +67,22 @@ CODE:
 OUTPUT:
      RETVAL
 
+
+SV*
+cowrefcnt_shared_hash(sv)
+  SV *sv;
+CODE:
+{
+#if !B_CAN_COW
+    XSRETURN_UNDEF;
+#else
+    if ( SvIsCOW_shared_hash(sv) ) XSRETURN_IV( CowREFCNT(sv) );
+#endif
+    XSRETURN_UNDEF;
+}
+OUTPUT:
+     RETVAL
+
 SV*
 cowrefcnt_max()
 CODE:
diff --git a/lib/B/COW.pm b/lib/B/COW.pm
index 287629a..4a7c055 100644
--- a/lib/B/COW.pm
+++ b/lib/B/COW.pm
@@ -13,7 +13,7 @@ use XSLoader;
 
 XSLoader::load(__PACKAGE__);
 
-my @all_export = qw{ can_cow is_cow cowrefcnt cowrefcnt_max };
+my @all_export = qw{ can_cow is_cow cowrefcnt cowrefcnt_shared_hash cowrefcnt_max };
 
 our @EXPORT_OK = (
     @all_export,
@@ -78,4 +78,4 @@ If the string is not COW, then it will return undef.
 =head2 cowrefcnt_max()
 
 Will return the SV_COW_REFCNT_MAX of your Perl. (if COW is supported, this should
-be 255 unless customized).
\ No newline at end of file
+be 255 unless customized).
diff --git a/t/01-basic.t b/t/01-basic.t
index 7d4be88..41f4f65 100644
--- a/t/01-basic.t
+++ b/t/01-basic.t
@@ -55,7 +55,7 @@ if ( can_cow() ) {
         my %h = ( 'a'..'d' );
         foreach my $k ( sort keys %h ) {
             ok is_cow( $k ), "key $k is cowed";
-            is cowrefcnt( $k ), 0, "hash key $k cowrefcnt is 0" or Dump($k);
+            is cowrefcnt_shared_hash( $k ), 0, "hash key $k cowrefcnt is 0" or Dump($k);
         }
     }
 

but I get the same result running the test:

figment% prove -b t/01-basic.t 
t/01-basic.t .. 1/? 
#   Failed test 'hash key a cowrefcnt is 0'
#   at t/01-basic.t line 58.
#          got: '1'
#     expected: '0'
SV = PV(0x10312fa8) at 0x107a9bf0
  REFCNT = 2
  FLAGS = (POK,IsCOW,pPOK)
  PV = 0x10485404 "a"
  CUR = 1
  LEN = 0

#   Failed test 'hash key c cowrefcnt is 0'
#   at t/01-basic.t line 58.
#          got: '1'
#     expected: '0'
SV = PV(0x10743370) at 0x107a9b50
  REFCNT = 2
  FLAGS = (POK,IsCOW,pPOK)
  PV = 0x107a358c "c"
  CUR = 1
  LEN = 0
# Looks like you failed 2 tests of 22.
t/01-basic.t .. Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/22 subtests 

Test Summary Report
-------------------
t/01-basic.t (Wstat: 512 Tests: 22 Failed: 2)
  Failed tests:  20, 22
  Non-zero exit status: 2
Files=1, Tests=22,  1 wallclock secs ( 0.06 usr  0.02 sys +  0.18 cusr  0.02 csys =  0.28 CPU)
Result: FAIL

Or maybe I was trying the wrong thing? I am happy to try other variations while I happen to have a BE machine to test on.

@perlpunk
Copy link

I can also test things on ppc if it helps

@atoomic
Copy link
Owner

atoomic commented Apr 23, 2020

@perlpunk any chance I could use the ppc machine/vm a few hours to debug?

@perlpunk
Copy link

@atoomic Sorry, that's not possible. I can only test it by letting the SUSE Open Build Service run a build on a package. I think we also have ppc vms where I can login, but they are in the intranet only.

@perlpunk
Copy link

@atoomic You could create an account on https://build.opensuse.org/ and branch the https://build.opensuse.org/package/show/devel:languages:perl/perl-B-COW package and put debug statements in your tests. But a build can take a while.

@pghmcfc
Copy link

pghmcfc commented Apr 23, 2020

Dan Horák should be able to provide you with access to an s390x machine:

https://fedoraproject.org/wiki/Architectures/s390x#Shell_access_for_debugging

@atoomic
Copy link
Owner

atoomic commented Apr 23, 2020

thanks @perlpunk and @pghmcfc

@atoomic
Copy link
Owner

atoomic commented Apr 23, 2020

Thanks to the help of @Tux I can now reproduce the issue,
I hope to have a better understanding / fix soon.

@atoomic
Copy link
Owner

atoomic commented Apr 23, 2020

I think this is a bug in

 #   define CowREFCNT(sv)        (*(U8 *)(SvPVX(sv)+SvLEN(sv)-1))

for example, when increasing the length of one hash key using my_hash_key
we got 11 instead of 0

#   Failed test 'hash key cowrefcnt is 0'
#   at t/01-basic.t line 58.
#          got: '11'
#     expected: '0'
SV = PV(0x404fd780) at 0x40523ce0
  REFCNT = 1
  FLAGS = (POK,IsCOW,pPOK)
  PV = 0x40550fc4 "my_hash_key"
  CUR = 11
  LEN = 0

atoomic added a commit that referenced this issue Apr 23, 2020
Fix #1

If the SvPV has a LEN=0, we should
not try to read what's in front to
access the CowREFCNT and consider its
value is 0.
@atoomic
Copy link
Owner

atoomic commented Apr 23, 2020

this was reported upstream and I'm going to provide a fix upstream for it
Perl/perl5#17745

atoomic added a commit that referenced this issue Apr 23, 2020
Fix #1

If the SvPV has a LEN=0, we should
not try to read what's in front to
access the CowREFCNT and consider its
value is 0.

Upstream-URL: Perl/perl5#17745
@atoomic
Copy link
Owner

atoomic commented Apr 23, 2020

This is going to be fixed in v0.004 which was just uploaded to CPAN

@perlpunk
Copy link

Thanks!

@gregoa
Copy link
Author

gregoa commented Apr 24, 2020 via email

@atoomic
Copy link
Owner

atoomic commented Apr 24, 2020

I've performed some extra tests using one IBM/S390 using Fedora release 31 (Thirty One) and perl v5.30.2 and can confirm that 0.004 seems to work fine whereas it was broken on the previous version

@pghmcfc
Copy link

pghmcfc commented Apr 24, 2020

Built OK for multiple architectures and Fedora/EPEL versions:
https://bodhi.fedoraproject.org/updates/?packages=perl-B-COW
Thank you!

@gregoa
Copy link
Author

gregoa commented Apr 24, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants