Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC on windows build status #16417

Open
p5pRT opened this issue Feb 12, 2018 · 27 comments
Open

RFC on windows build status #16417

p5pRT opened this issue Feb 12, 2018 · 27 comments

Comments

@p5pRT
Copy link

@p5pRT p5pRT commented Feb 12, 2018

Migrated from rt.perl.org#132860 (status was 'open')

Searchable as RT132860$

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 12, 2018

From @dur-randir

Created by @dur-randir

I've recently done a Perl build on a Windows machine using VS 2017 and
found it's status a bit alarming​:

- perlwin32.html (aka README.win32) states "There should be no test
failures.", but at least three tests failed for 64-bit build (console
window is by default too small, so I haven't recorded them) and I
ended up with cpan/IPC-Cmd/t/01_IPC-Cmd.t stuck (for at least 10
minutes, until I've aborted it). Are those failures known? If not,
should I report them, or there're not enough volunteers to fix them,
so it's time to change the docs?

- during both 32-bit and 64-bit builds, I've seen a lot of warnings
about variables size mismatch conversions (in both ways). Most of them
seem to have the same underlying source, but they're really numerous.

Or is windows build currently supported only as a MinGW Strawberry
build by it's community?

Perl Info

Flags:
    category=core
    severity=low

Site configuration information for perl 5.27.9:

Configured by root at Sun Feb 11 01:07:53 MSK 2018.

Summary of my perl5 (revision 5 version 27 subversion 9) configuration:
  Commit id: c0e3b4b51cabf15ed8fc5f564dfeea31c25f5239
  Platform:
    osname=linux
    osvers=4.9.0-5-amd64
    archname=x86_64-linux
    uname='linux dorothy 4.9.0-5-amd64 #1 smp debian 4.9.65-3+deb9u2
(2018-01-04) x86_64 gnulinux '
    config_args='-de -Dusedevel'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-fwrapv -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
    optimize='-O2'
    cppflags='-fwrapv -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='6.3.0 20170516'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib
/usr/lib/gcc/x86_64-linux-gnu/6/include-fixed
/usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu
/lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib
    libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.24.so
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.24'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'



@INC for perl 5.27.9:
    lib
    /usr/local/lib/perl5/site_perl/5.27.9/x86_64-linux
    /usr/local/lib/perl5/site_perl/5.27.9
    /usr/local/lib/perl5/5.27.9/x86_64-linux
    /usr/local/lib/perl5/5.27.9


Environment for perl 5.27.9:
    HOME=/home/afl
    LANG=en_US.UTF-8
    LANGUAGE=en_US:en
    LC_CTYPE=en_US.UTF-8
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/afl/perlbrew/bin:/home/afl/perlbrew/perls/perl-5.20.2/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
    PERLBREW_BASHRC_VERSION=0.78
    PERLBREW_HOME=/home/afl/.perlbrew
    PERLBREW_MANPATH=/home/afl/perlbrew/perls/perl-5.20.2/man
    PERLBREW_PATH=/home/afl/perlbrew/bin:/home/afl/perlbrew/perls/perl-5.20.2/bin
    PERLBREW_PERL=perl-5.20.2
    PERLBREW_ROOT=/home/afl/perlbrew
    PERLBREW_VERSION=0.78
    PERL_BADLANG (unset)
    SHELL=/usr/bin/zsh

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 12, 2018

From @steve-m-hay

On 12 February 2018 at 20​:47, Sergey Aleynikov
<perlbug-followup@​perl.org> wrote​:

# New Ticket Created by Sergey Aleynikov
# Please include the string​: [perl #132860]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132860 >

This is a bug report for perl from sergey.aleynikov@​gmail.com,
generated with the help of perlbug 1.41 running under perl 5.27.9.

-----------------------------------------------------------------
[Please describe your issue here]

I've recently done a Perl build on a Windows machine using VS 2017 and
found it's status a bit alarming​:

- perlwin32.html (aka README.win32) states "There should be no test
failures.", but at least three tests failed for 64-bit build (console
window is by default too small, so I haven't recorded them) and I
ended up with cpan/IPC-Cmd/t/01_IPC-Cmd.t stuck (for at least 10
minutes, until I've aborted it). Are those failures known? If not,
should I report them, or there're not enough volunteers to fix them,
so it's time to change the docs?

I just did a build with VS2015 and found three or four tests hung. I
killed the hung perl.exe processes to let it continue more quickly.
Some have a "watchdog" timeout on them so would continue eventually.
This used to be quite a problem but I thought things had got better
more recently. I will try some more tests with different
compilers/systems. No tests failed other than the ones I killed.

There shouldn't be any test failures, other than what is documented in
README.win32. If there are any then they need fixing, skipping or
documenting as appropriate.

- during both 32-bit and 64-bit builds, I've seen a lot of warnings
about variables size mismatch conversions (in both ways). Most of them
seem to have the same underlying source, but they're really numerous.

Lots of warnings are, unfortunately, normal with some versions. I
think older ones (e.g. VC6) and 64-bit builds tend to be worst. I've
often tried to iron them out, but they always creep back in over time.
I thought recent versions (e.g. VS2015) in 32-bit builds only had a
small number of warnings, but I've not looked closely for a while.

Or is windows build currently supported only as a MinGW Strawberry
build by it's community?

Visual C++ builds are definitely very much still supported.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 12, 2018

The RT System itself - Status changed from 'new' to 'open'

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 12, 2018

From @steve-m-hay

On 12 February 2018 at 21​:49, Steve Hay <steve.m.hay@​googlemail.com> wrote​:

- during both 32-bit and 64-bit builds, I've seen a lot of warnings
about variables size mismatch conversions (in both ways). Most of them
seem to have the same underlying source, but they're really numerous.

Lots of warnings are, unfortunately, normal with some versions. I
think older ones (e.g. VC6) and 64-bit builds tend to be worst. I've
often tried to iron them out, but they always creep back in over time.
I thought recent versions (e.g. VS2015) in 32-bit builds only had a
small number of warnings, but I've not looked closely for a while.

From a 32-bit VS2015 build I have only these warnings from core code​:

..\mathoms.c(694)​: warning C4028​: formal parameter 2 different from declaration
..\pad.c(2755)​: warning C4267​: '='​: conversion from 'size_t' to 'U8',
possible loss of data
..\pp_sys.c(2234)​: warning C4244​: 'initializing'​: conversion from
'__int64' to 'NV', possible loss of data
..\pp_sys.c(4719)​: warning C4244​: 'initializing'​: conversion from
'time_t' to 'IV', possible loss of data
..\pp_sys.c(4919)​: warning C4244​: 'initializing'​: conversion from
'time_t' to 'IV', possible loss of data
..\regcomp.c(3393)​: warning C4267​: '+='​: conversion from 'size_t' to
'U8', possible loss of data
..\regcomp.c(12538)​: warning C4267​: '='​: conversion from 'size_t' to
'U8', possible loss of data
..\regcomp.c(13989)​: warning C4267​: '='​: conversion from 'size_t' to
'U8', possible loss of data
..\regcomp.c(19150)​: warning C4244​: 'initializing'​: conversion from
'U32' to 'const U8', possible loss of data
..\regexec.c(2253)​: warning C4244​: 'function'​: conversion from 'U32'
to 'const U8', possible loss of data
..\sv.c(12562)​: warning C4244​: '='​: conversion from 'PERL_INTMAX_T' to
'IV', possible loss of data
..\sv.c(12617)​: warning C4244​: '='​: conversion from 'PERL_UINTMAX_T'
to 'UV', possible loss of data
vdir.h(649)​: warning C4244​: 'argument'​: conversion from 'const WCHAR'
to 'char', possible loss of data
win32.c(4495)​: warning C4996​: 'GetVersionExA'​: was declared deprecated

and these from modules​:

MD5.xs(763)​: warning C4244​: '='​: conversion from 'U32' to 'char',
possible loss of data
MD5.xs(764)​: warning C4244​: '='​: conversion from 'U32' to 'char',
possible loss of data
MD5.xs(765)​: warning C4244​: '='​: conversion from 'U32' to 'char',
possible loss of data
MD5.xs(766)​: warning C4244​: '='​: conversion from 'U32' to 'char',
possible loss of data
sdbm.c(38)​: warning C4273​: 'malloc'​: inconsistent dll linkage
sdbm.c(39)​: warning C4273​: 'free'​: inconsistent dll linkage
Socket.xs(959)​: warning C4244​: '='​: conversion from 'unsigned long' to
'unsigned short', possible loss of data
Win32.xs(1771)​: warning C4996​: 'GetVersionExA'​: was declared deprecated

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 19, 2018

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 19, 2018

From @steve-m-hay

On 19 February 2018 at 00​:06, Sergey Aleynikov via RT
<perlbug-followup@​perl.org> wrote​:

Here's what I've got with a x64 build using MSVC 2017.

Build warnings part​:

[...]

Build warnings in modules​:

[...]

That's about the same as I get with VS2015 or VS2017 in x64 builds.
x86 builds fare much better, though VC6 has lots of warnings.

Test failures part​:

../cpan/IPC-Cmd/t/01_IPC-Cmd.t - hangs and has to be killed

I occasionally see certain tests (01_IPC-Cmd.t is one of them) hanging
and needing to be killed, but it only seems to happen on my Windows 7
machine. On Windows 8 I've yet to see it happen. I don't know if
that's due to the OS, or other differences, e.g. they have different
anti-virus software on them.

Full result is​:

Test Summary Report
-------------------
porting/checkcfgvar.t (Wstat​: 0 Tests​: 24 Failed​: 1)
Failed test​: 23
porting/customized.t (Wstat​: 0 Tests​: 28 Failed​: 28)
Failed tests​: 1-28
porting/pod_rules.t (Wstat​: 0 Tests​: 6 Failed​: 5)
Failed tests​: 2-6
Parse errors​: Bad plan. You planned 8 tests but ran 6.
porting/regen.t (Wstat​: 65280 Tests​: 11 Failed​: 1)
Failed test​: 11
Non-zero exit status​: 255
Parse errors​: Bad plan. You planned 43 tests but ran 11.
../cpan/CPAN-Meta-YAML/t/30_yaml_spec_tml.t (Wstat​: 512 Tests​: 0 Failed​: 0)
Non-zero exit status​: 2
Parse errors​: No plan found in TAP output
../cpan/CPAN-Meta-YAML/t/31_local_tml.t (Wstat​: 512 Tests​: 0 Failed​: 0)
Non-zero exit status​: 2
Parse errors​: No plan found in TAP output
../cpan/CPAN-Meta-YAML/t/32_world_tml.t (Wstat​: 512 Tests​: 0 Failed​: 0)
Non-zero exit status​: 2
Parse errors​: No plan found in TAP output
../cpan/IPC-Cmd/t/01_IPC-Cmd.t (Wstat​: 256 Tests​: 20 Failed​: 0)
Non-zero exit status​: 1
Parse errors​: No plan found in TAP output
../cpan/podlators/t/text/encoding.t (Wstat​: 65280 Tests​: 1 Failed​: 0)
Non-zero exit status​: 255
Parse errors​: Bad plan. You planned 7 tests but ran 1.

This is MUCH more worrying. I don't get any test failures on either OS
with either VS2015 or VS2017, whether x86 or x64, other than from
tests that hung and I had to kill.

Is this run from a clean git workspace? I'm trying to picture how
something like porting/customized.t could possibly fail. Do you have
any modified/staged/committed files?

Are you using the full VS2017, or the free (Community Edition)
version? I don't think I ever got round to trying the latter, though I
can't imagine it would cause porting/customized.t to fail.

Have you set CCTYPE=MSVC141?

Please can you re-run some of those failed tests with the verbose
option to get a better idea of what's going on?
e.g. (from the t/ folder)​: .\perl harness -v porting\customized.t

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 19, 2018

From @sisyphus

-----Original Message-----
From​: Steve Hay via perl5-porters
Sent​: Monday, February 19, 2018 8​:11 PM
To​: perlbug-followup@​perl.org
Subject​: Re​: [perl #132860] RFC on windows build status

I occasionally see certain tests (01_IPC-Cmd.t is one of them) hanging and
needing to be killed, but it only seems to happen on my Windows 7 machine.
On Windows 8 I've yet to see it happen. I don't know if that's due to the
OS, or other differences, e.g. they have different anti-virus software on
them.

cpan/IPC-Cmd/t/01_IPC-Cmd.t always hangs for me with my x64 mingw builds on
Windows 7, as too does cpan/IO-Compress/t/105oneshot-rawdeflate.t.
These are the only two test files that hang for me, and there's no such
hanging with x86 builds.

Also, the hangs occur only when run in 'make test'. When run normally as
perl scripts, there's no problem at all and both of those offending test
scripts then pass.
This leads me to believe that it's unlikely that anti-virus software is to
blame, rather that it's maybe something that Test​::Harness is doing. (My
Windows 7 box doesn't have any active anti-virus software.)

I've pretty much given up on being bothered by these hangs. I just kill them
in Process Explorer (procexp.exe) , which allows the rest of the test suite
to be run.

Cheers,
Rob

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 19, 2018

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 19, 2018

From @dur-randir

On Mon, 19 Feb 2018 01​:11​:35 -0800, shay wrote​:

From a 32-bit VS2015 build I have only these warnings from core code​:

Yes, x86 build is much more clearer. I haven't compared it on a line-by-line basis, but warnings look really similar to what you've posted before.

I occasionally see certain tests (01_IPC-Cmd.t is one of them) hanging
and needing to be killed, but it only seems to happen on my Windows 7
machine. On Windows 8 I've yet to see it happen. I don't know if
that's due to the OS, or other differences, e.g. they have different
anti-virus software on them.

This is indeed Windows 7, but it's a clean machine without any antivirus software installed (MS default one is switched off too). But this contradicts documented behaviour "all tests should pass".

Is this run from a clean git workspace? I'm trying to picture how
something like porting/customized.t could possibly fail. Do you have
any modified/staged/committed files?

Have you set CCTYPE=MSVC141?

This is a build done after 'git clean -dfx' was made from 'c​:/perl-git' dir. I'll try to build from a fresh checkout. There're only two modified files​:

- Makefile for CCTYPE and/or WIN64
- config.vc for 'ar'

I'll check exact value set for CCTYPE, but it's not the default one (I've picked one from README.win32 recommendations).

Are you using the full VS2017, or the free (Community Edition)
version? I don't think I ever got round to trying the latter, though I
can't imagine it would cause porting/customized.t to fail.

This is a CE VS 2017.

Please can you re-run some of those failed tests with the verbose
option to get a better idea of what's going on?
e.g. (from the t/ folder)​: .\perl harness -v porting\customized.t

Yes, I'm planning to report all individual failures/warnings for all modules involved, and will send detailed report for core tests here.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 19, 2018

From @dur-randir

On Mon, 19 Feb 2018 01​:11​:35 -0800, shay wrote​:

Is this run from a clean git workspace? I'm trying to picture how
something like porting/customized.t could possibly fail. Do you have
any modified/staged/committed files?

So, here's a run from a clean checkout with the following config applied​:

c​:\perl-git-fresh>git diff

Inline Patch
diff --git a/win32/Makefile b/win32/Makefile
index 3889ff9ec5..4020b22621 100644
--- a/win32/Makefile
+++ b/win32/Makefile
@@ -21,7 +21,7 @@
 # newly built perl.
 #
 INST_DRV       = c:
-INST_TOP       = $(INST_DRV)\perl
+INST_TOP       = $(INST_DRV)\perl-blead

 #
 # Uncomment if you want to build a 32-bit Perl using a 32-bit compiler
@@ -112,7 +112,7 @@ DEFAULT_INC_EXCLUDES_DOT = define
 # uncomment exactly one of the following
 #
 # Visual C++ 6.0 (aka Visual C++ 98)
-CCTYPE         = MSVC60
+#CCTYPE                = MSVC60
 # Visual C++ .NET 2002/2003 (aka Visual C++ 7.0/7.1) (full version)
 #CCTYPE                = MSVC70
 # Visual C++ Toolkit 2003 (aka Visual C++ 7.1) (free command-line tools)
@@ -132,7 +132,7 @@ CCTYPE              = MSVC60
 # Visual C++ 2015 (aka Visual C++ 14.0) (full version or Express Edition)
 #CCTYPE                = MSVC140
 # Visual C++ 2017 (aka Visual C++ 14.1) (full version or Community Edition)
-#CCTYPE                = MSVC141
+CCTYPE         = MSVC141

 #
 # If you are using Intel C++ Compiler uncomment this
@@ -161,6 +161,7 @@ CCTYPE              = MSVC60
 # which causes warnings to be printed on STDERR, which in turn causes a
 # few tests to fail.)
 #
+
 #CFG           = Debug

 #
diff --git a/win32/config.vc b/win32/config.vc
index a7cea270ce..3b0fc92ca4 100644
--- a/win32/config.vc
+++ b/win32/config.vc
@@ -21,7 +21,8 @@ api_revision='~PERL_API_REVISION~'
 api_subversion='~PERL_API_SUBVERSION~'
 api_version='~PERL_API_VERSION~'
 api_versionstring='~PERL_API_REVISION~.~PERL_API_VERSION~.~PERL_API_SUBVERSION~'
-ar='lib -ltcg'
+#ar='lib -ltcg'
+ar='link /lib'
 archlib='~INST_TOP~~INST_VER~\lib~INST_ARCH~'
 archlibexp='~INST_TOP~~INST_VER~\lib~INST_ARCH~'
 archname64=''

and then nmake; nmake test

with the same results​:

Test Summary Report


porting/checkcfgvar.t (Wstat​: 0 Tests​: 24 Failed​: 1)
  Failed test​: 23
porting/customized.t (Wstat​: 0 Tests​: 28 Failed​: 28)
  Failed tests​: 1-28
porting/pod_rules.t (Wstat​: 0 Tests​: 6 Failed​: 5)
  Failed tests​: 2-6
  Parse errors​: Bad plan. You planned 8 tests but ran 6.
porting/regen.t (Wstat​: 65280 Tests​: 11 Failed​: 1)
  Failed test​: 11
  Non-zero exit status​: 255
  Parse errors​: Bad plan. You planned 43 tests but ran 11.
../cpan/CPAN-Meta-YAML/t/30_yaml_spec_tml.t (Wstat​: 512 Tests​: 0 Failed​: 0)
  Non-zero exit status​: 2
  Parse errors​: No plan found in TAP output
../cpan/CPAN-Meta-YAML/t/31_local_tml.t (Wstat​: 512 Tests​: 0 Failed​: 0)
  Non-zero exit status​: 2
  Parse errors​: No plan found in TAP output
../cpan/CPAN-Meta-YAML/t/32_world_tml.t (Wstat​: 512 Tests​: 0 Failed​: 0)
  Non-zero exit status​: 2
  Parse errors​: No plan found in TAP output
../cpan/Encode/t/truncated_utf8.t (Wstat​: 0 Tests​: 9 Failed​: 0)
  TODO passed​: 9
../cpan/IPC-Cmd/t/01_IPC-Cmd.t (Wstat​: 256 Tests​: 20 Failed​: 0)
  Non-zero exit status​: 1
  Parse errors​: No plan found in TAP output
../cpan/podlators/t/text/encoding.t (Wstat​: 65280 Tests​: 1 Failed​: 0)
  Non-zero exit status​: 255
  Parse errors​: Bad plan. You planned 7 tests but ran 1.
../ext/IPC-Open3/t/IPC-Open3.t (Wstat​: 0 Tests​: 45 Failed​: 0)
  TODO passed​: 25
Files=2650, Tests=1085958, 1305 wallclock secs (70.58 usr + 6.48 sys = 77.06 CPU)
Result​: FAIL
NMAKE : fatal error U1077​: '.\perl.exe' : return code '0x23'
Stop.

Here're individual runs for all failing test​:

c​:\perl-git-fresh\t>.\perl.exe harness -v porting/checkcfgvar.t
porting/checkcfgvar.t ..
1..24
ok 1 - Cross/config.sh-arm-linux sorted
ok 2 - Cross/config.sh-arm-linux has no missing keys
ok 3 - Cross/config.sh-arm-linux-n770 sorted
ok 4 - Cross/config.sh-arm-linux-n770 has no missing keys
ok 5 - NetWare/config.wc sorted
ok 6 - NetWare/config.wc has no missing keys
ok 7 - Porting/config.sh sorted
ok 8 - Porting/config.sh has no missing keys
ok 9 # skip configure.com doesn't need to be sorted
ok 10 - configure.com has no missing keys
ok 11 - plan9/config_sh.sample sorted
ok 12 - plan9/config_sh.sample has no missing keys
ok 13 - symbian/config.sh sorted
ok 14 - symbian/config.sh has no missing keys
ok 15 - uconfig.sh sorted
ok 16 - uconfig.sh has no missing keys
ok 17 - uconfig64.sh sorted
ok 18 - uconfig64.sh has no missing keys
ok 19 - win32/config.ce sorted
ok 20 - win32/config.ce has no missing keys
ok 21 - win32/config.gc sorted
ok 22 - win32/config.gc has no missing keys
not ok 23 - win32/config.vc is not sorted
ok 24 - win32/config.vc has no missing keys
Failed 1/24 subtests
  (less 1 skipped subtest​: 22 okay)

Test Summary Report


porting/checkcfgvar.t (Wstat​: 0 Tests​: 24 Failed​: 1)
  Failed test​: 23
Files=1, Tests=24, 0 wallclock secs ( 0.03 usr + 0.00 sys = 0.03 CPU)
Result​: FAIL

c​:\perl-git-fresh\t>.\perl.exe harness -v porting/customized.t
porting/customized.t .. # Failed test 1 - SHA for dist/Devel-PPPort/parts/embed.fnc matches stashed SHA at porting/custo
mized.t line 112
# got "789560e0c0006bf8e4dc99a50af4ca48f5d97872"
# expected "e030719d9c6921810554a8e2d398543348b4878c"
# Failed test 2 - SHA for cpan/Digest/Digest.pm matches stashed SHA at porting/customized.t line 112
# got "d7b33968269dc0bc1bff76e48fe635f6d97e68ba"
# expected "43f7f544cb11842b2f55c73e28930da50774e081"
# Failed test 3 - SHA for cpan/Locale-Maketext-Simple/lib/Locale/Maketext/Simple.pm matches stashed SHA at porting/custo
mized.t line 112
# got "413e5aa2531fb5189d2817b4681eb31618cb705d"
# expected "57ed38905791a17c150210cd6f42ead22a7707b6"
# Failed test 4 - SHA for cpan/Math-Complex/lib/Math/Complex.pm matches stashed SHA at porting/customized.t line 112
# got "5773f45124c3843db148b4a9d6c7de4d17ce6a8c"
# expected "198ea6c6c584f5ea79a0fd7e9d411d0878f3b2af"
# Failed test 5 - SHA for cpan/Math-Complex/t/Complex.t matches stashed SHA at porting/customized.t line 112
# got "10c528abaebe4273d55b027de99e8a3cd6f1074e"
# expected "4f307ed6fc59f1e5fb0e6b11103fc631b6bdb335"
# Failed test 6 - SHA for cpan/Math-Complex/t/Trig.t matches stashed SHA at porting/customized.t line 112
# got "9ab6450e051d7c0f0f1301e3fb8a0e7ad124dc68"
# expected "2682526e23a161d54732c2a66393fe4a234d1865"
# Failed test 7 - SHA for cpan/Memoize/Memoize.pm matches stashed SHA at porting/customized.t line 112
# got "c91c85ec68460af15c344b34f5eb8c446fee0d3a"
# expected "902092ff91cdec9c7b4bd06202eb179e1ce26ca2"
# Failed test 8 - SHA for cpan/NEXT/lib/NEXT.pm matches stashed SHA at porting/customized.t line 112
# got "eb849b978f022a93b64186497e2e661916f95da3"
# expected "2c83d03ee361816e53ccb931137d314ab878d19f"
# Failed test 9 - SHA for cpan/NEXT/t/next.t matches stashed SHA at porting/customized.t line 112
# got "5056b6c3f898d7311da5694a43f68866ded654b0"
# expected "66fd60eb0f75b6f3eea95d1dee745f9a7a348146"
# Failed test 10 - SHA for dist/Net-Ping/t/000_load.t matches stashed SHA at porting/customized.t line 112
# got "c35df6b4abc3f103016697d722fdd30462e7574c"
# expected "deff5dc2ca54dae28cb19d3631427db127279ac2"
# Failed test 11 - SHA for dist/Net-Ping/t/001_new.t matches stashed SHA at porting/customized.t line 112
# got "3d23e0b76000dc6aaee6d6b6b15a8b005a41d516"
# expected "90c9d63509b3efc8941449fbd1ca8b807fa42040"
# Failed test 12 - SHA for dist/Net-Ping/t/500_ping_icmp.t matches stashed SHA at porting/customized.t line 112
# got "fd0c29a3a2ad5f694e2a30c787ca2742df70441e"
# expected "a003daa5eaf215e58234786bb1fbfbebf669bf44"
# Failed test 13 - SHA for cpan/Pod-Checker/t/pod/contains_bad_pod.xr matches stashed SHA at porting/customized.t line 1
12
# got "40318e6c7516fa4d4e52438548a594f6575305bd"
# expected "73538fd80dfe6e19ad561fe034009b44460208f6"
# Failed test 14 - SHA for cpan/Pod-Checker/t/pod/selfcheck.t matches stashed SHA at porting/customized.t line 112
# got "2a3ecb8c20e1ae7ca9c7967aaf415cc3f47c4ec1"
# expected "8ce3cfd38e4b9bcf5bc7fe7f2a14195e49aed7d8"
# Failed test 15 - SHA for cpan/Pod-Checker/t/pod/testcmp.pl matches stashed SHA at porting/customized.t line 112
# got "72674d02b6116678c7aba72ef0f96a6c81137866"
# expected "a0cd5c8eca775c7753f4464eee96fa916e3d8a16"
# Failed test 13 - SHA for cpan/Pod-Checker/t/pod/contains_bad_pod.xr matches stashed SHA at porting/customized.t line 1
12
# got "40318e6c7516fa4d4e52438548a594f6575305bd"
# expected "73538fd80dfe6e19ad561fe034009b44460208f6"
# Failed test 14 - SHA for cpan/Pod-Checker/t/pod/selfcheck.t matches stashed SHA at porting/customized.t line 112
# got "2a3ecb8c20e1ae7ca9c7967aaf415cc3f47c4ec1"
# expected "8ce3cfd38e4b9bcf5bc7fe7f2a14195e49aed7d8"
# Failed test 15 - SHA for cpan/Pod-Checker/t/pod/testcmp.pl matches stashed SHA at porting/customized.t line 112
# got "72674d02b6116678c7aba72ef0f96a6c81137866"
# expected "a0cd5c8eca775c7753f4464eee96fa916e3d8a16"
# Failed test 16 - SHA for cpan/Pod-Checker/t/pod/testpchk.pl matches stashed SHA at porting/customized.t line 112
# got "3b0115c6de08c5bafb62e5a6a07f5211b7a0b69e"
# expected "b2072c7f4379fd050e15424175d7cac5facf5b3b"
# Failed test 17 - SHA for cpan/Pod-Perldoc/lib/Pod/Perldoc.pm matches stashed SHA at porting/customized.t line 112
# got "a2884426d40f0348bcef2e65488f4b2a7af4e3c5"
# expected "582be34c077c9ff44d99914724a0cc2140bcd48c"
# Failed test 18 - SHA for cpan/autodie/lib/autodie/exception.pm matches stashed SHA at porting/customized.t line 112
# got "173e1a0e770a9cb03921952155959d812994f045"
# expected "b99e4e35a9ed36de94d54437888822ced4936207"
# Failed test 19 - SHA for cpan/autodie/lib/autodie/hints.pm matches stashed SHA at porting/customized.t line 112
# got "4633c6ad845d3664d200a181df6eaa1d7325c449"
# expected "e1998fec61fb4e82fe46585bd82c73200be6f262"
# Failed test 20 - SHA for cpan/autodie/t/exceptions.t matches stashed SHA at porting/customized.t line 112
# got "06ecb0614cace0e602610a5fb5ac2c6f25b8030e"
# expected "ad315a208f875e06b0964012ce8d65daa438c036"
# Failed test 21 - SHA for cpan/autodie/t/lib/Hints_pod_examples.pm matches stashed SHA at porting/customized.t line 112

# got "4cc48268b9316505c095dccb1523ae380b5f2f61"
# expected "6944c218e9754b3613c8d0c90a5ae8aceccb5c99"
# Failed test 22 - SHA for cpan/autodie/t/mkdir.t matches stashed SHA at porting/customized.t line 112
# got "fd39f302df9d1fea9e6778a28d4ea1b220392363"
# expected "9e70d2282a3cc7d76a78bf8144fccba20fb37dac"
# Failed test 23 - SHA for cpan/experimental/t/basic.t matches stashed SHA at porting/customized.t line 112
# got "db0fb454ef950e9d6f11bea52658c74d9aca2884"
# expected "a073ea03ccc98dec496569f3648ab01a5fe1c7a0"
# Failed test 24 - SHA for cpan/perlfaq/lib/perlfaq5.pod matches stashed SHA at porting/customized.t line 112
# got "2afa50bb00ff79c8116a97d770709246a93c51c1"
# expected "bcc1b6af3b6dff3973643acf8d5e741463374123"
# Failed test 25 - SHA for cpan/perlfaq/lib/perlfaq8.pod matches stashed SHA at porting/customized.t line 112
# got "4bb6c2c10ab2334f1e56f08bd55cb924e3de532d"
# expected "bffbc0c8fa828aead24e0891a5e789369a8e0743"
# Failed test 26 - SHA for pod/perlpodstyle.pod matches stashed SHA at porting/customized.t line 112
# got "b86fde30695c1747c9684aab6a66aafa9c743019"
# expected "c6500c9950b46e8228d4adbc09a3ee2ef23de2d0"
# Failed test 27 - SHA for cpan/version/lib/version.pm matches stashed SHA at porting/customized.t line 112
# got "f92dcd1af4079bb40457ea43828df80d8a456a01"
# expected "bee369df1bd84e09107a90d72ec12c38bcb97cce"
# Failed test 28 - SHA for vxs.inc matches stashed SHA at porting/customized.t line 112
# got "2a68815d963bf0d4822ce85291569054004aafea"
# expected "8498104713e5ce189602dda55dca38bee780c297"

not ok 1 - SHA for dist/Devel-PPPort/parts/embed.fnc matches stashed SHA
not ok 2 - SHA for cpan/Digest/Digest.pm matches stashed SHA
not ok 3 - SHA for cpan/Locale-Maketext-Simple/lib/Locale/Maketext/Simple.pm matches stashed SHA
not ok 4 - SHA for cpan/Math-Complex/lib/Math/Complex.pm matches stashed SHA
not ok 5 - SHA for cpan/Math-Complex/t/Complex.t matches stashed SHA
not ok 6 - SHA for cpan/Math-Complex/t/Trig.t matches stashed SHA
not ok 7 - SHA for cpan/Memoize/Memoize.pm matches stashed SHA
not ok 8 - SHA for cpan/NEXT/lib/NEXT.pm matches stashed SHA
not ok 9 - SHA for cpan/NEXT/t/next.t matches stashed SHA
not ok 10 - SHA for dist/Net-Ping/t/000_load.t matches stashed SHA
not ok 11 - SHA for dist/Net-Ping/t/001_new.t matches stashed SHA
not ok 12 - SHA for dist/Net-Ping/t/500_ping_icmp.t matches stashed SHA
not ok 13 - SHA for cpan/Pod-Checker/t/pod/contains_bad_pod.xr matches stashed SHA
not ok 14 - SHA for cpan/Pod-Checker/t/pod/selfcheck.t matches stashed SHA
not ok 15 - SHA for cpan/Pod-Checker/t/pod/testcmp.pl matches stashed SHA
not ok 16 - SHA for cpan/Pod-Checker/t/pod/testpchk.pl matches stashed SHA
not ok 17 - SHA for cpan/Pod-Perldoc/lib/Pod/Perldoc.pm matches stashed SHA
not ok 18 - SHA for cpan/autodie/lib/autodie/exception.pm matches stashed SHA
not ok 19 - SHA for cpan/autodie/lib/autodie/hints.pm matches stashed SHA
not ok 20 - SHA for cpan/autodie/t/exceptions.t matches stashed SHA
not ok 21 - SHA for cpan/autodie/t/lib/Hints_pod_examples.pm matches stashed SHA
not ok 22 - SHA for cpan/autodie/t/mkdir.t matches stashed SHA
not ok 23 - SHA for cpan/experimental/t/basic.t matches stashed SHA
not ok 24 - SHA for cpan/perlfaq/lib/perlfaq5.pod matches stashed SHA
not ok 25 - SHA for cpan/perlfaq/lib/perlfaq8.pod matches stashed SHA
not ok 26 - SHA for pod/perlpodstyle.pod matches stashed SHA
not ok 27 - SHA for cpan/version/lib/version.pm matches stashed SHA
not ok 28 - SHA for vxs.inc matches stashed SHA
1..28
Failed 28/28 subtests

Test Summary Report


porting/customized.t (Wstat​: 0 Tests​: 28 Failed​: 28)
  Failed tests​: 1-28
Files=1, Tests=28, 1 wallclock secs ( 0.03 usr + 0.00 sys = 0.03 CPU)
Result​: FAIL

c​:\perl-git-fresh\t>.\perl.exe harness -v porting/pod_rules.t
porting/pod_rules.t .. Porting/pod_rules.pl​: unix contains no copy rules at ./Porting/pod_lib.pl line 210.
  main​::verify_contiguous("unix", "#!/bin/sh\x{d}\x{a}\x{d}\x{a}# quote() - Creates a shell literal\x{d}\x{a}# Usa
ge​: e"..., qr((\x{a}pod/perl[a-z0-9_]+\.pod​: pod/perl[a-z0-9_]+\.pod\x{a}\x)...ms, "copy rules") called at Porting/pod_
rules.pl line 213
  main​::do_unix("unix", "#!/bin/sh\x{d}\x{a}\x{d}\x{a}# quote() - Creates a shell literal\x{d}\x{a}# Usage​: e"...
) called at ./Porting/pod_lib.pl line 265
  main​::process("unix", "Makefile.SH", CODE(0x215df28), 7, undef) called at Porting/pod_rules.pl line 227

1..8
ok 1 # got Pod metadata
not ok 2 # win32/makefile.mk is up to date
not ok 3 # win32/GNUmakefile is up to date
not ok 4 # MANIFEST is up to date
not ok 5 # win32/Makefile is up to date
not ok 6 # win32/pod.mak is up to date
Failed 7/8 subtests

Test Summary Report


porting/pod_rules.t (Wstat​: 0 Tests​: 6 Failed​: 5)
  Failed tests​: 2-6
  Parse errors​: Bad plan. You planned 8 tests but ran 6.
Files=1, Tests=6, 1 wallclock secs ( 0.03 usr + 0.00 sys = 0.03 CPU)
Result​: FAIL

c​:\perl-git-fresh\t>.\perl.exe harness -v porting/regen.t
porting/regen.t ..
1..43
# db0472e0ad4f44bd0816cad799d63b60d1bbd7e11cef40ea15bf0d00f69669f6 regen/keywords.pl
ok 1 - generated keywords.c is up to date
# db0472e0ad4f44bd0816cad799d63b60d1bbd7e11cef40ea15bf0d00f69669f6 regen/keywords.pl
ok 2 - generated keywords.h is up to date
# b35fd150a5540d0fe06540db67bc67864b14dd744550f105e13b78ef97a953d5 config_h.SH
# 02e3ab990e287cd32a5d000bc0b75168aa96496175b82315bdd47f898d00b0a6 uconfig.sh
ok 3 - generated uconfig.h is up to date
# 4d64b650346ff9ddbcd3986f0da13f3be2379d64c574a120ca7f534d8d17934c lib/Unicode/UCD.pm
# ff4404ec64f308bdf7714c50f9fdf0d1d0bf3c34db4d0a67e58ef0c6f88e818f lib/unicore/ArabicShaping.txt
# 292171a0a1c13d7e581e8781eb4cdf248243b1ab267354a63c7a14429dea2740 lib/unicore/BidiBrackets.txt
# 8f2695cc42989a79a715ab0d2892bd0c998759180cfdfb998674447f48231940 lib/unicore/BidiMirroring.txt
# 5ae1649a42ed8ae8cb885af79563f00a9ae17e602405a56ed8aca214da14eea7 lib/unicore/Blocks.txt
# 97b43ed3f4b80572c2114200e5e43a6b923d984c74a6caaf27de5b8462c04cb0 lib/unicore/CaseFolding.txt
# eedcf6932b4081ee65878454588c803af910a5aed7c8c67e3c38701cbee9b9e4 lib/unicore/CompositionExclusions.txt
# 90e48995643f45b08f0ef67fb90de3bb18e26765272bcc0c35c840cbc10b37c1 lib/unicore/DAge.txt
# e6ca152259189ec4bc2297e93c6c88f86e162cc52814198263497f3c9b46cbe9 lib/unicore/DCoreProperties.txt
# 3e255ccdff4b00cfe0be53bbd583e4fe2e7d4039138579543548a5ecfce45242 lib/unicore/DNormalizationProps.txt
# 9f34e2d3ea27ca82f4f14b62411861d1f07c4b9e296d54da112a09cad5b9a48d lib/unicore/EastAsianWidth.txt
# 983810c739b56b9ff0fcb5db018c67d584ee515e8a5f4d9348c79ee85167ec37 lib/unicore/HangulSyllableType.txt
# 7d514a779ee4baf91262bd83c400cb502c3e435dba4b880c876506be32b8e1d0 lib/unicore/IndicPositionalCategory.txt
# f6acead8f84df5c72f3fb70dfc9375279926e4d8ef3480ffff3723095e9804aa lib/unicore/IndicSyllabicCategory.txt
# c8ed526f70443535ca6b705980a08c774017ff17e921202dcb7b71ae554047b6 lib/unicore/Jamo.txt
# 6b204c3727b77699d04a574b22b1e44facab038642095b8565b49762970d9bf8 lib/unicore/LineBreak.txt
# f2357d2bd3526b9e830de72ab038dcfc65a2dff24bcb4c6325c92071eb341f88 lib/unicore/NameAliases.txt
# 6c3e6bd1e58b640076a23b83318a8bf6a691d7fc2b2106114d77c5c5a898bced lib/unicore/NamedSequences.txt
# 58dbf8fedbd5bf67a3bd5c10eda2f2acf2eae59df5f77884a6f158e98f75cf8c lib/unicore/PropList.txt
# a6b0467c3cc7aa4e57d4e5cc7f6e9562b79cf4426dfe438517c28b368ed3e673 lib/unicore/PropValueAliases.txt
# 9ca521224e08d30696516ae6bc3d4434659c45df16047c0d31e440783c163a3b lib/unicore/PropertyAliases.txt
# 3fd0d744a816ddfd06809f92151ae4a73ec970ac2006806c269732d8951f3911 lib/unicore/ScriptExtensions.txt
# d02e24e4c516e9090b6bc9c2d2c8f4c89510b6ed8c5e859d0a861b0dc5cf372d lib/unicore/Scripts.txt
# e9947a0e86f27353f0e776403c4826675001210bd39d7114118a8864a57f7472 lib/unicore/SpecialCasing.txt
# 52423e4d7492167b62f518f68d54db88930abbbff7f11edfcaec8f726498cab1 lib/unicore/UnicodeData.txt
# f28caf260635cebf25fd58124bdc9aa22af08ba4d039ffc584365fb41a31cda5 lib/unicore/VerticalOrientation.txt
# 718d174957712410bfad782b10d557e1047574d7ef1642d6bb122f8ca5662c82 lib/unicore/auxiliary/GCBTest.txt
# 3b66caefc4fa877d0e50dbbbfa39658c86e29b26c6f206f68d7aec192d4c59b6 lib/unicore/auxiliary/GraphemeBreakProperty.txt
# 2aad3836c37fc4c2fa2a24f21586fb3a931dacaf0a1c845a6dc6395f30bd79a7 lib/unicore/auxiliary/LBTest.txt
# eacc03e39dddc60cc59ec9bd274b8ac8dfa25d61745ef0d5c8aa1c151a5b68ba lib/unicore/auxiliary/SBTest.txt
# e45fa8195bb413b901cc1e3772dfea2cead86805d46a51e3480a5a256e8c24d9 lib/unicore/auxiliary/SentenceBreakProperty.txt
# 95789f62e3b1e781dc9ed78f3983d39ff1a5e36ff0b497d6e610446df902b0f6 lib/unicore/auxiliary/WBTest.txt
# c207e8ebd06ee591a27b1087f2971f4cd93e960103c453d85d1d9ba26fb8b202 lib/unicore/auxiliary/WordBreakProperty.txt
# a3c0839826a30166b2bb06ba58df403547b8c3d9eae995ef889d20d115f4b223 lib/unicore/extracted/DBidiClass.txt
# 280afe22f6c4d56566d17d6d1400f33465a979c96f3d99ff3bff9bd14d17e734 lib/unicore/extracted/DBinaryProperties.txt
# db7fd6a5e6f068c47dbc3b74fb633fb1d09d17073410fe435295d05ce925c5f6 lib/unicore/extracted/DCombiningClass.txt
# 8204c07a7c217bdf22525030ad7b4fb991edf463bffcca7e6dba46b9992e0d99 lib/unicore/extracted/DDecompositionType.txt
# e343113719b660bdd81217ec101ce751f844fca0e8d6f15fb21c8ee7dfe7c14c lib/unicore/extracted/DEastAsianWidth.txt
# 07c55b0ed7271fe1a5f4d68059291288b1a8ad61940602d18956fd87390c2d9e lib/unicore/extracted/DGeneralCategory.txt
# d788b9362ec7681e98f8b9d6ef276546e1a6207dda05317ede55bd686b0940a9 lib/unicore/extracted/DJoinGroup.txt
# ebbea3c93eeb7431378885aebac0490d77f6900239c9176f90b6fee030903d96 lib/unicore/extracted/DJoinType.txt
# be0f129691d479aa38646e4ca0ec1ee576ae7f75b0300a5624a7fa862fa8abba lib/unicore/extracted/DLineBreak.txt
# 92449d354d9f6b6f2f97a292ebb59f6344ffdeb83d120d7d23e569c43ba67cd5 lib/unicore/extracted/DNumType.txt
# e3a319527153b0c6c0c549b40fc6f3a01a7a0dcd6620784391db25901df3b154 lib/unicore/extracted/DNumValues.txt
# 5671c3de473b25e7ea47097e4906260624dfabe3e9b1739f490aecbc3d858459 lib/unicore/mktables
# 21653d2744fdd071f9ef138c805393901bb9547cf3e777ebf50215a191f986ea lib/unicore/version
# 913d2f93f3cb6cdf1664db888bf840bc4eb074eef824e082fceda24a9445e60c regen/charset_translations.pl
# 4898ec84e2b81e8bf948dcdb1c015c14f258cc652337122719885a276ea46d7b regen/mk_invlists.pl
ok 4 - generated charclass_invlists.h is up to date
# 4d64b650346ff9ddbcd3986f0da13f3be2379d64c574a120ca7f534d8d17934c lib/Unicode/UCD.pm
# ff4404ec64f308bdf7714c50f9fdf0d1d0bf3c34db4d0a67e58ef0c6f88e818f lib/unicore/ArabicShaping.txt
# 292171a0a1c13d7e581e8781eb4cdf248243b1ab267354a63c7a14429dea2740 lib/unicore/BidiBrackets.txt
# 8f2695cc42989a79a715ab0d2892bd0c998759180cfdfb998674447f48231940 lib/unicore/BidiMirroring.txt
# 5ae1649a42ed8ae8cb885af79563f00a9ae17e602405a56ed8aca214da14eea7 lib/unicore/Blocks.txt
# 97b43ed3f4b80572c2114200e5e43a6b923d984c74a6caaf27de5b8462c04cb0 lib/unicore/CaseFolding.txt
# eedcf6932b4081ee65878454588c803af910a5aed7c8c67e3c38701cbee9b9e4 lib/unicore/CompositionExclusions.txt
# 90e48995643f45b08f0ef67fb90de3bb18e26765272bcc0c35c840cbc10b37c1 lib/unicore/DAge.txt
# e6ca152259189ec4bc2297e93c6c88f86e162cc52814198263497f3c9b46cbe9 lib/unicore/DCoreProperties.txt
# 3e255ccdff4b00cfe0be53bbd583e4fe2e7d4039138579543548a5ecfce45242 lib/unicore/DNormalizationProps.txt
# 9f34e2d3ea27ca82f4f14b62411861d1f07c4b9e296d54da112a09cad5b9a48d lib/unicore/EastAsianWidth.txt
# 983810c739b56b9ff0fcb5db018c67d584ee515e8a5f4d9348c79ee85167ec37 lib/unicore/HangulSyllableType.txt
# 7d514a779ee4baf91262bd83c400cb502c3e435dba4b880c876506be32b8e1d0 lib/unicore/IndicPositionalCategory.txt
# f6acead8f84df5c72f3fb70dfc9375279926e4d8ef3480ffff3723095e9804aa lib/unicore/IndicSyllabicCategory.txt
# c8ed526f70443535ca6b705980a08c774017ff17e921202dcb7b71ae554047b6 lib/unicore/Jamo.txt
# 6b204c3727b77699d04a574b22b1e44facab038642095b8565b49762970d9bf8 lib/unicore/LineBreak.txt
# f2357d2bd3526b9e830de72ab038dcfc65a2dff24bcb4c6325c92071eb341f88 lib/unicore/NameAliases.txt
# 6c3e6bd1e58b640076a23b83318a8bf6a691d7fc2b2106114d77c5c5a898bced lib/unicore/NamedSequences.txt
# 58dbf8fedbd5bf67a3bd5c10eda2f2acf2eae59df5f77884a6f158e98f75cf8c lib/unicore/PropList.txt
# a6b0467c3cc7aa4e57d4e5cc7f6e9562b79cf4426dfe438517c28b368ed3e673 lib/unicore/PropValueAliases.txt
# 9ca521224e08d30696516ae6bc3d4434659c45df16047c0d31e440783c163a3b lib/unicore/PropertyAliases.txt
# 3fd0d744a816ddfd06809f92151ae4a73ec970ac2006806c269732d8951f3911 lib/unicore/ScriptExtensions.txt
# d02e24e4c516e9090b6bc9c2d2c8f4c89510b6ed8c5e859d0a861b0dc5cf372d lib/unicore/Scripts.txt
# e9947a0e86f27353f0e776403c4826675001210bd39d7114118a8864a57f7472 lib/unicore/SpecialCasing.txt
# 52423e4d7492167b62f518f68d54db88930abbbff7f11edfcaec8f726498cab1 lib/unicore/UnicodeData.txt
# f28caf260635cebf25fd58124bdc9aa22af08ba4d039ffc584365fb41a31cda5 lib/unicore/VerticalOrientation.txt
# 718d174957712410bfad782b10d557e1047574d7ef1642d6bb122f8ca5662c82 lib/unicore/auxiliary/GCBTest.txt
# 3b66caefc4fa877d0e50dbbbfa39658c86e29b26c6f206f68d7aec192d4c59b6 lib/unicore/auxiliary/GraphemeBreakProperty.txt
# 2aad3836c37fc4c2fa2a24f21586fb3a931dacaf0a1c845a6dc6395f30bd79a7 lib/unicore/auxiliary/LBTest.txt
# eacc03e39dddc60cc59ec9bd274b8ac8dfa25d61745ef0d5c8aa1c151a5b68ba lib/unicore/auxiliary/SBTest.txt
# e45fa8195bb413b901cc1e3772dfea2cead86805d46a51e3480a5a256e8c24d9 lib/unicore/auxiliary/SentenceBreakProperty.txt
# 95789f62e3b1e781dc9ed78f3983d39ff1a5e36ff0b497d6e610446df902b0f6 lib/unicore/auxiliary/WBTest.txt
# c207e8ebd06ee591a27b1087f2971f4cd93e960103c453d85d1d9ba26fb8b202 lib/unicore/auxiliary/WordBreakProperty.txt
# a3c0839826a30166b2bb06ba58df403547b8c3d9eae995ef889d20d115f4b223 lib/unicore/extracted/DBidiClass.txt
# 280afe22f6c4d56566d17d6d1400f33465a979c96f3d99ff3bff9bd14d17e734 lib/unicore/extracted/DBinaryProperties.txt
# db7fd6a5e6f068c47dbc3b74fb633fb1d09d17073410fe435295d05ce925c5f6 lib/unicore/extracted/DCombiningClass.txt
# 8204c07a7c217bdf22525030ad7b4fb991edf463bffcca7e6dba46b9992e0d99 lib/unicore/extracted/DDecompositionType.txt
# e343113719b660bdd81217ec101ce751f844fca0e8d6f15fb21c8ee7dfe7c14c lib/unicore/extracted/DEastAsianWidth.txt
# 07c55b0ed7271fe1a5f4d68059291288b1a8ad61940602d18956fd87390c2d9e lib/unicore/extracted/DGeneralCategory.txt
# d788b9362ec7681e98f8b9d6ef276546e1a6207dda05317ede55bd686b0940a9 lib/unicore/extracted/DJoinGroup.txt
# ebbea3c93eeb7431378885aebac0490d77f6900239c9176f90b6fee030903d96 lib/unicore/extracted/DJoinType.txt
# be0f129691d479aa38646e4ca0ec1ee576ae7f75b0300a5624a7fa862fa8abba lib/unicore/extracted/DLineBreak.txt
# 92449d354d9f6b6f2f97a292ebb59f6344ffdeb83d120d7d23e569c43ba67cd5 lib/unicore/extracted/DNumType.txt
# e3a319527153b0c6c0c549b40fc6f3a01a7a0dcd6620784391db25901df3b154 lib/unicore/extracted/DNumValues.txt
# 5671c3de473b25e7ea47097e4906260624dfabe3e9b1739f490aecbc3d858459 lib/unicore/mktables
# 21653d2744fdd071f9ef138c805393901bb9547cf3e777ebf50215a191f986ea lib/unicore/version
# 913d2f93f3cb6cdf1664db888bf840bc4eb074eef824e082fceda24a9445e60c regen/charset_translations.pl
# 9ea6338945a7d70e5ea4b31ac7856c0b521df96be002e94b4b3b7d31debbf3ab regen/regcharclass.pl
# 393f8d882713a3ba227351ad0f00ea4839fda74fcf77dcd1cdf31519925adba5 regen/regcharclass_multi_char_folds.pl
ok 5 - generated regcharclass.h is up to date
# c85e1793baa49bfdf6f1329f04fd8b57a616cfc2f5dce01702d3d727f6511157 perly.y
# b6fae5748f9bef6db4740aa5e122b84ac5181852d42474d0ecad621fa4253306 regen_perly.pl
ok 6 - generated perly.act is up to date
# c85e1793baa49bfdf6f1329f04fd8b57a616cfc2f5dce01702d3d727f6511157 perly.y
# b6fae5748f9bef6db4740aa5e122b84ac5181852d42474d0ecad621fa4253306 regen_perly.pl
ok 7 - generated perly.h is up to date
# c85e1793baa49bfdf6f1329f04fd8b57a616cfc2f5dce01702d3d727f6511157 perly.y
# b6fae5748f9bef6db4740aa5e122b84ac5181852d42474d0ecad621fa4253306 regen_perly.pl
ok 8 - generated perly.tab is up to date
ok - regen/ebcdic.pl ebcdic_tables.h
ok - regen/genpacksizetables.pl packsizetables.inc
regen/lib_cleanup.pl​: win32/Makefile contains no Win32 lib directory rmdir rules at ./Porting/pod_lib.pl line 210.
  main​::verify_contiguous("win32/Makefile", "#\x{d}\x{a}# Makefile to build perl on Windows using Microsoft NMAKE.
"..., qr(\\x{9}\-del\ \/f\ \*\.def\ \*\.map\n(?​:\t-if exist (\$\(LIBDI)...ms, "Win32 lib directory rmdir rules") called
at regen/lib_cleanup.pl line 138
  main​::edit_win32_makefile("win32/Makefile", "#\x{d}\x{a}# Makefile to build perl on Windows using Microsoft NMAK
E."...) called at ./Porting/pod_lib.pl line 265
  main​::process("win32/Makefile", "win32/Makefile", CODE(0x31c4e0), "", 0) called at regen/lib_cleanup.pl line 157

not ok # Makefile.SH is up to date
Failed to run C​:\perl-git-fresh\t\perl.exe -I. regen/lib_cleanup.pl --tap​: 65280 at porting/regen.t line 108.
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 33/43 subtests

Test Summary Report


porting/regen.t (Wstat​: 65280 Tests​: 11 Failed​: 1)
  Failed test​: 11
  Non-zero exit status​: 255
  Parse errors​: Bad plan. You planned 43 tests but ran 11.
Files=1, Tests=11, 1 wallclock secs ( 0.01 usr + 0.00 sys = 0.01 CPU)
Result​: FAIL

c​:\perl-git-fresh\t>.\perl.exe harness -v ../cpan/CPAN-Meta-YAML/t/30_yaml_spec_tml.t
../cpan/CPAN-Meta-YAML/t/30_yaml_spec_tml.t .. Invalid TestML line​:
' at t/lib/TestML/Tiny.pm line 281.
Dubious, test returned 2 (wstat 512, 0x200)
No subtests run

Test Summary Report


../cpan/CPAN-Meta-YAML/t/30_yaml_spec_tml.t (Wstat​: 512 Tests​: 0 Failed​: 0)
  Non-zero exit status​: 2
  Parse errors​: No plan found in TAP output
Files=1, Tests=0, 0 wallclock secs ( 0.00 usr + 0.01 sys = 0.01 CPU)
Result​: FAIL

../cpan/IPC-Cmd/t/01_IPC-Cmd.t indeed passes when run as a separate test file

c​:\perl-git-fresh\t>.\perl.exe harness -v ../cpan/podlators/t/text/encoding.t
../cpan/podlators/t/text/encoding.t ..
1..7
ok 1 - use Pod​::Text;
# Looks like you planned 7 tests but ran 1.
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 6/7 subtests

Test Summary Report


../cpan/podlators/t/text/encoding.t (Wstat​: 65280 Tests​: 1 Failed​: 0)
  Non-zero exit status​: 255
  Parse errors​: Bad plan. You planned 7 tests but ran 1.
Files=1, Tests=1, 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU)
Result​: FAIL

../ext/IPC-Open3/t/IPC-Open3.t also passes when run as a separate test file

And here's the -V output for this build​:

c​:\perl-git-fresh>.\perl.exe -Ilib -V
Summary of my perl5 (revision 5 version 27 subversion 9) configuration​:
  Derived from​: 6d37ab4
  Platform​:
  osname=MSWin32
  osvers=6.1.7601
  archname=MSWin32-x64-multi-thread
  uname=''
  config_args='undef'
  hint=recommended
  useposix=true
  d_sigaction=undef
  useithreads=define
  usemultiplicity=define
  use64bitint=define
  use64bitall=undef
  uselongdouble=undef
  usemymalloc=n
  default_inc_excludes_dot=define
  bincompat5005=undef
  Compiler​:
  cc='cl'
  ccflags ='-nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -fp​:precise -DWIN32 -D_CONSOLE -DNO_STRICT -DWIN64 -DCONSERVATIVE
-D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -D_WINSOCK_DEPRECATED_NO_WARNINGS -DPERL_TEXTMODE_SCRIPTS -DPER
L_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS'
  optimize='-O1 -MD -Zi -DNDEBUG -GL -fp​:precise'
  cppflags='-DWIN32'
  ccversion='19.10.25017'
  gccversion=''
  gccosandvers=''
  intsize=4
  longsize=4
  ptrsize=8
  doublesize=8
  byteorder=12345678
  doublekind=3
  d_longlong=undef
  longlongsize=8
  d_longdbl=define
  longdblsize=8
  longdblkind=0
  ivtype='__int64'
  ivsize=8
  nvtype='double'
  nvsize=8
  Off_t='__int64'
  lseeksize=8
  alignbytes=8
  prototype=define
  Linker and Libraries​:
  ld='link'
  ldflags ='-nologo -nodefaultlib -debug -opt​:ref,icf -ltcg -libpath​:"c​:\perl-blead\lib\CORE" -machine​:AMD64'
  libpth="C​:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.10.25017\\lib\x64"
  libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib o
eaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt
lib vcruntime.lib ucrt.lib
  perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.l
b oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib ms
crt.lib vcruntime.lib ucrt.lib
  libc=ucrt.lib
  so=dll
  useshrplib=true
  libperl=perl527.lib
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_win32.xs
  dlext=dll
  d_dlsymun=undef
  ccdlflags=' '
  cccdlflags=' '
  lddlflags='-dll -nologo -nodefaultlib -debug -opt​:ref,icf -ltcg -libpath​:"c​:\perl-blead\lib\CORE" -machine​:AMD64'

Characteristics of this binary (from libperl)​:
  Compile-time options​:
  HAS_TIMES
  HAVE_INTERP_INTERN
  MULTIPLICITY
  PERLIO_LAYERS
  PERL_COPY_ON_WRITE
  PERL_DONT_CREATE_GVSV
  PERL_IMPLICIT_CONTEXT
  PERL_IMPLICIT_SYS
  PERL_MALLOC_WRAP
  PERL_OP_PARENT
  PERL_PRESERVE_IVUV
  USE_64_BIT_INT
  USE_ITHREADS
  USE_LARGE_FILES
  USE_LOCALE
  USE_LOCALE_COLLATE
  USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC
  USE_LOCALE_TIME
  USE_PERLIO
  USE_PERL_ATOF
  Locally applied patches​:
  uncommitted-changes
  Built under MSWin32
  Compiled at Feb 19 2018 22​:28​:55
  @​INC​:
  lib
  c​:/perl-git-fresh/lib

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 19, 2018

From @craigberry

On Mon, Feb 19, 2018 at 2​:21 PM, Sergey Aleynikov via RT
<perlbug-followup@​perl.org> wrote​:

On Mon, 19 Feb 2018 01​:11​:35 -0800, shay wrote​:

Is this run from a clean git workspace? I'm trying to picture how
something like porting/customized.t could possibly fail. Do you have
any modified/staged/committed files?

So, here's a run from a clean checkout with the following config applied​:

What line ending policy do you have set in your git repository? I
think the recommendation is to check out with CRLF endings and push
with Unix-style endings (maybe someone can confirm that?). I ask
because things that checksum file contents seem to be getting the
wrong answer.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 19, 2018

From @dur-randir

On Mon, 19 Feb 2018 12​:41​:35 -0800, craig.a.berry@​gmail.com wrote​:

What line ending policy do you have set in your git repository? I
think the recommendation is to check out with CRLF endings and push
with Unix-style endings (maybe someone can confirm that?). I ask
because things that checksum file contents seem to be getting the
wrong answer.

All build have been done with core.autocrlf set to '' (default). I've tries another one with 'true', it changed nothing.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 20, 2018

From @steve-m-hay

On 19 February 2018 at 21​:39, Sergey Aleynikov via RT
<perlbug-followup@​perl.org> wrote​:

On Mon, 19 Feb 2018 12​:41​:35 -0800, craig.a.berry@​gmail.com wrote​:

What line ending policy do you have set in your git repository? I
think the recommendation is to check out with CRLF endings and push
with Unix-style endings (maybe someone can confirm that?). I ask
because things that checksum file contents seem to be getting the
wrong answer.

All build have been done with core.autocrlf set to '' (default). I've tries another one with 'true', it changed nothing.

I agree with Craig it's highly likely to be EOL problems - that's
surely the only reason that all those porting tests could be failing.

My .gitconfig file contains

[core]
  autocrlf = false

and all the files in my workspace have UNIX-style EOLs.

Make a new workspace like that and try again?

Btw, what is the reason that you change the 'ar' entry in config.vc? I
never do that (though it would be nothing to do with the porting test
failures, obviously). Is that because of a limitation in the Community
Edition of Visual Studio?

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 20, 2018

From @demerphq

On 19 Feb 2018 19​:54, "demerphq" <demerphq@​gmail.com> wrote​:

On 19 Feb 2018 08​:07, "Sergey Aleynikov via RT" <perlbug-followup@​perl.org>
wrote​:

Here's what I've got with a x64 build using MSVC 2017.

Build warnings part​:

c​:\perl-git\zaphod32_hash.h(221)​: warning C4267​: 'initializing'​: conversion
from 'size_t' to 'U32', possible loss of data

Fwiw, this is unexpected to me. Zaphod32 should not by default be used on
64 bit builds. There would appear to be a failure in our detection.

I don't have the code available to check myself, but it would be nice to
fix whatever is going wrong in hv_func.h so that you use stadtx64, possibly
with sbox hash, instead. The warning I think is from this.

Steve while it's not the main focus of this ticket is there any chance you
could poke into why it's not using Stadtx64?

Yves

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 20, 2018

From @steve-m-hay

On 20 February 2018 at 10​:55, demerphq <demerphq@​gmail.com> wrote​:

On 19 Feb 2018 19​:54, "demerphq" <demerphq@​gmail.com> wrote​:

On 19 Feb 2018 08​:07, "Sergey Aleynikov via RT" <perlbug-followup@​perl.org>
wrote​:

Here's what I've got with a x64 build using MSVC 2017.

Build warnings part​:

c​:\perl-git\zaphod32_hash.h(221)​: warning C4267​: 'initializing'​: conversion
from 'size_t' to 'U32', possible loss of data

Fwiw, this is unexpected to me. Zaphod32 should not by default be used on 64
bit builds. There would appear to be a failure in our detection.

I don't have the code available to check myself, but it would be nice to fix
whatever is going wrong in hv_func.h so that you use stadtx64, possibly with
sbox hash, instead. The warning I think is from this.

Steve while it's not the main focus of this ticket is there any chance you
could poke into why it's not using Stadtx64?

hv_func.h correctly chooses STATDX, but it includes sbox32_hash.h just
before the choice is made. That includes zaphod32_hash.h, hence the
warning appears even though we aren't using that algorithm.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 20, 2018

From @demerphq

On 20 Feb 2018 21​:39, "Steve Hay" <steve.m.hay@​googlemail.com> wrote​:

On 20 February 2018 at 10​:55, demerphq <demerphq@​gmail.com> wrote​:

On 19 Feb 2018 19​:54, "demerphq" <demerphq@​gmail.com> wrote​:

On 19 Feb 2018 08​:07, "Sergey Aleynikov via RT" <perlbug-followup@​perl.org

wrote​:

Here's what I've got with a x64 build using MSVC 2017.

Build warnings part​:

c​:\perl-git\zaphod32_hash.h(221)​: warning C4267​: 'initializing'​:
conversion
from 'size_t' to 'U32', possible loss of data

Fwiw, this is unexpected to me. Zaphod32 should not by default be used on
64
bit builds. There would appear to be a failure in our detection.

I don't have the code available to check myself, but it would be nice to
fix
whatever is going wrong in hv_func.h so that you use stadtx64, possibly
with
sbox hash, instead. The warning I think is from this.

Steve while it's not the main focus of this ticket is there any chance you
could poke into why it's not using Stadtx64?

hv_func.h correctly chooses STATDX, but it includes sbox32_hash.h just
before the choice is made. That includes zaphod32_hash.h, hence the
warning appears even though we aren't using that algorithm.

I see. Ok, that explains it for sure. I'll see if I can figure out how to
silence the warning.

Thanks a lot, much obliged to you for digging.

Yves

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 20, 2018

From @dur-randir

On Tue, 20 Feb 2018 00​:06​:57 -0800, shay wrote​:

I agree with Craig it's highly likely to be EOL problems - that's
surely the only reason that all those porting tests could be failing.
Make a new workspace like that and try again?

Explicitly setting core.autocrlf to 'false' instead of the default '' helped indeed. Thank you.

Btw, what is the reason that you change the 'ar' entry in config.vc? I
never do that (though it would be nothing to do with the porting test
failures, obviously). Is that because of a limitation in the Community
Edition of Visual Studio?

There're some words about it in the readme, so I've changed it to what it suggests (maybe for some other compiler version?). I've tested a build without that change and it succeeded.

So, now there're only IPC-under-harness failures and a lot of build warnings. Should I report them as a separate tickets for each on of the dist/ and ext/ modules, or this ticket is sufficient? As I've already reported them for all cpan/ ones.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 21, 2018

From @steve-m-hay

On 20 February 2018 at 22​:23, Sergey Aleynikov via RT
<perlbug-followup@​perl.org> wrote​:

On Tue, 20 Feb 2018 00​:06​:57 -0800, shay wrote​:

I agree with Craig it's highly likely to be EOL problems - that's
surely the only reason that all those porting tests could be failing.
Make a new workspace like that and try again?

Explicitly setting core.autocrlf to 'false' instead of the default '' helped indeed. Thank you.

Btw, what is the reason that you change the 'ar' entry in config.vc? I
never do that (though it would be nothing to do with the porting test
failures, obviously). Is that because of a limitation in the Community
Edition of Visual Studio?

There're some words about it in the readme, so I've changed it to what it suggests (maybe for some other compiler version?). I've tested a build without that change and it succeeded.

That's in the section talking specifically about Microsoft Visual C++
Toolkit 2003. It shouldn't be necessary for any other version.

So, now there're only IPC-under-harness failures and a lot of build warnings. Should I report them as a separate tickets for each on of the dist/ and ext/ modules, or this ticket is sufficient? As I've already reported them for all cpan/ ones.

I think this ticket is sufficient since dist/ and ext/ are maintained
here by p5p anyway.

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 21, 2018

From @bulk88

On Tue, 20 Feb 2018 06​:45​:46 -0800, demerphq wrote​:

hv_func.h correctly chooses STATDX, but it includes sbox32_hash.h just
before the choice is made. That includes zaphod32_hash.h, hence the
warning appears even though we aren't using that algorithm.

I see. Ok, that explains it for sure. I'll see if I can figure out how to
silence the warning.

Thanks a lot, much obliged to you for digging.

Yves

Every Win64 .o/XS module produces this warning described above, and stable perl can't ship if all XS module produce CC warnings in perl headers.


cl -c -I. -nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -fp​:precise -DWIN32 -D_
CONSOLE -DNO_STRICT -DWIN64 -DCONSERVATIVE -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NON
STDC_NO_DEPRECATE -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICI
T_SYS -DWIN32_NO_REGISTRY -O1 -MD -Zi -DNDEBUG -GL -fp​:precise -DVERSION=\"2.
074\" -DXS_VERSION=\"2.074\" "-I..\..\lib\CORE" -DBZ_NO_STDIO Bzip2.c
Bzip2.c
c​:\p527\srcnew\lib\core\zaphod32_hash.h(221) : warning C4267​: 'initializing' : c
onversion from 'size_t' to 'U32', possible loss of data
Bzip2.xs(488) : warning C4267​: '=' : conversion from 'size_t' to 'unsigned int',
possible loss of data


.i file output of problem line


static __inline U32
zaphod32_hash_with_state(const U8 * state_ch,
  const U8 * key, const STRLEN key_len)
{
  U32 *state = (U32 *) state_ch;
  const U8 *end;
  STRLEN len = key_len;
  U32 v0 = state[0];
  U32 v1 = state[1];
  U32 v2 = state[2] ^ (0xC41A7AB1 * (key_len + 1));<<<<<<PROBLEM<<<<<<<<<<<

  ;
#line 226 "c​:\\p527\\srcnew\\lib\\core\\zaphod32_hash.h"


Since this is Win64 key_len is 64 bit int, that is XORed with state[2] which is a U32 and the result is assigned to a U32, truncating high bytes. Hence the warning. It seems zaphod32_hash_with_state code is unique to you (demerphq) and not FOSS code borrowed from somewhere else, so there is no upstream to go consult about the truncation.

--
bulk88 ~ bulk88 at hotmail.com

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 21, 2018

From @bulk88

On Wed, 21 Feb 2018 09​:05​:19 -0800, bulk88 wrote​:

Since this is Win64 key_len is 64 bit int, that is XORed with state[2]
which is a U32 and the result is assigned to a U32, truncating high
bytes. Hence the warning. It seems zaphod32_hash_with_state code is
unique to you (demerphq) and not FOSS code borrowed from somewhere
else, so there is no upstream to go consult about the truncation.

I see a bigger problem with the new hash code. I put a __builtin_trap() right before "U32 v2 = state[2] ^ (0xC41A7AB1 * (key_len + 1));" and did a "make test". It NEVER executed. I checked the machine code, and yes zaphod32_hash_with_state/__builtin_trap is linked in and I can see how theoretically executed, but it is unreachable in real life.

zaphod32_hash_with_state is ONLY referenced by sbox32_hash_with_state.


static __inline U32
sbox32_hash_with_state(const U8 * state_ch,
  const U8 * key, const STRLEN key_len)
{
  U32 *state = (U32 *) state_ch;
  U32 hash = *state;
  switch (key_len) {
  default​:
  return zaphod32_hash_with_state(state_ch, key, key_len);

  case 24​:


That is the only reference.

But if I look at the callers of sbox32_hash_with_state, there are 18 callers, but all of then do the len <= 24 check. Example of usage.


static __inline U32
S_perl_hash_with_seed(const U8 * const seed, const U8 * const str,
  const STRLEN len)
{
  U8 state[(32 + ((1 + (256 * 24)) * sizeof(U32)))];
  do {
  stadtx_seed_state(seed, state);
  sbox32_seed_state96(seed + 16, state + 32);
  } while (0);
  return ((((len <=
  24) ? (char) 1 : (char) 0)) ? sbox32_hash_with_state((state
  + 32),
  (U8
  *)
  (str),
  (len))
  : (U32) stadtx_hash_with_state(((state)), (U8 *) ((str)),
  ((len))));
}


So sbox32_hash_with_state can't be reached with a key longer than 24, yet sbox32_hash_with_state has a default​: for a key longer than 24. The default​: in sbox32_hash_with_state needs to be "NOT_REACHED;" and maybe an assert. That would remove zaphod32_hash_with_state() from the binary since on 64 bit OSes stadtx is used, not zaphod for long keys.

I also think there is too much code not ifdefed out in hv_func.h

sbox32_hash128 has no uses, it should be #if 0'ed so it doesn't reach the C->Asm stage. sbox32_seed_state128 whose only ref is sbox32_hash128 also can #if 0, since S_perl_hash_with_seed uses sbox32_seed_state96. sbox32_hash96 can also be if #0.

S_perl_hash_siphash_2_4
S_perl_hash_siphash_2_4_with_state
S_perl_hash_siphash_1_3
S_perl_hash_siphash_1_3_with_state
S_perl_siphash_seed_state

are also in the .i file event though perl is uses the PERL_HASH_USE_SBOX32_ALSO sbox/stadtx or sbox/zaphos pair. Those need ifdefs too.

--
bulk88 ~ bulk88 at hotmail.com

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 22, 2018

From @demerphq

On 22 Feb 2018 03​:04, "bulk88 via RT" <perlbug-followup@​perl.org> wrote​:

On Wed, 21 Feb 2018 09​:05​:19 -0800, bulk88 wrote​:

Since this is Win64 key_len is 64 bit int, that is XORed with state[2]
which is a U32 and the result is assigned to a U32, truncating high
bytes. Hence the warning. It seems zaphod32_hash_with_state code is
unique to you (demerphq) and not FOSS code borrowed from somewhere
else, so there is no upstream to go consult about the truncation.

It is Foss, but it is also my work.😀

The truncation shouldn't matter, but it shouldn't throw an error either.

I see a bigger problem with the new hash code. I put a __builtin_trap()
right before "U32 v2 = state[2] ^ (0xC41A7AB1 * (key_len + 1));" and did a
"make test". It NEVER executed. I checked the machine code, and yes
zaphod32_hash_with_state/__builtin_trap is linked in and I can see how
theoretically executed, but it is unreachable in real life.

This is a build option issue. Sbox hashing is very fast, but has a finite
capacity, and falls through to zaphod32 when that capacity is exceeded.

However with the way that we use sbox32 in perl we will do a similar fall
through logic *before* we enter sbox32, but on 64 bit builds we will use
stadtx64 instead of zaphod32.

zaphod32_hash_with_state is ONLY referenced by sbox32_hash_with_state.


static __inline U32
sbox32_hash_with_state(const U8 * state_ch,
  const U8 * key, const STRLEN key_len)
{
  U32 *state = (U32 *) state_ch;
  U32 hash = *state;
  switch (key_len) {
  default​:
  return zaphod32_hash_with_state(state_ch, key, key_len);

  case 24​:


That is the only reference.

But if I look at the callers of sbox32_hash_with_state, there are 18
callers, but all of then do the len <= 24 check. Example of usage.

Yep. You could change when this happens with the right -D to configure.


static __inline U32
S_perl_hash_with_seed(const U8 * const seed, const U8 * const str,
  const STRLEN len)
{
  U8 state[(32 + ((1 + (256 * 24)) * sizeof(U32)))];
  do {
  stadtx_seed_state(seed, state);
  sbox32_seed_state96(seed + 16, state + 32);
  } while (0);
  return ((((len <=
  24) ? (char) 1 : (char) 0)) ? sbox32_hash_with_state((state
  + 32),
  (U8
  *)
  (str),
  (len))
  : (U32) stadtx_hash_with_state(((state)), (U8 *) ((str)),
  ((len))));
}


So sbox32_hash_with_state can't be reached with a key longer than 24, yet
sbox32_hash_with_state has a default​: for a key longer than 24. The
default​: in sbox32_hash_with_state needs to be "NOT_REACHED;" and maybe an
assert. That would remove zaphod32_hash_with_state() from the binary since
on 64 bit OSes stadtx is used, not zaphod for long keys.

Yep. Will do.

I also think there is too much code not ifdefed out in hv_func.h

sbox32_hash128 has no uses, it should be #if 0'ed so it doesn't reach the
C->Asm stage. sbox32_seed_state128 whose only ref is sbox32_hash128 also
can #if 0, since S_perl_hash_with_seed uses sbox32_seed_state96.
sbox32_hash96 can also be if #0.

S_perl_hash_siphash_2_4
S_perl_hash_siphash_2_4_with_state
S_perl_hash_siphash_1_3
S_perl_hash_siphash_1_3_with_state
S_perl_siphash_seed_state

are also in the .i file event though perl is uses the
PERL_HASH_USE_SBOX32_ALSO sbox/stadtx or sbox/zaphos pair. Those need
ifdefs too.

I assumed a good compiler would elide this unused code...

I'll dig further when I get a chance.

Thanks,
Yves

--
bulk88 ~ bulk88 at hotmail.com


via perlbug​: queue​: perl5 status​: open
https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132860

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 22, 2018

From @bulk88

On Wed, 21 Feb 2018 19​:04​:12 -0800, demerphq wrote​:

On 22 Feb 2018 03​:04, "bulk88 via RT" <perlbug-followup@​perl.org> wrote​:

On Wed, 21 Feb 2018 09​:05​:19 -0800, bulk88 wrote​:

Since this is Win64 key_len is 64 bit int, that is XORed with state[2]
which is a U32 and the result is assigned to a U32, truncating high
bytes. Hence the warning. It seems zaphod32_hash_with_state code is
unique to you (demerphq) and not FOSS code borrowed from somewhere
else, so there is no upstream to go consult about the truncation.

It is Foss, but it is also my work.😀

The truncation shouldn't matter, but it shouldn't throw an error either.

Cast to U32 before assigning to U32 will quiet MSVC.

However with the way that we use sbox32 in perl we will do a similar fall
through logic *before* we enter sbox32, but on 64 bit builds we will use
stadtx64 instead of zaphod32.

If you dont do the NOT_REACHED (I prefer that) more macros are needed so the inside of sbox32 matches the primary long key algo. Or have the primary long key algo only from the inside and make it so there is no way to use sbox32 for long keys.

Yep. You could change when this happens with the right -D to configure.

Probably nobody ever will, but still, if the fallback is from inside sbox, then the fallback should only be from sbox, not from the outer _PERL_HASH_WITH_STATE. Basically sbox CANT be compiled to be used in isolation.

Yep. Will do.

Yes please.

I assumed a good compiler would elide this unused code...

A good linker will toss the code, a compiler CANT toss the code and had to spend CPU to generate ASM. Really bad linkers/bad OSes wont toss the code due to GOT/"debugging" reasons (call a static func explicitly with gdb).

I'll dig further when I get a chance.

A U32 cast is the quick and dirty way to fix it aslong as truncation is okay of that XOR, truncation is happening anyway on all 64b OSes whether a CC warns or not.

--
bulk88 ~ bulk88 at hotmail.com

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 24, 2018

From @demerphq

On 22 February 2018 at 06​:34, bulk88 via RT <perlbug-followup@​perl.org> wrote​:

On Wed, 21 Feb 2018 19​:04​:12 -0800, demerphq wrote​:

On 22 Feb 2018 03​:04, "bulk88 via RT" <perlbug-followup@​perl.org> wrote​:

On Wed, 21 Feb 2018 09​:05​:19 -0800, bulk88 wrote​:

Since this is Win64 key_len is 64 bit int, that is XORed with state[2]
which is a U32 and the result is assigned to a U32, truncating high
bytes. Hence the warning. It seems zaphod32_hash_with_state code is
unique to you (demerphq) and not FOSS code borrowed from somewhere
else, so there is no upstream to go consult about the truncation.

It is Foss, but it is also my work.😀

The truncation shouldn't matter, but it shouldn't throw an error either.

Cast to U32 before assigning to U32 will quiet MSVC.

Done in​: e5a5512

However with the way that we use sbox32 in perl we will do a similar fall
through logic *before* we enter sbox32, but on 64 bit builds we will use
stadtx64 instead of zaphod32.

If you dont do the NOT_REACHED (I prefer that) more macros are needed so the inside of sbox32 matches the primary long key algo. Or have the primary long key algo only from the inside and make it so there is no way to use sbox32 for long keys.

Yep. You could change when this happens with the right -D to configure.

Probably nobody ever will, but still, if the fallback is from inside sbox, then the fallback should only be from sbox, not from the outer _PERL_HASH_WITH_STATE. Basically sbox CANT be compiled to be used in isolation.

Er...

That isnt clear to me, but i trust you. I will look into reversing
this somehow. For now the patch above will have to suffice.

Yep. Will do.

Yes please.

I assumed a good compiler would elide this unused code...

A good linker will toss the code, a compiler CANT toss the code and had to spend CPU to generate ASM. Really bad linkers/bad OSes wont toss the code due to GOT/"debugging" reasons (call a static func explicitly with gdb).

I'll dig further when I get a chance.

A U32 cast is the quick and dirty way to fix it aslong as truncation is okay of that XOR, truncation is happening anyway on all 64b OSes whether a CC warns or not.

Yeah, i did the cast. I will look into a more graceful fix another day.

Thanks for you help, i will come back to you when i have further patches.

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 25, 2018

From @bulk88

On Mon, 12 Feb 2018 12​:47​:07 -0800, randir wrote​:

- perlwin32.html (aka README.win32) states "There should be no test
failures.", but at least three tests failed for 64-bit build (console
window is by default too small, so I haven't recorded them) and I
ended up with cpan/IPC-Cmd/t/01_IPC-Cmd.t stuck (for at least 10
minutes, until I've aborted it). Are those failures known? If not,
should I report them, or there're not enough volunteers to fix them,
so it's time to change the docs?

cpan/IPC-Cmd/t/01_IPC-Cmd.t hangs/failures with Windows 7 (I dont see it on XP) random/race failures are caused by perl calling closesocket() on non-sockets I've talked about it at https://rt-archive.perl.org/perl5/Ticket/Display.html?id=118127

Win 7-32bit callstack of frozen with under-harness 01_IPC-Cmd.t, perl's curcop is at https://perl5.git.perl.org/perl.git/blob/c4a2ac437ed67b458e466f3724f82c10afc3eb40:/dist/IO/lib/IO/Handle.pm#l379


  ntdll.dll!_KiFastSystemCallRet@​0�() Unknown
  ntdll.dll!_ZwDeviceIoControlFile@​40�() Unknown
  mswsock.dll!_SockImportHandle@​12�() Unknown
  mswsock.dll!_SockFindAndReferenceSocket@​8�() Unknown
  mswsock.dll!_WSPGetSockOpt@​24�() Unknown
  ws2_32.dll!DCATALOG​::FindIFSProviderForSocket(unsigned int) Unknown
  ws2_32.dll!DSOCKET​::FindIFSSocket(unsigned int) Unknown
  ws2_32.dll!_closesocket@​4�() Unknown
  perl527.dll!my_close(int fd) Line 696 C
  perl527.dll!PerlIOUnix_close(interpreter * my_perl=0x000f2e8c, _PerlIO * * f=0x024441dc) Line 2816 C
  perl527.dll!PerlIOBase_close(interpreter * my_perl=0x000f2e8c, _PerlIO * * f=0x00105c24) Line 2132 C
  perl527.dll!PerlIOBuf_close(interpreter * my_perl=0x000f2e8c, _PerlIO * * f=0x00105c24) Line 4257 C
  perl527.dll!Perl_PerlIO_close(interpreter * my_perl=0x000f2e8c, _PerlIO * * f=0x00105c24) Line 1373 C
  perl527.dll!S_openn_cleanup(interpreter * my_perl, gv * gv, io * io=0x0010f7b4, _PerlIO * * fp=0x00105c24, char * mode=0x0031f698, const char * oname=0x02361f2c, _PerlIO * * saveifp=0x001059b4, _PerlIO * * saveofp=0x00105c0c, int savefd=0x00000002, char savetype='s', int writing=0x00000001, char was_fdopen='\0', const char * type=0x00000000, _stat64 * statbufp=0x00000000) Line 1024 C
  perl527.dll!Perl_do_open6(interpreter * my_perl, gv * gv, const char * oname=0x02361f2c, unsigned int len=0x00000013, _PerlIO * * supplied_fp=0x00000000, sv * * svp=0x00ab1580, unsigned long num_svs=0x00000000) Line 876 C
  perl527.dll!Perl_pp_open(interpreter * my_perl=0x00000000) Line 640 C
  perl527.dll!Perl_runops_standard(interpreter * my_perl=0x000f2e8c) Line 41 C
  perl527.dll!S_run_body(interpreter * my_perl, long oldscope) Line 2755 C
  perl527.dll!perl_run(interpreter * my_perl=0x000f2e8c) Line 2671 C
  perl527.dll!RunPerl(int argc=0x00000004, char * * argv=0x000eeab0, char * * env=0x000ef540) Line 252 C++
  perl.exe!__tmainCRTStartup() Line 626 C
  kernel32.dll!@​BaseThreadInitThunk@​12�() Unknown
  ntdll.dll!___RtlUserThreadStart@​8�() Unknown
  ntdll.dll!__RtlUserThreadStart@​8�() Unknown


--
bulk88 ~ bulk88 at hotmail.com

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 25, 2018

From @khwilliamson

On 02/12/2018 01​:47 PM, Sergey Aleynikov (via RT) wrote​:

# New Ticket Created by Sergey Aleynikov
# Please include the string​: [perl #132860]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132860 >

This is a bug report for perl from sergey.aleynikov@​gmail.com,
generated with the help of perlbug 1.41 running under perl 5.27.9.

-----------------------------------------------------------------
[Please describe your issue here]

I've recently done a Perl build on a Windows machine using VS 2017 and
found it's status a bit alarming​:

- perlwin32.html (aka README.win32) states "There should be no test
failures.", but at least three tests failed for 64-bit build (console
window is by default too small, so I haven't recorded them) and I
ended up with cpan/IPC-Cmd/t/01_IPC-Cmd.t stuck (for at least 10
minutes, until I've aborted it). Are those failures known? If not,
should I report them, or there're not enough volunteers to fix them,
so it's time to change the docs?

- during both 32-bit and 64-bit builds, I've seen a lot of warnings
about variables size mismatch conversions (in both ways). Most of them
seem to have the same underlying source, but they're really numerous.

Or is windows build currently supported only as a MinGW Strawberry
build by it's community?

On the Windows attached to dromedary (I presume it's a VM), the build
now hangs in the trying to probe the Storable stack size. This means
that blead can not be run on this machine until this is fixed

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 25, 2018

From @khwilliamson

On 02/12/2018 01​:47 PM, Sergey Aleynikov (via RT) wrote​:

# New Ticket Created by Sergey Aleynikov
# Please include the string​: [perl #132860]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132860 >

This is a bug report for perl from sergey.aleynikov@​gmail.com,
generated with the help of perlbug 1.41 running under perl 5.27.9.

-----------------------------------------------------------------
[Please describe your issue here]

I've recently done a Perl build on a Windows machine using VS 2017 and
found it's status a bit alarming​:

- perlwin32.html (aka README.win32) states "There should be no test
failures.", but at least three tests failed for 64-bit build (console
window is by default too small, so I haven't recorded them) and I
ended up with cpan/IPC-Cmd/t/01_IPC-Cmd.t stuck (for at least 10
minutes, until I've aborted it). Are those failures known? If not,
should I report them, or there're not enough volunteers to fix them,
so it's time to change the docs?

- during both 32-bit and 64-bit builds, I've seen a lot of warnings
about variables size mismatch conversions (in both ways). Most of them
seem to have the same underlying source, but they're really numerous.

Or is windows build currently supported only as a MinGW Strawberry
build by it's community?

I've wondered about the common warning message from the Windows compiler
to the effect that we should be using sprintf_s() and gmtime_s() (for
example) instead of what we are using. How do these differ from various
reentrant foo_r() functions that we automatically use if available?

@p5pRT
Copy link
Author

@p5pRT p5pRT commented Feb 25, 2018

From @khwilliamson

On 02/24/2018 10​:13 PM, Karl Williamson wrote​:

On 02/12/2018 01​:47 PM, Sergey Aleynikov (via RT) wrote​:

# New Ticket Created by  Sergey Aleynikov
# Please include the string​:  [perl #132860]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=132860 >

This is a bug report for perl from sergey.aleynikov@​gmail.com,
generated with the help of perlbug 1.41 running under perl 5.27.9.

-----------------------------------------------------------------
[Please describe your issue here]

I've recently done a Perl build on a Windows machine using VS 2017 and
found it's status a bit alarming​:

- perlwin32.html (aka README.win32) states "There should be no test
failures.", but at least three tests failed for 64-bit build (console
window is by default too small, so I haven't recorded them) and I
ended up with cpan/IPC-Cmd/t/01_IPC-Cmd.t stuck (for at least 10
minutes, until I've aborted it). Are those failures known? If not,
should I report them, or there're not enough volunteers to fix them,
so it's time to change the docs?

- during both 32-bit and 64-bit builds, I've seen a lot of warnings
about variables size mismatch conversions (in both ways). Most of them
seem to have the same underlying source, but they're really numerous.

Or is windows build currently supported only as a MinGW Strawberry
build by it's community?

On the Windows attached to dromedary  (I presume it's a VM), the build
now hangs in the trying to probe the Storable stack size.  This means
that blead can not be run on this machine until this is fixed

This is working again; perhaps it was my fault somehow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants