Cygwin: nm detects memmem, but headers don't expose it, leading to tests crashing

[p5pRT]

Migrated from rt.perl.org#128358 (status was 'open')

Searchable as RT128358$

[p5pRT]

From @dcollinsn

In my cygwin system, the declaration for memmem in included in string.h, with #include guards​:

  #if __GNU_VISIBLE
  _PTR _EXFUN(mempcpy,(_PTR, const _PTR, size_t));
  _PTR _EXFUN(memmem, (const _PTR, size_t, const _PTR, size_t));
  _PTR _EXFUN(memrchr,(const _PTR, int, size_t));
  _PTR _EXFUN(rawmemchr,(const _PTR, int));
  #endif

__GNU_VISIBLE is in turn defined in sys/features.h​:

  #ifdef _GNU_SOURCE
  #define __GNU_VISIBLE 1
  #else
  #define __GNU_VISIBLE 0
  #endif

However, ./Configure with the Cygwin hints will not set _GNU_SOURCE. So, nm detects memmem, util.c uses it, and nothing declares it, leading to this warning​:

  gcc -c -DPERL_CORE -DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -D_FORTIFY_SOURCE=2 -std=c89 -O3 -g -Wall -Werror=declaration-after-statement -Wextra -Wc++-compat -Wwrite-strings util.c
  In file included from perl.h​:3902​:0,
  from util.c​:26​:
  util.c​: In function ‘Perl_ninstr’​:
  util.h​:236​:23​: warning​: implicit declaration of function ‘memmem’ [-Wimplicit-function-declaration]
  ((char *) memmem(big, bigend - big, little, lend - little))
  ^
  util.c​:589​:12​: note​: in expansion of macro ‘ninstr’
  return ninstr(big, bigend, little, lend);
  ^
  util.h​:236​:14​: warning​: cast to pointer from integer of different size [-Wint-to-pointer-cast]
  ((char *) memmem(big, bigend - big, little, lend - little))
  ^
  util.c​:589​:12​: note​: in expansion of macro ‘ninstr’
  return ninstr(big, bigend, little, lend);
  ^
  util.c​: In function ‘Perl_fbm_instr’​:
  util.h​:236​:14​: warning​: cast to pointer from integer of different size [-Wint-to-pointer-cast]
  ((char *) memmem(big, bigend - big, little, lend - little))
  ^
  util.c​:937​:19​: note​: in expansion of macro ‘ninstr’
  char * const b = ninstr((char*)big,(char*)bigend,
  ^
  vutil.c​: In function ‘Perl_new_version’​:
  util.h​:236​:14​: warning​: cast to pointer from integer of different size [-Wint-to-pointer-cast]
  ((char *) memmem(big, bigend - big, little, lend - little))
  ^
  vutil.c​:533​:14​: note​: in expansion of macro ‘ninstr’
  under = ninstr(raw, raw+len, underscore, underscore + 1);
  ^
 
And a number of test failures​:

  Failed 25 tests out of 2359, 98.94% okay.
  ../cpan/CPAN-Meta/t/no-index.t
  ../cpan/Encode/t/jperl.t
  ../cpan/Filter-Util-Call/t/call.t
  ../cpan/Filter-Util-Call/t/rt_101033.t
  ../cpan/HTTP-Tiny/t/060_http_date.t
  ../cpan/Module-Metadata/t/encoding.t
  ../cpan/version/t/01base.t
  ../cpan/version/t/02derived.t
  ../cpan/version/t/03require.t
  ../cpan/version/t/10_lyon.t
  ../dist/Filter-Simple/t/code_no_comments.t
  ../dist/Filter-Simple/t/data.t
  ../dist/Filter-Simple/t/export.t
  ../dist/Filter-Simple/t/filter.t
  ../dist/Filter-Simple/t/filter_only.t
  ../dist/Filter-Simple/t/import.t
  ../dist/Time-HiRes/t/ualarm.t
  ../ext/B/t/concise.t
  op/closure.t
  op/incfilter.t
  op/index.t
  op/index_thr.t
  op/packagev.t
  op/threads.t
  uni/overload.t
 
Many of those tests exit with "Out of memory", despite not being out of memory. Adding -D_GNU_SOURCE to hints/cygwin, as done in the attached patch, fixes all but two failures​:

  ../dist/Time-HiRes/t/ualarm.t
  op/threads.t
 
Are there any risks or downsides to enabling this flag on Cygwin? Are there any other configurations you'd like me to test?

Summary of my perl5 (revision 5 version 25 subversion 2) configuration​:
  Derived from​: 8c47236
  Platform​:
  osname=cygwin, osvers=2.5.1(0.29753), archname=cygwin-thread-multi
  uname='cygwin_nt-10.0 desktop-er38tcq 2.5.1(0.29753) 2016-04-21 22​:14 x86_64 cygwin '
  config_args='-Dusedevel -des'
  hint=recommended, useposix=true, d_sigaction=define
  useithreads=define, usemultiplicity=define
  use64bitint=define, use64bitall=define, uselongdouble=undef
  usemymalloc=n, bincompat5005=undef
  Compiler​:
  cc='gcc', ccflags ='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -D_FORTIFY_SOURCE=2',
  optimize='-O3',
  cppflags='-DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong'
  ccversion='', gccversion='5.3.0', gccosandvers=''
  intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678, doublekind=3
  d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16, longdblkind=3
  ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
  alignbytes=8, prototype=define
  Linker and Libraries​:
  ld='g++', ldflags =' -Wl,--enable-auto-import -Wl,--export-all-symbols -Wl,--enable-auto-image-base -fstack-protector-strong -L/usr/local/lib'
  libpth=/usr/lib /usr/lib/../lib/../include/w32api /usr/local/lib /lib
  libs=-lpthread -ldl
  perllibs=-lpthread -ldl
  libc=/usr/lib/libc.a, so=dll, useshrplib=true, libperl=cygperl5_25_2.dll
  gnulibc_version=''
  Dynamic Linking​:
  dlsrc=dl_dlopen.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' '
  cccdlflags=' ', lddlflags=' --shared -Wl,--enable-auto-import -Wl,--export-all-symbols -Wl,--enable-auto-image-base -L/usr/local/lib -fstack-protector-strong'

Characteristics of this binary (from libperl)​:
  Compile-time options​: HAS_TIMES MULTIPLICITY PERLIO_LAYERS
  PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV
  PERL_HASH_FUNC_ONE_AT_A_TIME_HARD
  PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP PERL_OP_PARENT
  PERL_PRESERVE_IVUV PERL_USE_DEVEL
  PERL_USE_SAFE_PUTENV USE_64_BIT_ALL USE_64_BIT_INT
  USE_ITHREADS USE_LARGE_FILES USE_LOCALE
  USE_LOCALE_COLLATE USE_LOCALE_CTYPE
  USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
  USE_PERL_ATOF USE_REENTRANT_API
  Locally applied patches​:
  uncommitted-changes
  Built under cygwin
  Compiled at Jun 9 2016 17​:31​:32
  @​INC​:
  lib
  /usr/local/lib/perl5/site_perl/5.25.2/cygwin-thread-multi
  /usr/local/lib/perl5/site_perl/5.25.2
  /usr/local/lib/perl5/5.25.2/cygwin-thread-multi
  /usr/local/lib/perl5/5.25.2
  .

[p5pRT]

From @dcollinsn

0001-Cygwin-headers-don-t-declare-memmem-without-_GNU_SOU.patch

From 031834dba0bb43bac07fa27f0af85d908e03fb09 Mon Sep 17 00:00:00 2001
From: Dan Collins <dcollinsn@gmail.com>
Date: Thu, 9 Jun 2016 18:29:51 -0400
Subject: [PATCH] Cygwin headers don't declare memmem() without _GNU_SOURCE

As a result, while nm detected memmem in libc, it was not present
in the system headers as preprocessed using Perl's ccflags, causing
gcc to treat it as an implicit definition, guessing the wrong
return type, generating compiler warnings and worse whereever it
was used. This patch gets my clean Cygwin 2.5.1 install from 25
failing test scripts to 2.
---
 hints/cygwin.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hints/cygwin.sh b/hints/cygwin.sh
index 21997db..20e0e58 100644
--- a/hints/cygwin.sh
+++ b/hints/cygwin.sh
@@ -31,7 +31,7 @@ test -z "$optimize" && optimize='-O3'
 man3ext='3pm'
 test -z "$use64bitint" && use64bitint='define'
 test -z "$useithreads" && useithreads='define'
-ccflags="$ccflags -DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__"
+ccflags="$ccflags -DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__ -D_GNU_SOURCE"
 # - otherwise i686-cygwin
 archname='cygwin'
 
-- 
2.8.3

[p5pRT]

From @dcollinsn

perl_test.txt

[p5pRT]

From Stromeko@nexgo.de

Dan Collins (via RT) writes​:

Are there any risks or downsides to enabling this flag on Cygwin? Are
there any other configurations you'd like me to test?

FWIW, here's my patch for the Perl that comes with Cygwin (and don't use
nm, it doesn't work correctly)​:

--- buildperl/hints/cygwin.sh.orig
+++ buildperl/hints/cygwin.sh
@​@​ -25,13 +25,22 @​@​
libswanted=`echo " $libswanted " | sed -e 's/ m / /g'`
# - eliminate -lutil, symbols are all in libcygwin.a
libswanted=`echo " $libswanted " | sed -e 's/ util / /g'`
+ignore_versioned_solibs='y'
+usenm='no'
+libc='/usr/lib/libcygwin.a'
+loclibpth=' '
+glibpth=' '
+plibpth=' '
+libpth=' '
+PATH='.​:/usr/bin/'
# - add libgdbm_compat $libswanted
libswanted="$libswanted gdbm_compat"
test -z "$optimize" && optimize='-O3'
man3ext='3pm'
test -z "$use64bitint" && use64bitint='define'
test -z "$useithreads" && useithreads='define'
-ccflags="$ccflags -DPERL_USE_SAFE_PUTENV -U__STRICT_ANSI__"
+test -z "$usemymalloc" && usemymalloc='undef'
+ccflags="$ccflags -DPERL_USE_SAFE_PUTENV -D_GNU_SOURCE -U__STRICT_ANSI__"
# - otherwise i686-cygwin
archname='cygwin'

@​@​ -62,6 +71,13 @​@​
  d_inetpton='undef'
esac

+case "$osvers" in
+ 2.[0-4].*|1.*)
+ # newlib finitel is buggy before cygwin-2.5.0
+ d_finitel='undef'
+ ;;
+esac
+
# compile Win32CORE "module" as static. try to avoid the space.
if test -z "$static_ext"; then
  static_ext="Win32CORE"

Regards
Achim.
--
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf rackAttack​:
http​://Synth.Stromeko.net/Downloads.html#WaldorfSounds

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'