Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.
Sign upPlease define _GNU_SOURCE unconditionally on Linux to make memmem() available #16807
Comments
This comment has been minimized.
This comment has been minimized.
From aranea@aixah.deCreated by aranea@aixah.deSince 5.25, Perl uses the memmem() function to implement ninstr(). On systems with the glibc libc, things appear to work regardless (I $ perl -e '($foo) = "abc" =~ /^(.*)$/; print index("abc", $foo)' Please define _GNU_SOURCE unconditionally to fix this issue. Perl Info |
This comment has been minimized.
This comment has been minimized.
From @jkeenanOn Thu, 10 Jan 2019 16:24:06 GMT, aranea@aixah.de wrote:
How can I safely test this behavior? On Debian/Ubuntu, I see that the following packages are available via 'apt': ##### If I were to install them, would that interfere with 'libc' at all? What arguments would I pass to perl's ./Configure to say "use musl instead of libc"?
My initial reaction: That's overkill. We should detect 'musl' first and only then extend use of _GNU_SOURCE. But this is not an area of expertise for me, so others should comment. Thank you very much. -- |
This comment has been minimized.
This comment has been minimized.
|
The RT System itself - Status changed from 'new' to 'open' |
This comment has been minimized.
This comment has been minimized.
From aranea@aixah.deOn Thu, 10 Jan 2019 18:14:38 -0800
You can't just install musl on a system that uses a different libc;
I disagree. glibc too only defines memmem() conditionally (#ifdef The glibc manpage for memmem() even explicitly states that you should Thanks a lot! Luis Ressel |
This comment has been minimized.
This comment has been minimized.
From @tonycozOn Sat, 12 Jan 2019 13:20:23 -0800, aranea@aixah.de wrote:
It's not pure luck. The generated config.h with glibc has: #define HAS_GNULIBC /**/ Does musl define symbols similar to __GLIBC__ that can be used to detect musl at build-time? Tony |
This comment has been minimized.
This comment has been minimized.
From aranea@aixah.deOn Sun, 13 Jan 2019 15:22:20 -0800
Ahh, thanks for digging that up. No, musl intentionally doesn't have Defining a feature test macro such as _GNU_SOURCE globally and Regards, |
This comment has been minimized.
This comment has been minimized.
From @craigberryOn Mon, Jan 14, 2019 at 6:12 PM Luis Ressel <aranea@aixah.de> wrote:
The author's original statement is here: <https://www.openwall.com/lists/musl/2013/03/29/13> and says, "it's a bug to assume a certain implementation has
That does not sound like a good idea to me. It means assuming that <http://www.musl-libc.org/faq.html> It also means assuming that any hypothetical libc replacement on Linux I think the only way around this is to define _GNU_SOURCE when using LLVM does this to determine if musl is present: #if defined(__linux__) and !defined(__GLIBC__) and !defined(__ANDROID__) which assumes there are (and never will be) any other libc This node module: <https://github.com/lovell/detect-libc/blob/master/lib/detect-libc.js> parses the output of ldd --version to detect musl. OpenJDK does the same: <http://cr.openjdk.java.net/~mikael/webrevs/portola/musl-config.guess/webrev.00/webrev/common/autoconf/build-aux/config.guess.udiff.html> So adding something like that to hints/linux.sh may be the best way forward. |
This comment has been minimized.
This comment has been minimized.
From @doughera88On Sat, Jan 19, 2019 at 11:29:17AM -0600, Craig A. Berry wrote:
Perl's Configure test should be better. Currently, it tests that -- |
This comment has been minimized.
This comment has been minimized.
From @doughera88On Sat, Jan 19, 2019 at 01:56:55PM -0500, Andy Dougherty wrote:
I have pushed an improved test: commit ca152fd Improve Configure detection of memmem() [perl #133760]. I haven't finished up the corresponding metaconfig patch yet, but hope to soon. -- |
This comment has been minimized.
This comment has been minimized.
From @doughera88On Tue, Jan 22, 2019 at 02:33:22PM -0500, Andy Dougherty wrote:
I am attaching the accompanying metaconfig patch. I don't have a github account, -- |
This comment has been minimized.
This comment has been minimized.
From @doughera88Inline Patchdiff --git a/U/perl/d_memmem.U b/U/perl/d_memmem.U
index 404b098..1c01239 100644
--- a/U/perl/d_memmem.U
+++ b/U/perl/d_memmem.U
@@ -1,7 +1,7 @@
?RCS: You may distribute under the terms of either the GNU General Public
?RCS: License or the Artistic License, as specified in the README file.
?RCS:
-?MAKE:d_memmem: Inlibc
+?MAKE:d_memmem: d_gnulibc Compile cat rm_try run Setvar
?MAKE: -pick add $@ %<
?S:d_memmem:
?S: This variable conditionally defines the HAS_MEMMEM symbol, which
@@ -13,11 +13,55 @@
?C: This symbol, if defined, indicates that the memmem routine is
?C: available to return a pointer to the start of the first occurance
?C: of a substring in a memory area (or NULL if not found).
+?C: In glibc, memmem is a GNU extension. The function is visible in
+?C: libc, but the prototype is only visible if _GNU_SOURCE is #defined.
+?C: Thus we compile & run a test to be sure we have a valid prototype.
?C:.
?H:#$d_memmem HAS_MEMMEM /**/
?H:.
?LINT:set d_memmem
+?T:rc
+?F:!try
: see if memmem exists
-set memmem d_memmem
-eval $inlibc
+echo " "
+echo "Checking if you have a working memmem()" >&4
+$cat >try.c <<EOCP
+#$d_gnulibc HAS_GNULIBC /**/
+#if defined(HAS_GNULIBC) && !defined(_GNU_SOURCE)
+# define _GNU_SOURCE
+#endif
+#include <stdio.h>
+#include <stdlib.h>
+#include <stddef.h>
+#include <string.h>
+int main(int argc, char **argv)
+{
+ char *big = "abcdefghiabcdefghi";
+ char *little = "def";
+ char *rtn;
+ ptrdiff_t diff;
+ rtn = (char *) memmem(big, strlen(big), little, strlen(little));
+ diff = rtn - big;
+ exit(diff == 3 ? EXIT_SUCCESS : EXIT_FAILURE);
+}
+EOCP
+set try
+if eval $compile; then
+ `$run ./try`
+ rc=$?
+ case "$rc" in
+ 0) echo "Yes, you do." >&4
+ val="$define"
+ ;;
+ *) echo "Well, you have memmem, but it isn't working." >&4
+ val="$undef"
+ ;;
+ esac
+else
+ echo "No, you do not." >&4
+ val="$undef"
+fi
+set d_memmem
+eval $setvar
+$rm_try
|
This comment has been minimized.
This comment has been minimized.
From @tonycozOn Tue, Jan 22, 2019 at 02:33:22PM -0500, Andy Dougherty wrote:
I'm not sure this is sufficient. Currently the code in perl that uses memmem() compiles successfuly, Configure has hasproto, would that be a better check to see if there's Tony |
This comment has been minimized.
This comment has been minimized.
From @doughera88On Wed, Jan 23, 2019 at 09:27:55AM +1100, Tony Cook wrote:
Good point. I hadn't thought of that, and didn't encounter it in my testing.
Good idea. I'll investigate. Thanks for the thoughtful feedback! -- |
This comment has been minimized.
This comment has been minimized.
From @LeontOn Mon, Jan 14, 2019 at 12:23 AM Tony Cook via RT
Is there any circumstance in which we wouldn't want to define Leon |
This comment has been minimized.
This comment has been minimized.
From @tonycozOn Wed, Jan 23, 2019 at 02:08:35AM +0100, Leon Timmermans wrote:
I don't know. I don't think it's likely that anyone would produce a Linux libc that Tony |
This comment has been minimized.
This comment has been minimized.
From @doughera88On Tue, Jan 22, 2019 at 07:54:35PM -0500, Andy Dougherty wrote:
Now implemented in the following 2 commits. (The first enhances the hasproto commit 63c1fa6 Add ability to include literal text in the prototype check. commit f8d82a1 Another attempt to improve Configure detection of memmem() [perl #133760]. -- |
This comment has been minimized.
This comment has been minimized.
From @TuxOn Wed, 23 Jan 2019 21:58:04 -0500, Andy Dougherty
Sorry to chime in late. I was on a ski-trip Do you have the final meta-patch? I'll apply -- |
This comment has been minimized.
This comment has been minimized.
From @doughera88On Wed, Jan 23, 2019 at 09:58:04PM -0500, Andy Dougherty wrote:
The following 2 commits handle the other problematic functions in musl I will send metaconfig updates separately. commit 04db542 Define _GNU_SOURCE if using the musl libc on linux. Together with prior commits ba73a4c, f8d82a1, and 63c1fa6, commit ba73a4c Improve detection of memrchr, strlcat, and strlcpy. This is continuation of commit f8d82a1 addressing [perl #133760]. Still to be done is automatically adding _GNU_SOURCE if the musl library -- |
This comment has been minimized.
This comment has been minimized.
From @doughera88On Thu, Jan 31, 2019 at 03:33:31PM -0500, Andy Dougherty wrote:
I have attached the relevant metaconfig updates. -- |
This comment has been minimized.
This comment has been minimized.
From @doughera880001-Improve-detection-of-memrchr-strlcat-and-strlcpy.patchFrom d69c1265abdfdeb41bf35021e36f1b417462e7f3 Mon Sep 17 00:00:00 2001
From: Andy Dougherty <doughera@lafayete.edu>
Date: Thu, 31 Jan 2019 15:39:27 -0500
Subject: [PATCH] Improve detection of memrchr, strlcat, and strlcpy.
This is continuation of commit b6f9c611a4 addressing [perl #133760].
Linux systems using the musl C library have memmem, memrchr, strlcat, and
strlcpy, but the prototypes are only visible if _GNU_SOURCE is defined.
This patch makes Configure test both whether the prototype is visible
and whether the C symbol is visible.
---
U/perl/d_memrchr.U | 24 +++++++++++++++++++++---
U/perl/d_strlcat.U | 24 +++++++++++++++++++++---
U/perl/d_strlcpy.U | 24 +++++++++++++++++++++---
3 files changed, 63 insertions(+), 9 deletions(-)
diff --git a/U/perl/d_memrchr.U b/U/perl/d_memrchr.U
index 4fea9a6..8b0cf88 100644
--- a/U/perl/d_memrchr.U
+++ b/U/perl/d_memrchr.U
@@ -1,7 +1,7 @@
?RCS: You may distribute under the terms of either the GNU General Public
?RCS: License or the Artistic License, as specified in the README file.
?RCS:
-?MAKE:d_memrchr: Inlibc
+?MAKE:d_memrchr: d_gnulibc Hasproto Inlibc Setvar
?MAKE: -pick add $@ %<
?S:d_memrchr:
?S: This variable conditionally defines the HAS_MEMRCHR symbol, which
@@ -17,7 +17,25 @@
?H:#$d_memrchr HAS_MEMRCHR /**/
?H:.
?LINT:set d_memrchr
+?T:xx1 xx2 xx3 xx4 d_memrchr_proto
: see if memrchr exists
-set memrchr d_memrchr
-eval $inlibc
+: We need both a prototype in string.h and the symbol in libc.
+echo " "
+d_memrchr_proto=''
+xx1="#$d_gnulibc HAS_GNULIBC"
+xx2='#if defined(HAS_GNULIBC) && !defined(_GNU_SOURCE)'
+xx3='# define _GNU_SOURCE'
+xx4='#endif'
+set d_memrchr_proto memrchr literal "$xx1" literal "$xx2" literal "$xx3" literal "$xx4" define string.h
+eval $hasproto
+case "$d_memrchr_proto" in
+ define) # see if memrchr exists
+ set memrchr d_memrchr
+ eval $inlibc
+ ;;
+ *) val=$undef
+ set d_memrchr
+ eval $setvar
+ ;;
+esac
diff --git a/U/perl/d_strlcat.U b/U/perl/d_strlcat.U
index c082be9..0fb1d82 100644
--- a/U/perl/d_strlcat.U
+++ b/U/perl/d_strlcat.U
@@ -5,7 +5,7 @@
?RCS: You may distribute under the terms of either the GNU General Public
?RCS: License or the Artistic License, as specified in the README file.
?RCS:
-?MAKE:d_strlcat: Inlibc
+?MAKE:d_strlcat: d_gnulibc Hasproto Inlibc Setvar
?MAKE: -pick add $@ %<
?S:d_strlcat:
?S: This variable conditionally defines the HAS_STRLCAT symbol, which
@@ -18,7 +18,25 @@
?H:#$d_strlcat HAS_STRLCAT /**/
?H:.
?LINT:set d_strlcat
+?T:xx1 xx2 xx3 xx4 d_strlcat_proto
: see if strlcat exists
-set strlcat d_strlcat
-eval $inlibc
+: We need both a prototype in string.h and the symbol in libc.
+echo " "
+d_strlcat_proto=''
+xx1="#$d_gnulibc HAS_GNULIBC"
+xx2='#if defined(HAS_GNULIBC) && !defined(_GNU_SOURCE)'
+xx3='# define _GNU_SOURCE'
+xx4='#endif'
+set d_strlcat_proto strlcat literal "$xx1" literal "$xx2" literal "$xx3" literal "$xx4" define string.h
+eval $hasproto
+case "$d_strlcat_proto" in
+ define) # see if strlcat exists
+ set strlcat d_strlcat
+ eval $inlibc
+ ;;
+ *) val=$undef
+ set d_strlcat
+ eval $setvar
+ ;;
+esac
diff --git a/U/perl/d_strlcpy.U b/U/perl/d_strlcpy.U
index 02193c3..8f0a572 100644
--- a/U/perl/d_strlcpy.U
+++ b/U/perl/d_strlcpy.U
@@ -5,7 +5,7 @@
?RCS: You may distribute under the terms of either the GNU General Public
?RCS: License or the Artistic License, as specified in the README file.
?RCS:
-?MAKE:d_strlcpy: Inlibc
+?MAKE:d_strlcpy: d_gnulibc Hasproto Inlibc Setvar
?MAKE: -pick add $@ %<
?S:d_strlcpy:
?S: This variable conditionally defines the HAS_STRLCPY symbol, which
@@ -18,7 +18,25 @@
?H:#$d_strlcpy HAS_STRLCPY /**/
?H:.
?LINT:set d_strlcpy
+?T:xx1 xx2 xx3 xx4 d_strlcpy_proto
: see if strlcpy exists
-set strlcpy d_strlcpy
-eval $inlibc
+: We need both a prototype in string.h and the symbol in libc.
+echo " "
+d_strlcpy_proto=''
+xx1="#$d_gnulibc HAS_GNULIBC"
+xx2='#if defined(HAS_GNULIBC) && !defined(_GNU_SOURCE)'
+xx3='# define _GNU_SOURCE'
+xx4='#endif'
+set d_strlcpy_proto strlcpy literal "$xx1" literal "$xx2" literal "$xx3" literal "$xx4" define string.h
+eval $hasproto
+case "$d_strlcpy_proto" in
+ define) # see if strlcpy exists
+ set strlcpy d_strlcpy
+ eval $inlibc
+ ;;
+ *) val=$undef
+ set d_strlcpy
+ eval $setvar
+ ;;
+esac
--
2.11.0
|
This comment has been minimized.
This comment has been minimized.
From @tonycozOn Thu, 31 Jan 2019 12:33:45 -0800, doughera wrote:
Thanks, closing. I've proposed the fixes for backporting to 5.28. Tony |
This comment has been minimized.
This comment has been minimized.
|
@tonycoz - Status changed from 'open' to 'pending release' |
This comment has been minimized.
This comment has been minimized.
From @khwilliamsonThank you for filing this report. You have helped make Perl better. With the release today of Perl 5.30.0, this and 160 other issues have been Perl 5.30.0 may be downloaded via: If you find that the problem persists, feel free to reopen this ticket. |
This comment has been minimized.
This comment has been minimized.
|
@khwilliamson - Status changed from 'pending release' to 'resolved' |
Migrated from rt.perl.org#133760 (status was 'resolved')
Searchable as RT133760$