Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot compile uClibc-ng with the Clang compiler... #7

Open
JonathanBelanger opened this issue Jul 24, 2019 · 11 comments
Open

Cannot compile uClibc-ng with the Clang compiler... #7

JonathanBelanger opened this issue Jul 24, 2019 · 11 comments

Comments

@JonathanBelanger
Copy link

I've converted from make to CMake (it has better toolchain support - imho). I was trying to compile using clang, but there is at least one nested function in the Native pthread code, in particular allocatestack.c. It would be great to fix these as well, so that we can compile using other compilers.

@fallen
Copy link
Contributor

fallen commented Jul 25, 2019

Hello,
Can you propose a patch?
Can you also explain better the issue? (copy paste of compiling line with compiler arguments and error message)

@JonathanBelanger
Copy link
Author

Here is the example from allocatestack.c (lines 777-886, the inline function is at line 800), in the following function (I removed some code so that this reply would not be too long):

/* In case of a fork() call the memory allocation in the child will be
   the same but only one thread is running.  All stacks except that of
   the one running thread are not used anymore.  We have to recycle
   them.  */
void
__reclaim_stacks (void)
{
  struct pthread *self = (struct pthread *) THREAD_SELF;
    .
    .
    .
  if (in_flight_stack != 0)
    {
      bool add_p = in_flight_stack & 1;
      list_t *elem = (list_t *)(uintptr_t)(in_flight_stack & ~UINTMAX_C (1));

      if (add_p)
	{
	  /* We always add at the beginning of the list.  So in this
	     case we only need to check the beginning of these lists.  */
	  int check_list (list_t *l)
	  {
	    if (l->next->prev != l)
	      {
		assert (l->next->prev == elem);

		elem->next = l->next;
		elem->prev = l;
		l->next = elem;

		return 1;
	      }

	    return 0;
	  }

	  if (check_list (&stack_used) == 0)
	    (void) check_list (&stack_cache);
	}
      else
	{
	  /* We can simply always replay the delete operation.  */
	  elem->next->prev = elem->prev;
	  elem->prev->next = elem->next;
	}
    }
    .
    .
    .
}

The function int check_list(...) should be define outside of __reclaim_stacks. I would suggest changing it as follows:

  /* We always add at the beginning of the list.  So in this
     case we only need to check the beginning of these lists.  */
static inline int __attribute__((always_inline))
check_list (list_t *l, list_t **elemPtr)
  {
   list_t elem = *elemPtr;

    if (l->next->prev != l)
      {
	assert (l->next->prev == elem);

	elem->next = l->next;
	elem->prev = l;
	l->next = elem;

	return 1;
      }

    return 0;
  }

/* In case of a fork() call the memory allocation in the child will be
   the same but only one thread is running.  All stacks except that of
   the one running thread are not used anymore.  We have to recycle
   them.  */
void
__reclaim_stacks (void)
{
  struct pthread *self = (struct pthread *) THREAD_SELF;
    .
    .
    .
  if (in_flight_stack != 0)
    {
      bool add_p = in_flight_stack & 1;
      list_t *elem = (list_t *)(uintptr_t)(in_flight_stack & ~UINTMAX_C (1));

      if (add_p)
	{
	  if (check_list (&stack_used, &elem) == 0)
	    (void) check_list (&stack_cache, &elem);
	}
      else
	{
	  /* We can simply always replay the delete operation.  */
	  elem->next->prev = elem->prev;
	  elem->prev->next = elem->next;
	}
    }
    .
    .
    .
}

@fallen
Copy link
Contributor

fallen commented Jul 27, 2019

I guess this kind of change would be accepted easily.
Can you send a patch to the list?

@JonathanBelanger
Copy link
Author

JonathanBelanger commented Jul 27, 2019

Also, there is another thing that would need to be changed. For the generation of header files from .sym files, clang complains about names with @ in them. The script gen-as-const.awk, generates a stream with symbols with @@@name@@@, @@@value@@@, and @@@end@@@. Eventually @@@name@@@ becomes #define, a dollar-sign $ in the value gets removed and @@@end@@@ is removed.

gen-as-const.awk should be changed to generate it stream from:

__asm__("@@@name@@@<constant-name>@@@value@@@%0@@@end@@@" " "i" ((long) sizeof(<structure-name>)));

to:

__asm__("#define <constant-name> %0" " "i" ((long) sizeof(<structure-name>)));

The compiler will be used to generate lines that look like the following:

        #define <constant-name> $<constant-value>

The $ needs to be removed. There will also be other lines in the output from the compile (particularly with clang), but the need to go as well. This change will work for both gcc and clang.

Finally, the -no-nonnull-compare qualifier is no longer value. It should be changed to -no-nonnull. This problem never really shows up, except when another warning or error is reported.

I hope this helps and is what you asked.

@JonathanBelanger
Copy link
Author

Here is the pat\ch:

diff --git a/libpthread/nptl/allocatestack.c b/libpthread/nptl/allocatestack.c
index 137979542..e7e0b4d8f 100644
--- a/libpthread/nptl/allocatestack.c
+++ b/libpthread/nptl/allocatestack.c
@@ -773,6 +773,25 @@ __make_stacks_executable (void **stack_endp)
   return err;
 }
 
+/* We always add at the beginning of the list.  So in this
+   case we only need to check the beginning of these lists.  */
+static inline int __attribute__((always_inline))
+check_list (list_t *l, list_t **elemPtr)
+{
+  list_t *elem = *elemPtr;
+  if (l->next->prev != l)
+    {
+      assert (l->next->prev == elem);
+
+      elem->next = l->next;
+      elem->prev = l;
+      l->next = elem;
+
+      return 1;
+    }
+
+  return 0;
+}
 
 /* In case of a fork() call the memory allocation in the child will be
    the same but only one thread is running.  All stacks except that of
@@ -794,26 +813,8 @@ __reclaim_stacks (void)
 
       if (add_p)
 	{
-	  /* We always add at the beginning of the list.  So in this
-	     case we only need to check the beginning of these lists.  */
-	  int check_list (list_t *l)
-	  {
-	    if (l->next->prev != l)
-	      {
-		assert (l->next->prev == elem);
-
-		elem->next = l->next;
-		elem->prev = l;
-		l->next = elem;
-
-		return 1;
-	      }
-
-	    return 0;
-	  }
-
-	  if (check_list (&stack_used) == 0)
-	    (void) check_list (&stack_cache);
+	  if (check_list (&stack_used, &elem) == 0)
+	    (void) check_list (&stack_cache, &elem);
 	}
       else
 	{

@JonathanBelanger
Copy link
Author

I found a couple of other issues, with clang. For one, it does not like hints on jne ASM statements. For example:

jne,pt 10f

Clang complains about the ,pt. Since this is just a hint, it can be removed. The only place that has these is in the libpthread/nptl/sysdeps/unix/sysv/linux/x86_64/pthread_barrier_wait.S, at lines 85 and 122.

@fallen
Copy link
Contributor

fallen commented Jul 30, 2019

Could you send the patch to the mailing list? devel@uclibc-ng.org
You can generate the patch using git format-patch and then send it using git send-email

According to the ",pt" hint, it seems this is not interpreted at all (nop) on post Pentium-4 CPU. So it seems useless and also increases binary size.
I guess removal of such hints could be a good idea.

@JonathanBelanger
Copy link
Author

Will you be rolling it into this repository?

@fallen
Copy link
Contributor

fallen commented Aug 2, 2019

I don't have the commit bit on this repo.
Usual way to get something integrated here is to post a patch using git send-email to the mailing list.
The more the patch is explained in the commit message (why this is needed, what it fixes etc), the more chance it has to get integrated (quickly).

@JonathanBelanger
Copy link
Author

sent

@fallen
Copy link
Contributor

fallen commented Oct 12, 2020

It seems your email never got through to the list, according to the archives: https://mailman.uclibc-ng.org/pipermail/devel/
Maybe you need to first sign up and then re-send it?

mikpe pushed a commit to mikpe/uclibc-ng that referenced this issue Dec 13, 2021
- use the provided __res_state() method instead of direct access
  to struct __res_state pointer &_res/*__resp

- change the __UCLIBC_HAS_TLS__ protected __res_state() implementation
  to the one where the comment 'When threaded, _res may be a per-thread
  variable.' indicates this should be used with threads/TLS enabled

Fixes the following segfaults with buildroot raspberrypi3_64_defconfig
(uclibc, -Os, Note: runs fine using the raspberrypi3_defconfig):

  $ /usr/sbin/ntpd -n -d
  1970-01-01T00:01:49 ntpd[249]: INIT: ntpd ntpsec-1.2.0 2021-11-03T20:39:50Z: Starting
  1970-01-01T00:01:49 ntpd[249]: INIT: Command line: /usr/sbin/ntpd -n -d
  1970-01-01T00:01:49 ntpd[249]: INIT: precision = 7.240 usec (-17)
  1970-01-01T00:01:49 ntpd[249]: INIT: successfully locked into RAM
  1970-01-01T00:01:49 ntpd[249]: CONFIG: readconfig: parsing file: /etc/ntp.conf
  1970-01-01T00:01:49 ntpd[249]: CONFIG: restrict nopeer ignored
  1970-01-01T00:01:49 ntpd[249]: INIT: Using SO_TIMESTAMPNS
  1970-01-01T00:01:49 ntpd[249]: IO: Listen and drop on 0 v6wildcard [::]:123
  1970-01-01T00:01:49 ntpd[249]: IO: Listen and drop on 1 v4wildcard 0.0.0.0:123
  1970-01-01T00:01:49 ntpd[249]: IO: Listen normally on 2 lo 127.0.0.1:123
  1970-01-01T00:01:49 ntpd[249]: IO: Listen normally on 3 eth0 172.16.0.30:123
  1970-01-01T00:01:49 ntpd[249]: IO: Listen normally on 4 lo [::1]:123
  1970-01-01T00:01:49 ntpd[249]: IO: Listen normally on 5 eth0 [fe80::ba27:ebff:fea6:340%2]:123
  1970-01-01T00:01:49 ntpd[249]: IO: Listening on routing socket on fd #22 for interface updates
  1970-01-01T00:01:50 ntpd[249]: SYNC: Found 10 servers, suggest minsane at least 3
  1970-01-01T00:01:50 ntpd[249]: INIT: MRU 10922 entries, 13 hash bits, 65536 bytes
  1970-01-01T00:01:50 ntpd[249]: statistics directory /var/NTP/ does not exist or is unwriteable, error No such file or directory
  1970-01-01T00:01:51 ntpd[249]: DNS: dns_probe: 0.pool.ntp.org, cast_flags:8, flags:101
  Segmentation fault (core dumped)

  $ ./host/bin/aarch64-buildroot-linux-uclibc-gdb ./build/ntpsec-1_2_0/build/main/ntpd/ntpd core
  Core was generated by `/usr/sbin/ntpd -n -d'.
  Program terminated with signal SIGSEGV, Segmentation fault.
  (gdb) where
  #0  0x0000007f8ff1f150 in res_sync_func () at libc/inet/resolv.c:3356
  wbx-github#1  0x0000007f8ff1c468 in __open_nameservers () at libc/inet/resolv.c:949
  wbx-github#2  0x0000007f8ff1b498 in __dns_lookup (name=0x55943c67f0 "0.pool.ntp.org",
      type=1, outpacket=0x7f8fe91c48, a=0x7f8fe91c08) at libc/inet/resolv.c:1134
  wbx-github#3  0x0000007f8ff1d744 in __GI_gethostbyname_r (
      name=0x55943c67f0 "0.pool.ntp.org", result_buf=0x7f8fe92628,
      buf=0x7f8fe91d90 "", buflen=992, result=0x7f8fe92670,
      h_errnop=0x7f8fe92668) at libc/inet/resolv.c:1966
  wbx-github#4  0x0000007f8ff1d9a0 in __GI_gethostbyname2_r (
      name=0x55943c67f0 "0.pool.ntp.org", family=2, result_buf=0x7f8fe92628,
      buf=0x7f8fe91d70 "0.pool.ntp.org", buflen=1024, result=0x7f8fe92670,
      h_errnop=0x7f8fe92668) at libc/inet/resolv.c:2065
  wbx-github#5  0x0000007f8ff16924 in gaih_inet (name=0x55943c67f0 "0.pool.ntp.org",
      service=0x7f8fe92828, req=0x7f8fe92890, pai=0x7f8fe92838)
      at libc/inet/getaddrinfo.c:596
  wbx-github#6  0x0000007f8ff17624 in __GI_getaddrinfo (
      name=0x55943c67f0 "0.pool.ntp.org",
      service=0x5582eb8acd "\377H\213D$\bL\211\367H\213\260\270",
      hints=0x7f8fe92890, pai=0x5582ee1bf8) at libc/inet/getaddrinfo.c:957
  wbx-github#7  0x0000005582ea60f4 in _start ()
  (gdb) p _res
  $1 = {options = 0, nsaddr_list = {{sin_family = 0, sin_port = 0, sin_addr = {
          s_addr = 0}, sin_zero = "\000\000\000\000\000\000\000"}, {
        sin_family = 0, sin_port = 0, sin_addr = {s_addr = 0},
        sin_zero = "\000\000\000\000\000\000\000"}, {sin_family = 0,
        sin_port = 0, sin_addr = {s_addr = 0},
        sin_zero = "\000\000\000\000\000\000\000"}}, dnsrch = {0x0, 0x0, 0x0,
      0x0, 0x0, 0x0, 0x0}, nscount = 0 '\000', ndots = 0 '\000',
    retrans = 0 '\000', retry = 0 '\000', defdname = '\000' <repeats 255 times>,
    nsort = 0 '\000', pfcode = 0, id = 0, res_h_errno = 0, sort_list = {{addr = {
          s_addr = 0}, mask = 0}, {addr = {s_addr = 0}, mask = 0}, {addr = {
          s_addr = 0}, mask = 0}, {addr = {s_addr = 0}, mask = 0}, {addr = {
          s_addr = 0}, mask = 0}, {addr = {s_addr = 0}, mask = 0}, {addr = {
          s_addr = 0}, mask = 0}, {addr = {s_addr = 0}, mask = 0}, {addr = {
          s_addr = 0}, mask = 0}, {addr = {s_addr = 0}, mask = 0}}, _u = {
      _ext = {nsaddrs = {0x0, 0x0, 0x0}, nscount = 0 '\000', nstimes = {0, 0,
          0}, nssocks = {0, 0, 0}, nscount6 = 0, nsinit = 0}}}
  (gdb) p &_res
  $2 = (struct __res_state *) 0x7f8ff8fd98 <_res>
  (gdb) p rp
  $3 = (struct __res_state *) 0x7fffffffff

  And the following uclibc code at libc/inet/resolv.c:3356:

  3345 static void res_sync_func(void)
  3346 {
  3347         struct __res_state *rp = &(_res);
  3348         int n;
  3349
  3350         /* If we didn't get malloc failure earlier... */
  3351         if (__nameserver != (void*) &__local_nameserver) {
  3352                 /* TODO:
  3353                  * if (__nameservers < rp->nscount) - try to grow __nameserver[]?
  3354                  */
  3355 #ifdef __UCLIBC_HAS_IPV6__
  3356                 if (__nameservers > rp->_u._ext.nscount)
  3357                         __nameservers = rp->_u._ext.nscount;
  3358                 n = __nameservers;

  The special thing about ntpsec is the DNS lookup in an extra thread
  and/or the call to res_init(), see ntpsec-1_2_0/ntpd/ntp_dns.c:

   69         msyslog(LOG_INFO, "DNS: dns_probe: %s, cast_flags:%x, flags:%x%s",
   70                 hostname, pp->cast_flags, pp->cfg.flags, busy);
   71         if (NULL != active)     /* normally redundant */
   72                 return false;
   73
   74         active = pp;
   75
   76         sigfillset(&block_mask);
   77         pthread_sigmask(SIG_BLOCK, &block_mask, &saved_sig_mask);
   78         rc = pthread_create(&worker, NULL, dns_lookup, pp);

  and

  165 static void* dns_lookup(void* arg)
  166 {
  167         struct peer *pp = (struct peer *) arg;
  168         struct addrinfo hints;
  169
  170 #ifdef HAVE_SECCOMP_H
  171         setup_SIGSYS_trap();      /* enable trap for this thread */
  172 #endif
  173
  174 #ifdef HAVE_RES_INIT
  175         /* Reload DNS servers from /etc/resolv.conf in case DHCP has updated it.
  176          * We only need to do this occasionally, but it's not expensive
  177          * and simpler to do it every time than it is to figure out when
  178          * to do it.
  179          * This res_init() covers NTS too.
  180          */
  181         res_init();
  182 #endif
  183
  184         if (pp->cfg.flags & FLAG_NTS) {
  185 #ifndef DISABLE_NTS
  186                 nts_probe(pp);
  187 #endif
  188         } else {
  189                 ZERO(hints);
  190                 hints.ai_protocol = IPPROTO_UDP;
  191                 hints.ai_socktype = SOCK_DGRAM;
  192                 hints.ai_family = AF(&pp->srcadr);
  193                 gai_rc = getaddrinfo(pp->hostname, NTP_PORTA, &hints, &answer);
  194         }

  $ /usr/lib/uclibc-ng-test/test/inet/tst-res
  Segmentation fault (core dumped)

  $ ./host/bin/aarch64-buildroot-linux-uclibc-gdb ./build/uclibc-ng-test-0844445e7358eb10e716155b55b0fb23e88d644a/test/inet/tst-res core
  Core was generated by `/usr/lib/uclibc-ng-test/test/inet/tst-res'.
  Program terminated with signal SIGSEGV, Segmentation fault.
  (gdb) where
  #0  __GI___res_init () at libc/inet/resolv.c:3514
  wbx-github#1  0x0000005591e507e4 in main (argc=<optimized out>, argv=<optimized out>)
      at tst-res.c:20

First reported here:
https://lore.kernel.org/buildroot/20211028230131.5f50d6e7@gmx.net/
https://www.mail-archive.com/devel@uclibc-ng.org/msg01085.html

Signed-off-by: Peter Seiderer <ps.report@gmx.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants