Failure to bind UDP sockets on Ubuntu Saucy

[cjdelisle]

In Ubuntu Saucy, SO_REUSEPORT is defined but the kernel doesn't support it causing setsocketopt() to fail with ENOPROTOOPT and aborting uv_bind(). This is currently breaking cjdns tests on saucy. https://code.launchpad.net/~calebdelisle/+archive/cjdns/

Even after Ubuntu upgrades to kernel 3.9 there are still going to be the realtime kernels and the 2.6.x kernels to contend with.

Here's the offending code:
https://github.com/joyent/libuv/blob/master/src/unix/udp.c#L317

We could:

1. Treat ENOPROTOOPT as a non-error which makes some sense if we want forward compatibility and would be critical for smoothly implementing linux: set SO_REUSEPORT on listen sockets #747
2. Remove the calls entirely because they only appear in UDP code where in my mind they don't make much sense.
3. Change the ifdef to an if defined(SO_REUSEPORT) && !defined(linux)

Personally I'm up in the air between 1 and 2 but I'm leaning toward 2.

WDYT?

[cjdelisle]

Info about SO_REUSEPORT I found useful http://stackoverflow.com/questions/14388706/socket-options-so-reuseaddr-and-so-reuseport-how-do-they-differ-do-they-mean-t

[Shnatsel]

Saucy is using 3.10 where SO_REUSEPORT is already present and should be working properly (it was introduced in 3.9).

[bnoordhuis]

I test on a range of systems that run 3.8 up to 3.11-rc4 and I haven't seen that issue.

What does e.g. the udp_send_and_recv test do on your system? If it fails, can you post the output of strace path/to/run-tests udp_send_and_recv udp_send_and_recv? Maybe add -e \!mmap,munmap,mprotect,brk,futex,gettimeofday,nanosleep to strace to filter out irrelevant system calls.

[bnoordhuis]

No follow-up in almost two weeks. Closing.

[cjdelisle]

Sorry about that, the guy who had the problem went on vacation and at this point I don't care about the issue either.

[bnoordhuis]

Okay, no problem. If you or the other guy wants to pick this up again, shout and I'll reopen the issue.

[Fusl]

=============================================================
[%  28|+  53|-   0|T   0|S   0]: udp_send_and_recv
`udp_send_and_recv` failed: exit code 6
Output from process `udp_send_and_recv`:
Assertion failed in test/test-udp-send-and-recv.c on line 178: r == 0
=============================================================
[%  29|+  53|-   1|T   0|S   0]: udp_dgram_too_big
`udp_dgram_too_big` failed: exit code 6
Output from process `udp_dgram_too_big`:
Assertion failed in test/test-udp-dgram-too-big.c on line 75: r == 0
=============================================================
[%  29|+  53|-   2|T   0|S   0]: udp_dual_stack
`udp_dual_stack` failed: exit code 6
Output from process `udp_dual_stack`:
Assertion failed in test/test-udp-ipv6.c on line 109: r == 0
=============================================================
[%  30|+  53|-   3|T   0|S   0]: udp_ipv6_only
`udp_ipv6_only` failed: exit code 6
Output from process `udp_ipv6_only`:
Assertion failed in test/test-udp-ipv6.c on line 109: r == 0
=============================================================
[%  30|+  53|-   4|T   0|S   0]: udp_options
`udp_options` failed: exit code 6
Output from process `udp_options`:
Assertion failed in test/test-udp-options.c on line 44: r == 0
=============================================================
[%  31|+  53|-   5|T   0|S   0]: udp_multicast_join
`udp_multicast_join` failed: exit code 6
Output from process `udp_multicast_join`:
Assertion failed in test/test-udp-multicast-join.c on line 112: r == 0
=============================================================
[%  31|+  53|-   6|T   0|S   0]: udp_multicast_ttl
`udp_multicast_ttl` failed: exit code 6
Output from process `udp_multicast_ttl`:
Assertion failed in test/test-udp-multicast-ttl.c on line 66: r == 0
=============================================================
[%  32|+  53|-   7|T   0|S   0]: udp_open
`udp_open` failed: exit code 6
Output from process `udp_open`:
uv_udp_recv_start: unknown error
Output from process `udp_open`:
Assertion failed in test/test-udp-open.c on line 136: r == 0
=============================================================
[%  52|+  89|-   8|T   0|S   0]: udp_ref3
`udp_ref3` failed: exit code 6
Output from process `udp_ref3`:
uv_udp_recv_start: unknown error
Output from process `udp_ref3`:
Assertion failed in test/test-ref.c on line 322: req_cb_called == 1
=============================================================
[%  62|+ 108|-   9|T   0|S   0]: getsockname_udp
`getsockname_udp` todo
Output from process `getsockname_udp`:
Bind error
=============================================================
[%  74|+ 128|-   9|T   1|S   0]: spawn_setuid_setgid
`spawn_setuid_setgid` failed: exit code 6
Output from process `spawn_setuid_setgid`:
exit_cb
Assertion failed in test/test-spawn.c on line 58: exit_status == 1
=============================================================
[%  81|+ 140|-  10|T   1|S   0]: fs_chown
`fs_chown` failed: exit code 6
Output from process `fs_chown`:
Assertion failed in test/test-fs.c on line 200: req->result == -1
=============================================================
[% 100|+ 174|-  11|T   1|S   0]: Done.
make: *** [test] Error 11

root@cjdnstest:/opt/cjdns/build/libuv# strace ./run-tests udp_send_and_recv
execve("./run-tests", ["./run-tests", "udp_send_and_recv"], [/* 16 vars */]) = 0
brk(0)                                  = 0x1040000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fa66341d000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=12581, ...}) = 0
mmap(NULL, 12581, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fa663419000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360>\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=530736, ...}) = 0
mmap(NULL, 2625768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fa662f7e000
mprotect(0x7fa662fff000, 2093056, PROT_NONE) = 0
mmap(0x7fa6631fe000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x80000) = 0x7fa6631fe000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\r\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=14768, ...}) = 0
mmap(NULL, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fa662d7a000
mprotect(0x7fa662d7c000, 2097152, PROT_NONE) = 0
mmap(0x7fa662f7c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7fa662f7c000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/librt.so.1", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220!\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=31744, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fa663418000
mmap(NULL, 2128856, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fa662b72000
mprotect(0x7fa662b79000, 2093056, PROT_NONE) = 0
mmap(0x7fa662d78000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7fa662d78000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\\\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=131107, ...}) = 0
mmap(NULL, 2208672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fa662956000
mprotect(0x7fa66296d000, 2093056, PROT_NONE) = 0
mmap(0x7fa662b6c000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16000) = 0x7fa662b6c000
mmap(0x7fa662b6e000, 13216, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fa662b6e000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\357\1\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1595408, ...}) = 0
mmap(NULL, 3709016, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fa6625cc000
mprotect(0x7fa66274c000, 2097152, PROT_NONE) = 0
mmap(0x7fa66294c000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x180000) = 0x7fa66294c000
mmap(0x7fa662951000, 18520, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fa662951000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fa663417000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fa663416000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fa663415000
arch_prctl(ARCH_SET_FS, 0x7fa663416700) = 0
mprotect(0x7fa66294c000, 16384, PROT_READ) = 0
mprotect(0x7fa662b6c000, 4096, PROT_READ) = 0
mprotect(0x7fa662d78000, 4096, PROT_READ) = 0
mprotect(0x7fa662f7c000, 4096, PROT_READ) = 0
mprotect(0x7fa6631fe000, 4096, PROT_READ) = 0
mprotect(0x7fa66341f000, 4096, PROT_READ) = 0
munmap(0x7fa663419000, 12581)           = 0
set_tid_address(0x7fa6634169d0)         = 15787
set_robust_list(0x7fa6634169e0, 0x18)   = 0
futex(0x7fff8f6ad94c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, NULL, 7fa663416700) = -1 EAGAIN (Resource temporarily unavailable)
rt_sigaction(SIGRTMIN, {0x7fa66295bad0, [], SA_RESTORER|SA_SIGINFO, 0x7fa662965030}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7fa66295bb60, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7fa662965030}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=10240*1024, rlim_max=RLIM_INFINITY}) = 0
rt_sigaction(SIGPIPE, {SIG_IGN, [PIPE], SA_RESTORER|SA_RESTART, 0x7fa6625fe4f0}, {SIG_DFL, [], 0}, 8) = 0
brk(0)                                  = 0x1040000
brk(0x1061000)                          = 0x1061000
unlink("/tmp/uv-test-sock")             = -1 ENOENT (No such file or directory)
nanosleep({0, 250000000}, NULL)         = 0
stat("/tmp", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=4096, ...}) = 0
open("/tmp/tmpf8EjUsC", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
unlink("/tmp/tmpf8EjUsC")               = 0
fcntl(3, F_GETFL)                       = 0x8002 (flags O_RDWR|O_LARGEFILE)
fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fa66341c000
lseek(3, 0, SEEK_CUR)                   = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fa6634169d0) = 15788
pipe([4, 5])                            = 0
mmap(NULL, 10489856, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7fa661bcb000
mprotect(0x7fa661bcb000, 4096, PROT_NONE) = 0
clone(child_stack=0x7fa6625caff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7fa6625cb9d0, tls=0x7fa6625cb700, child_tidptr=0x7fa6625cb9d0) = 15789
select(5, [4], NULL, NULL, {5, 0})      = ? ERESTARTNOHAND (To be restarted)
--- SIGCHLD (Child exited) @ 0 (0) ---
select(5, [4], NULL, NULL, {4, 999409}) = 1 (in [4], left {4, 999407})
close(4)                                = 0
close(5)                                = 0
write(2, "\n`udp_send_and_recv` failed: exi"..., 41
`udp_send_and_recv` failed: exit code 6
) = 41
fstat(3, {st_mode=S_IFREG|0600, st_size=70, ...}) = 0
write(2, "Output from process `udp_send_an"..., 41Output from process `udp_send_and_recv`:
) = 41
lseek(3, 0, SEEK_SET)                   = 0
read(3, "Assertion failed in test/test-ud"..., 4096) = 70
write(2, "Assertion failed in test/test-ud"..., 70Assertion failed in test/test-udp-send-and-recv.c on line 178: r == 0
) = 70
read(3, "", 4096)                       = 0
write(2, "================================"..., 62=============================================================
) = 62
close(3)                                = 0
munmap(0x7fa66341c000, 4096)            = 0
exit_group(6)                           = ?

root@cjdnstest:/opt/cjdns/build/libuv# strace ./run-tests udp_send_and_recv -e \!mmap,munmap,mprotect,brk,futex,gettimeofday,nanosleep
execve("./run-tests", ["./run-tests", "udp_send_and_recv", "-e", "!mmap,munmap,mprotect,brk,futex,"...], [/* 16 vars */]) = 0
brk(0)                                  = 0xa78000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f2d55647000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=12581, ...}) = 0
mmap(NULL, 12581, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f2d55643000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360>\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=530736, ...}) = 0
mmap(NULL, 2625768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f2d551a8000
mprotect(0x7f2d55229000, 2093056, PROT_NONE) = 0
mmap(0x7f2d55428000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x80000) = 0x7f2d55428000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\r\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=14768, ...}) = 0
mmap(NULL, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f2d54fa4000
mprotect(0x7f2d54fa6000, 2097152, PROT_NONE) = 0
mmap(0x7f2d551a6000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f2d551a6000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/librt.so.1", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220!\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=31744, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f2d55642000
mmap(NULL, 2128856, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f2d54d9c000
mprotect(0x7f2d54da3000, 2093056, PROT_NONE) = 0
mmap(0x7f2d54fa2000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7f2d54fa2000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\\\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=131107, ...}) = 0
mmap(NULL, 2208672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f2d54b80000
mprotect(0x7f2d54b97000, 2093056, PROT_NONE) = 0
mmap(0x7f2d54d96000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16000) = 0x7f2d54d96000
mmap(0x7f2d54d98000, 13216, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f2d54d98000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\357\1\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1595408, ...}) = 0
mmap(NULL, 3709016, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f2d547f6000
mprotect(0x7f2d54976000, 2097152, PROT_NONE) = 0
mmap(0x7f2d54b76000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x180000) = 0x7f2d54b76000
mmap(0x7f2d54b7b000, 18520, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f2d54b7b000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f2d55641000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f2d55640000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f2d5563f000
arch_prctl(ARCH_SET_FS, 0x7f2d55640700) = 0
mprotect(0x7f2d54b76000, 16384, PROT_READ) = 0
mprotect(0x7f2d54d96000, 4096, PROT_READ) = 0
mprotect(0x7f2d54fa2000, 4096, PROT_READ) = 0
mprotect(0x7f2d551a6000, 4096, PROT_READ) = 0
mprotect(0x7f2d55428000, 4096, PROT_READ) = 0
mprotect(0x7f2d55649000, 4096, PROT_READ) = 0
munmap(0x7f2d55643000, 12581)           = 0
set_tid_address(0x7f2d556409d0)         = 15791
set_robust_list(0x7f2d556409e0, 0x18)   = 0
futex(0x7fff31d0db5c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, NULL, 7f2d55640700) = -1 EAGAIN (Resource temporarily unavailable)
rt_sigaction(SIGRTMIN, {0x7f2d54b85ad0, [], SA_RESTORER|SA_SIGINFO, 0x7f2d54b8f030}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7f2d54b85b60, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7f2d54b8f030}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=10240*1024, rlim_max=RLIM_INFINITY}) = 0
rt_sigaction(SIGPIPE, {SIG_IGN, [PIPE], SA_RESTORER|SA_RESTART, 0x7f2d548284f0}, {SIG_DFL, [], 0}, 8) = 0
brk(0)                                  = 0xa78000
brk(0xa99000)                           = 0xa99000
write(2, "Too many arguments.\n", 20Too many arguments.
)   = 20
exit_group(1)                           = ?

This was on a Debian Wheezy system with linux-libc-dev upgraded to version 3.10.5-1~bpo70+1

[Fusl]

After i downgrade the package to 3.2.46-1, the tests completes almost successfully:

=============================================================
[%  74|+ 138|-   0|T   0|S   0]: spawn_setuid_setgid
`spawn_setuid_setgid` failed: exit code 6
Output from process `spawn_setuid_setgid`:
exit_cb
Assertion failed in test/test-spawn.c on line 58: exit_status == 1
=============================================================
[%  81|+ 150|-   1|T   0|S   0]: fs_chown
`fs_chown` failed: exit code 6
Output from process `fs_chown`:
Assertion failed in test/test-fs.c on line 200: req->result == -1
=============================================================
[% 100|+ 184|-   2|T   0|S   0]: Done.
make: *** [test] Error 2

(I think these errors are caused by running the tests in an OpenVZ-Container)

And testing works as expected:

root@cjdnstest:/opt/cjdns/build/libuv# strace ./run-tests udp_send_and_recv
execve("./run-tests", ["./run-tests", "udp_send_and_recv"], [/* 16 vars */]) = 0
brk(0)                                  = 0x20ae000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9909234000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=12581, ...}) = 0
mmap(NULL, 12581, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f9909230000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360>\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=530736, ...}) = 0
mmap(NULL, 2625768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9908d95000
mprotect(0x7f9908e16000, 2093056, PROT_NONE) = 0
mmap(0x7f9909015000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x80000) = 0x7f9909015000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\r\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=14768, ...}) = 0
mmap(NULL, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9908b91000
mprotect(0x7f9908b93000, 2097152, PROT_NONE) = 0
mmap(0x7f9908d93000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f9908d93000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/librt.so.1", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220!\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=31744, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f990922f000
mmap(NULL, 2128856, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9908989000
mprotect(0x7f9908990000, 2093056, PROT_NONE) = 0
mmap(0x7f9908b8f000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7f9908b8f000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\\\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=131107, ...}) = 0
mmap(NULL, 2208672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f990876d000
mprotect(0x7f9908784000, 2093056, PROT_NONE) = 0
mmap(0x7f9908983000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16000) = 0x7f9908983000
mmap(0x7f9908985000, 13216, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f9908985000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\357\1\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1595408, ...}) = 0
mmap(NULL, 3709016, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f99083e3000
mprotect(0x7f9908563000, 2097152, PROT_NONE) = 0
mmap(0x7f9908763000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x180000) = 0x7f9908763000
mmap(0x7f9908768000, 18520, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f9908768000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f990922e000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f990922d000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f990922c000
arch_prctl(ARCH_SET_FS, 0x7f990922d700) = 0
mprotect(0x7f9908763000, 16384, PROT_READ) = 0
mprotect(0x7f9908983000, 4096, PROT_READ) = 0
mprotect(0x7f9908b8f000, 4096, PROT_READ) = 0
mprotect(0x7f9908d93000, 4096, PROT_READ) = 0
mprotect(0x7f9909015000, 4096, PROT_READ) = 0
mprotect(0x7f9909236000, 4096, PROT_READ) = 0
munmap(0x7f9909230000, 12581)           = 0
set_tid_address(0x7f990922d9d0)         = 20638
set_robust_list(0x7f990922d9e0, 0x18)   = 0
futex(0x7fff7ea871fc, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, NULL, 7f990922d700) = -1 EAGAIN (Resource temporarily unavailable)
rt_sigaction(SIGRTMIN, {0x7f9908772ad0, [], SA_RESTORER|SA_SIGINFO, 0x7f990877c030}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7f9908772b60, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7f990877c030}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=10240*1024, rlim_max=RLIM_INFINITY}) = 0
rt_sigaction(SIGPIPE, {SIG_IGN, [PIPE], SA_RESTORER|SA_RESTART, 0x7f99084154f0}, {SIG_DFL, [], 0}, 8) = 0
brk(0)                                  = 0x20ae000
brk(0x20cf000)                          = 0x20cf000
unlink("/tmp/uv-test-sock")             = -1 ENOENT (No such file or directory)
nanosleep({0, 250000000}, NULL)         = 0
stat("/tmp", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=4096, ...}) = 0
open("/tmp/tmpfOd2Nga", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
unlink("/tmp/tmpfOd2Nga")               = 0
fcntl(3, F_GETFL)                       = 0x8002 (flags O_RDWR|O_LARGEFILE)
fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9909233000
lseek(3, 0, SEEK_CUR)                   = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f990922d9d0) = 20639
pipe([4, 5])                            = 0
mmap(NULL, 10489856, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f99079e2000
mprotect(0x7f99079e2000, 4096, PROT_NONE) = 0
clone(child_stack=0x7f99083e1ff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f99083e29d0, tls=0x7f99083e2700, child_tidptr=0x7f99083e29d0) = 20640
select(5, [4], NULL, NULL, {5, 0})      = ? ERESTARTNOHAND (To be restarted)
--- SIGCHLD (Child exited) @ 0 (0) ---
select(5, [4], NULL, NULL, {4, 999520}) = 1 (in [4], left {4, 999518})
close(4)                                = 0
close(5)                                = 0
close(3)                                = 0
munmap(0x7f9909233000, 4096)            = 0
exit_group(0)                           = ?

root@cjdnstest:/opt/cjdns/build/libuv# strace ./run-tests udp_send_and_recv -e \!mmap,munmap,mprotect,brk,futex,gettimeofday,nanosleep
execve("./run-tests", ["./run-tests", "udp_send_and_recv", "-e", "!mmap,munmap,mprotect,brk,futex,"...], [/* 16 vars */]) = 0
brk(0)                                  = 0x15af000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9c21328000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=12581, ...}) = 0
mmap(NULL, 12581, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f9c21324000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360>\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=530736, ...}) = 0
mmap(NULL, 2625768, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9c20e89000
mprotect(0x7f9c20f0a000, 2093056, PROT_NONE) = 0
mmap(0x7f9c21109000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x80000) = 0x7f9c21109000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340\r\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=14768, ...}) = 0
mmap(NULL, 2109696, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9c20c85000
mprotect(0x7f9c20c87000, 2097152, PROT_NONE) = 0
mmap(0x7f9c20e87000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f9c20e87000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/librt.so.1", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220!\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=31744, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9c21323000
mmap(NULL, 2128856, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9c20a7d000
mprotect(0x7f9c20a84000, 2093056, PROT_NONE) = 0
mmap(0x7f9c20c83000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7f9c20c83000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0@\\\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=131107, ...}) = 0
mmap(NULL, 2208672, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9c20861000
mprotect(0x7f9c20878000, 2093056, PROT_NONE) = 0
mmap(0x7f9c20a77000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16000) = 0x7f9c20a77000
mmap(0x7f9c20a79000, 13216, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f9c20a79000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\357\1\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1595408, ...}) = 0
mmap(NULL, 3709016, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f9c204d7000
mprotect(0x7f9c20657000, 2097152, PROT_NONE) = 0
mmap(0x7f9c20857000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x180000) = 0x7f9c20857000
mmap(0x7f9c2085c000, 18520, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f9c2085c000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9c21322000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9c21321000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9c21320000
arch_prctl(ARCH_SET_FS, 0x7f9c21321700) = 0
mprotect(0x7f9c20857000, 16384, PROT_READ) = 0
mprotect(0x7f9c20a77000, 4096, PROT_READ) = 0
mprotect(0x7f9c20c83000, 4096, PROT_READ) = 0
mprotect(0x7f9c20e87000, 4096, PROT_READ) = 0
mprotect(0x7f9c21109000, 4096, PROT_READ) = 0
mprotect(0x7f9c2132a000, 4096, PROT_READ) = 0
munmap(0x7f9c21324000, 12581)           = 0
set_tid_address(0x7f9c213219d0)         = 20642
set_robust_list(0x7f9c213219e0, 0x18)   = 0
futex(0x7fff15f4de8c, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, NULL, 7f9c21321700) = -1 EAGAIN (Resource temporarily unavailable)
rt_sigaction(SIGRTMIN, {0x7f9c20866ad0, [], SA_RESTORER|SA_SIGINFO, 0x7f9c20870030}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7f9c20866b60, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7f9c20870030}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=10240*1024, rlim_max=RLIM_INFINITY}) = 0
rt_sigaction(SIGPIPE, {SIG_IGN, [PIPE], SA_RESTORER|SA_RESTART, 0x7f9c205094f0}, {SIG_DFL, [], 0}, 8) = 0
brk(0)                                  = 0x15af000
brk(0x15d0000)                          = 0x15d0000
write(2, "Too many arguments.\n", 20Too many arguments.
)   = 20
exit_group(1)                           = ?

[cjdelisle]

@Fusl thanks for getting the info, it looks like this test forks and you missed the good stuff :(
try strace -f -e \!mmap,munmap,mprotect,brk,futex,gettimeofday,nanosleep ./run-tests udp_send_and_recv

@bnoordhuis You're not going to see this failure in tests until you build with a new version of libc-dev and an older kernel so if you're maytagging your binary to different machines to test it will never show up until you upgrade your build machine.

[Fusl]

It's now building correctly. Thanks. Can be closed now 👍
/Edit: I'll do the strace asap

[bnoordhuis]

Thanks for following up. Should be fixed as of 17452cd.

[cjdelisle]

I just changed the patch to a backport of your patch so we should be seeing test results from building on a reasonable variety of machines soon.
cjdelisle/cjdns@dfd2d77