Add SO_REUSEPORT for Android by SimonSapin · Pull Request #70 · rust-lang/libc

SimonSapin · 2015-11-22T13:30:36Z

The net2 crate currently fails to cross-compile to Android with error: unresolved name libc::SO_REUSEPORT``

According to http://stackoverflow.com/a/14388707/1162888, Android is essentially the same as Linux for the purpose of socket features support, and the Linux kernel has supported SO_REUSEPORT since version 3.9.

According to http://android.stackexchange.com/a/51656, Android has had more recent kernels than that since version 4.4 Kit Kat.

(This blocks servo/servo#8446.)

alexcrichton · 2015-11-22T18:17:32Z

The CI here is failing which'll prevent this from landing, and I'd prefer o have constants like this verified on Android. If it's actually present then are there more recent Android compilers and/or headers which have the constants?

SimonSapin · 2015-11-22T19:09:23Z

I’m not sure. FWIW, grep on my system finds SO_REUSEPORT under /opt/android-ndk/platforms/android-21, but not under android-19/ or earlier versions.

If in doubt, would you rather have the reuse_port methods on net2 not be there on android? I’d like to make net2 compile somehow.

alexcrichton · 2015-11-22T19:26:27Z

Ok, I've updated our docker images to use API level 21 instead of 18 (I at least don't know of a reason to use an older one right now) and I'm in the process of uploading the new image tagged as 2015-11-22, and if you update this line to the new tag it should pick it up automatically and enable verification.

Note that it takes awhile to upload, so it may be ~15min before the image is available (travis may take awhile to get around to it though)

rust-lang#70 (comment)

SimonSapin · 2015-11-22T21:03:45Z

I’ve pushed a new commit but https://travis-ci.org/rust-lang-nursery/libc/requests doesn’t show a request for a new build.

alexcrichton · 2015-11-23T04:28:06Z

Hm it looks like this PR doesn't yet include that change? (but seems referenced?)

rust-lang#70 (comment)

SimonSapin · 2015-11-23T07:13:42Z

I pushed to the wrong branch :)

rust-lang#70 (comment)

SimonSapin · 2015-11-23T13:43:09Z

https://travis-ci.org/rust-lang-nursery/libc/builds/92660417 failed with

Unable to find image 'alexcrichton/rust-slave-android:2015-11-22' locally
2015-11-22: Pulling from alexcrichton/rust-slave-android

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.

Pushing the same commit again to trigger another build.

SimonSapin · 2015-11-23T16:33:47Z

Same again. Any suggestion @alexcrichton ? 2015-11-22 does show up in https://hub.docker.com/r/alexcrichton/rust-slave-android/tags/ Could the 10 min limit be hit while Travis is downloading a 2 GB image?

alexcrichton · 2015-11-23T17:16:18Z

I hit restart on this build but it may be time to use travis_wait when running docker

alexcrichton · 2015-11-24T00:11:09Z

Looks like there was some trouble finding travis_wait?

SimonSapin · 2015-11-24T01:13:56Z

Yeah, it looks like what I did is not correct usage, but I don’t know what is. Is travis_wait a magic string rather than an executable?

SimonSapin · 2015-11-24T12:49:11Z

After trying a few random places to insert travis_wait:

The command "if [[ $TRAVIS_RUST_VERSION = nightly* ]]; then travis_wait sh ci/run-travis.sh; else cargo build; fi" exited with 1.
Log:

[…]

exec sh ci/run.sh arm-linux-androideabi
cannot enable tty mode on non tty input
/home/travis/build.sh: line 112:  5330 Terminated              travis_jigger $! $timeout $cmd

The only methods it define require libc::SO_REUSEPORT, which current versions on the libc crate does not provide for Android targets. This is a temporary work around to make this crate build at all on Android. It should be reverted when rust-lang/libc#70 lands.

The only methods it define require libc::SO_REUSEPORT, which current versions on the libc crate does not provide for Android targets. This is a temporary work around to make this crate build at all on Android. It should be reverted when rust-lang/libc#70 lands. This not a breaking change since the crate did not build at all before.

alexcrichton · 2015-11-25T05:08:46Z

I believe this is just because docker is being run with -t which says "Allocate a pseudo-TTY" which apparently travis_wait doesn't like. I'm gonna see if dropping that works -- #74.

SimonSapin · 2015-11-25T14:51:59Z

Trying a variation of #74:

before_script:
  - if [ "$TARGET" = "arm-linux-androideabi" ]; then
      travis_wait docker pull $(cat android-docker-image);
    fi

Timeout (20 minutes) reached. Terminating "docker pull alexcrichton/rust-slave-android:2015-11-22"

rust-lang#70 (comment)

SimonSapin · 2015-11-25T16:06:44Z

So I replaced travis_wait with strace -f and the raw log ends with:

The log length has exceeded the limit of 4 MB (this usually means that the test suite is raising the same exception over and over).
The job has been terminated

… but the job ran for 47 min 41 sec already, so even with another trick to have less output it’s likely to hit the 50 min timeout.

SimonSapin · 2015-11-25T16:20:00Z

The raw log also contains "Layer already being pulled by another client. Waiting."

metajack · 2015-11-25T20:08:41Z

cc @edunham

I think perhaps it's time to bring in our ops team :)

alexcrichton · 2015-12-18T21:51:40Z

Now that I merged #88, could this be rebased? I think it should "Just Work" to add the constant now. (sorry for the runaround!)

SimonSapin · 2015-12-27T22:51:20Z

Rebased.

The `net2` crate currently fails to cross-compile to Android with ``error: unresolved name `libc::SO_REUSEPORT` `` According to http://stackoverflow.com/a/14388707/1162888, Android is essentially the same as Linux for the purpose of socket features support, and the Linux kernel has supported `SO_REUSEPORT` since version 3.9. According to http://android.stackexchange.com/a/51656, Android has had more recent kernels than that since version 4.4 Kit Kat.

SimonSapin · 2015-12-28T07:22:40Z

Reverted version number bump, as it made tests fail to find libc at that version (presumably from crates.io?)

Travis failure on MIPS doesn’t looks related to this change?

error: linking with `mips-linux-gnu-gcc` failed: exit code: 1

note: "mips-linux-gnu-gcc" "-Wl,--as-needed" "-L" "/home/travis/rust/lib/rustlib/mips-unknown-linux-gnu/lib" "/home/travis/build/rust-lang-nursery/libc/libc-test/target/mips-unknown-linux-gnu/debug/libc_test.0.o" "-o" "/home/travis/build/rust-lang-nursery/libc/libc-test/target/mips-unknown-linux-gnu/debug/libc_test" "-Wl,--gc-sections" "-pie" "-nodefaultlibs" "-L" "/home/travis/build/rust-lang-nursery/libc/libc-test/target/mips-unknown-linux-gnu/debug" "-L" "/home/travis/build/rust-lang-nursery/libc/libc-test/target/mips-unknown-linux-gnu/debug/deps" "-L" "/home/travis/build/rust-lang-nursery/libc/libc-test/target/mips-unknown-linux-gnu/debug/build/libc-test-a6cd229ea2045071/out" "-L" "/home/travis/rust/lib/rustlib/mips-unknown-linux-gnu/lib" "-Wl,-Bstatic" "-Wl,--whole-archive" "-l" "all" "-Wl,--no-whole-archive" "-Wl,-Bdynamic" "/home/travis/build/rust-lang-nursery/libc/libc-test/target/mips-unknown-linux-gnu/debug/deps/liblibc-f8fceda7be8f9ea9.rlib" "/home/travis/rust/lib/rustlib/mips-unknown-linux-gnu/lib/libstd-17a8ccbd.rlib" "/home/travis/rust/lib/rustlib/mips-unknown-linux-gnu/lib/libcollections-17a8ccbd.rlib" "/home/travis/rust/lib/rustlib/mips-unknown-linux-gnu/lib/librustc_unicode-17a8ccbd.rlib" "/home/travis/rust/lib/rustlib/mips-unknown-linux-gnu/lib/librand-17a8ccbd.rlib" "/home/travis/rust/lib/rustlib/mips-unknown-linux-gnu/lib/liballoc-17a8ccbd.rlib" "/home/travis/rust/lib/rustlib/mips-unknown-linux-gnu/lib/liballoc_jemalloc-17a8ccbd.rlib" "/home/travis/rust/lib/rustlib/mips-unknown-linux-gnu/lib/liblibc-17a8ccbd.rlib" "/home/travis/rust/lib/rustlib/mips-unknown-linux-gnu/lib/libcore-17a8ccbd.rlib" "-l" "dl" "-l" "pthread" "-l" "gcc_s" "-l" "pthread" "-l" "c" "-l" "m" "-l" "rt" "-l" "compiler-rt"

note: /usr/lib/gcc/mips-linux-gnu/4.4.5/../../../../mips-linux-gnu/bin/ld: Warning: /home/travis/build/rust-lang-nursery/libc/libc-test/target/mips-unknown-linux-gnu/debug/libc_test uses hard float, /home/travis/rust/lib/rustlib/mips-unknown-linux-gnu/lib/libstd-17a8ccbd.rlib(fileline.o) uses soft float

/usr/lib/gcc/mips-linux-gnu/4.4.5/../../../../mips-linux-gnu/bin/ld: Warning: /home/travis/build/rust-lang-nursery/libc/libc-test/target/mips-unknown-linux-gnu/debug/libc_test uses hard float, /home/travis/rust/lib/rustlib/mips-unknown-linux-gnu/lib/libstd-17a8ccbd.rlib(posix.o) uses soft float

...

/usr/lib/gcc/mips-linux-gnu/4.4.5/../../../../mips-linux-gnu/bin/ld: Warning: /home/travis/build/rust-lang-nursery/libc/libc-test/target/mips-unknown-linux-gnu/debug/libc_test uses hard float, /home/travis/rust/lib/rustlib/mips-unknown-linux-gnu/lib/libcompiler-rt.a(mulodi4.o) uses soft float

/usr/lib/gcc/mips-linux-gnu/4.4.5/../../../../mips-linux-gnu/bin/ld: Dwarf Error: found dwarf version '4', this reader only handles version 2 and 3 information.

/usr/lib/gcc/mips-linux-gnu/4.4.5/../../../../mips-linux-gnu/bin/ld: Dwarf Error: found dwarf version '0', this reader only handles version 2 and 3 information.

/home/travis/rust/lib/rustlib/mips-unknown-linux-gnu/lib/liballoc_jemalloc-17a8ccbd.rlib(jemalloc.pic.o): In function `malloc_conf_init':

/buildslave/rust-buildbot/slave/nightly-dist-rustc-cross-linux/build/src/jemalloc/src/jemalloc.c:(.text.malloc_conf_init+0x94): undefined reference to `secure_getenv'

/usr/lib/gcc/mips-linux-gnu/4.4.5/../../../../mips-linux-gnu/bin/ld: Dwarf Error: found dwarf version '0', this reader only handles version 2 and 3 information.

/usr/lib/gcc/mips-linux-gnu/4.4.5/../../../../mips-linux-gnu/bin/ld: Dwarf Error: found dwarf version '3072', this reader only handles version 2 and 3 information.

/buildslave/rust-buildbot/slave/nightly-dist-rustc-cross-linux/build/src/jemalloc/src/jemalloc.c:(.text.malloc_conf_init+0x98): undefined reference to `secure_getenv'

collect2: ld returned 1 exit status

error: aborting due to previous error

Could not compile `libc-test`.

* Docs: Fix typo in module documentation s/paltform/platform/g * [x86] Implement sse4.2 string cmp intrinsics - Implement - _mm_cmpistrm - _mm_cmpistri - _mm_cmpistrz - _mm_cmpistrc - _mm_cmpistrs - _mm_cmpistro - _mm_cmpistra - _mm_cmpestrm - _mm_cmpestrz - _mm_cmpestrc - _mm_cmpestrs - _mm_cmpestro - _mm_cmpestra - Add documentation to _mm_cmpestri - Add missing constants

This was referenced Nov 22, 2015

Tracking issue for Rust high-pri or blocking issues for Servo & Gecko servo/servo#2853

Closed

Add portability hack for Android. deprecrated/net2-rs#21

Merged

SimonSapin added a commit to SimonSapin/libc that referenced this pull request Nov 22, 2015

Update Android APIs to version 21 for CI.

5ac16ed

rust-lang#70 (comment)

SimonSapin added a commit to SimonSapin/libc that referenced this pull request Nov 23, 2015

Update Android APIs to version 21 for CI.

92cf0c4

rust-lang#70 (comment)

SimonSapin force-pushed the SO_REUSEPORT-android branch from 92cf0c4 to eab5dac Compare November 23, 2015 13:41

SimonSapin added a commit to SimonSapin/libc that referenced this pull request Nov 23, 2015

Update Android APIs to version 21 for CI.

eab5dac

rust-lang#70 (comment)

SimonSapin force-pushed the SO_REUSEPORT-android branch from 4eefa8f to 23981ca Compare November 23, 2015 21:26

SimonSapin force-pushed the SO_REUSEPORT-android branch 2 times, most recently from 2cd24f0 to 779ffb4 Compare November 24, 2015 12:42

SimonSapin mentioned this pull request Nov 24, 2015

Disable the unix module on Android. deprecrated/net2-rs#22

Merged

SimonSapin force-pushed the SO_REUSEPORT-android branch 2 times, most recently from 0f07feb to 0240ec7 Compare November 25, 2015 09:21

SimonSapin force-pushed the SO_REUSEPORT-android branch from 535063e to fcc34c9 Compare November 25, 2015 16:04

SimonSapin added a commit to SimonSapin/libc that referenced this pull request Nov 25, 2015

Update Android APIs to version 21 for CI.

77a574e

rust-lang#70 (comment)

This was referenced Dec 1, 2015

Add getxattr()/setxattr() variations #87

Merged

Update the android CI image #88

Merged

SimonSapin force-pushed the SO_REUSEPORT-android branch from fcc34c9 to 71756d0 Compare December 27, 2015 22:50

SimonSapin force-pushed the SO_REUSEPORT-android branch from 71756d0 to 52f4f39 Compare December 27, 2015 23:25

alexcrichton merged commit 52f4f39 into rust-lang:master Jan 11, 2016

hlzhang mentioned this pull request May 22, 2017

revert https://github.com/rust-lang-nursery/net2-rs/pull/22 deprecrated/net2-rs#59

Merged

Comments

Conversation

SimonSapin commented Nov 22, 2015

Uh oh!

alexcrichton commented Nov 22, 2015

Uh oh!

SimonSapin commented Nov 22, 2015

Uh oh!

alexcrichton commented Nov 22, 2015

Uh oh!

SimonSapin commented Nov 22, 2015

Uh oh!

alexcrichton commented Nov 23, 2015

Uh oh!

SimonSapin commented Nov 23, 2015

Uh oh!

SimonSapin commented Nov 23, 2015

Uh oh!

SimonSapin commented Nov 23, 2015

Uh oh!

alexcrichton commented Nov 23, 2015

Uh oh!

alexcrichton commented Nov 24, 2015

Uh oh!

SimonSapin commented Nov 24, 2015

Uh oh!

SimonSapin commented Nov 24, 2015

Uh oh!

alexcrichton commented Nov 25, 2015

Uh oh!

SimonSapin commented Nov 25, 2015

Uh oh!

SimonSapin commented Nov 25, 2015

Uh oh!

SimonSapin commented Nov 25, 2015

Uh oh!

metajack commented Nov 25, 2015

Uh oh!

alexcrichton commented Dec 18, 2015

Uh oh!

SimonSapin commented Dec 27, 2015

Uh oh!

SimonSapin commented Dec 28, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants