Cannot start servo on android

[aydinkim]

With some fixes, I can reached android_start finally with newer rust.

But see error messages below.

12-17 12:50:58.160: I/native-activity(2341): go into android_start()
12-17 12:50:58.160: I/stderr(2341): warning: RUST_LOG set, but no crate map found.
12-17 12:50:58.160: A/libc(2341): Fatal signal 11 (SIGSEGV) at 0xdeadbaad (code=1), thread 2358 (le.ServoAndroid)
12-17 12:50:58.160: I/stderr(2341): task 'Constellation' failed at 'task '' failed at 'Unhandled condition: io_error: io::IoError{kind: IoUnavailable, desc: "I/O is unavailable", detail: None}', /home/aydin.kim/servo/mozilla/aaa/src/compiler/rust/src/libstd/condition.rs:Unhandled condition: io_error: io::IoError{kind: IoUnavailable, desc: "I/O is unavailable", detail: None}', /home/aydin.kim/servo/mozilla/aaa/src/compiler/rust/src/libstd/condition.rs:139

These messages are same case as with previous version of rust.
It had occured after rustuv change on 30 Oct.

In the past, The solution was rewinding the rust version back to before.
I expected that this problem would be solved with rust upgrade, but it doesn't.

Any ideas?

[aydinkim]

Somebody who wants to compile android properly, plz refer below.
(some fixes needed)

https://gist.github.com/aydinkim/8432761

[jdm]

0xdeadbaad is used by the Android libc abort() function when native heap corruption is detected.

[larsbergstrom]

Using a bit of hackery, we're definitely getting the abort because we call rust_fail. Break on abort:

set solib-search-path /Users/larsberg/servo-arm/src/platform/android/servo-android-glue/obj/local/armeabi/:/Users/larsberg/servo-arm/src/platform/android/servo-android-glue/libs/armeabi

#0  0x40085348 in ?? () from /Users/larsberg/servo-arm/src/platform/android/servo-android-glue/obj/local/armeabi/libc.so
#1  0x40083514 in abort () from /Users/larsberg/servo-arm/src/platform/android/servo-android-glue/obj/local/armeabi/libc.so
#2  0x732b84b4 in rt::util::abort::he6adc207374474b1a8::v0.9 () from /Users/larsberg/servo-arm/src/platform/android/servo-android-glue/libs/armeabi/libstd-3e5aeb83-0.9.so
#3  0x73385288 in rust_fail () from /Users/larsberg/servo-arm/src/platform/android/servo-android-glue/libs/armeabi/libstd-3e5aeb83-0.9.so

Break on rust_fail:

#0  0x73385184 in rust_fail () from /Users/larsberg/servo-arm/src/platform/android/servo-android-glue/libs/armeabi/libstd-3e5aeb83-0.9.so
#1  0x732b8680 in rt::unwind::Unwinder::begin_unwind::h6941c4780c89dc86nmaq::v0.9 () from /Users/larsberg/servo-arm/src/platform/android/servo-android-glue/libs/armeabi/libstd-3e5aeb83-0.9.so
#2  0x732b8680 in rt::unwind::Unwinder::begin_unwind::h6941c4780c89dc86nmaq::v0.9 () from /Users/larsberg/servo-arm/src/platform/android/servo-android-glue/libs/armeabi/libstd-3e5aeb83-0.9.so

[larsbergstrom]

OK, this is the cause of the task failure:
V/stderr (21556): task 'Constellation' failed at 'Unhandled condition: io_error: io::IoError{kind: IoUnavailable, desc: "I/O is unavailable", detail: None}', /Users/larsberg/servo-arm/src/compiler/rust/src/libstd/condition.rs:139

[larsbergstrom]

With help from @acrichto I was able to work around this. Basically, the problem is that we are unable to load the crate map and so libuv crashes with the error above. The code should just work when we statically link, but I also have some patches to libstd that fix it.

The new issue is:

#0  0x00000000 in ?? ()
#1  0x74837f58 in glutInit () from /Users/larsberg/servo-arm/src/platform/android/servo-android-glue/libs/armeabi/libglut-f186cebf-0.1.so
#2  0x74824b80 in fn2462 (command=0x75c1a408 "glut") at /Users/larsberg/servo-arm/src/support/glut/rust-glut/glut.rs:104
#3  0x748220d0 in c_str::CString::with_ref::hf50496686244300adnat::v0.1 () at /Users/larsberg/servo-arm/src/support/glut/rust-glut/glut.rs:101
#4  0x74821ee0 in glut::glut::init () at /Users/larsberg/servo-arm/src/support/glut/rust-glut/glut.rs:101
#5  0x74821ee0 in glut::glut::init () at /Users/larsberg/servo-arm/src/support/glut/rust-glut/glut.rs:101

I'll look into it more this weekend as time permits.

[larsbergstrom]

OK, we can load now:
http://www.youtube.com/watch?v=_f7QMRn77_o

Loading remote URLs does not work yet because of:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 28071]
rust_is_ipv4_sockaddr (addr=0x0) at /Users/larsberg/servo-arm/src/compiler/rust/src/rt/rust_uv.c:126
126 return addr->sa_family == AF_INET;
(gdb) bt
#0 rust_is_ipv4_sockaddr (addr=0x0) at /Users/larsberg/servo-arm/src/compiler/rust/src/rt/rust_uv.c:126
#1 0x74a36518 in net::sockaddr_to_socket_addr::h884db1f919054ed8ax::v0.9 ()
from /Users/larsberg/servo-arm/src/platform/android/servo-android-glue/libs/armeabi/librustuv-2ba3695a-0.9.so
#2 0x74a36518 in net::sockaddr_to_socket_addr::h884db1f919054ed8ax::v0.9 ()

[yichoi]

I did not try to load remote URL since I could not find proper way to set system permission to servo.
http://developer.android.com/guide/topics/security/permissions.html

[aydinkim]

@larsbergstrom Great works! Can you share the patch to fix them? I'm not sure that your problem is caused by android permission or not. Until now, we actually implemented servo-android port with local pages. I think it may be simple problem if rust-arm works well. We'll also look into it.

[aydinkim]

It looks that "#link(name = "XXX")" and #[link_args="-lXXX" are different each other in internal operation on android.
I didn't look into rust. Are there any changes?

[larsbergstrom]

@aydinkim https://gist.github.com/larsbergstrom/8513764

There are two main diffs:

1. in rust_crate_map, it was failing to find the crate map because Rust does a dlsym() that scans all loaded libraries for a given symbol, which does not appear to find the appropriate symbol on Android. As a workaround, I just load it from servo, but will work with @acrichto to find a better solution.

2. two copies of libglut were being loaded - one was the one with the hash in the file name (linked against by libservo) and the other was libglut.so. There were two copies of all of the code in memory, and we would load the symbols for the second one but then call code in the first, which would crash because the symbols had not been set in it.

[aydinkim]

@larsbergstrom Thanks. Ok. I understood it. There are duplicated libglut* actually so that it causes liking errors. As you know we need library pointer before rust main trigger because of glut callbacks linking. Until now, we changed hash of library in jni/main.cpp of servo-android-glue by manual, we need to change it more conveniently. I'll try to find a better solution.

[aydinkim]

@larsbergstrom Just on more question, crate map is used only for logging? I remember that there was no runtime crash even if rust runtime doesn't have any crate maps. In my opinion, It is better way that we can launch the program entry in certain library regardless of crate maps presence. Of course the best solution would be automatic create map maker from linked libraries without no hard coding.

[larsbergstrom]

According to @acrichto, the crate map is required by the new libuv framework used by the runtime. If the crate map is not found, then the runtime falls back to a different I/O manager. That I/O manager crashes immediately on Android. I did not check into why, but instead fixed the crate map (since it also fixed logging, which I wanted).

[aydinkim]

@larsbergstrom ok. I modified servo-android-glue. Probably you don't need to know exact name of libglut anymore with this commit - https://github.com/webconvforge/servo-android-glue/commit/d1b83336c8c15006f4bf600f28e0e763c4b03c36

And I had tried to run servo-android according to your diff(https://gist.github.com/larsbergstrom/8513764), I failed due to same error(no crate map). Do you think I need to do something more?

[larsbergstrom]

@aydinkim Did you force all of rust to rebuild (e.g., "touch arm-linux-androideabi/src/compiler/rust/rust-auto-clean-stamp") to get a new libstd and then did the build scripts pick up that .so file and generate a new APK file? I've had trouble sometimes with unreliable rebuilds.

[aydinkim]

@larsbergstrom Yeap. I had removed all build directory and rebuilded it.

01-21 10:28:15.100: I/native-activity(6132): go into android_start()
01-21 10:28:15.100: V/stdout(6132): Loading libservo.so
01-21 10:28:15.100: V/stdout(6132): Loading crate map symbol "_rust_crate_map_toplevel"

I think symbols had been loaded, but orrurs

01-21 10:28:15.110: V/stderr(6132): task '' failed at 'Unhandled condition: io_error: io::IoError{kind: IoUnavailable, desc: "I/O is unavailable", detail: None}', /home/aydin.kim/servo/mozilla/aaa/src/compiler/rust/src/libstd/condition.rs:139

[aydinkim]

@larsbergstrom librustuv is needed now? It seems there is no relation to other libraries. we need to load symbols of librustuv to memory manually?

[aydinkim]

I added some logs into crate_map.rs.

12-17 18:23:07.990: V/stdout(4522): Loading libservo.so
12-17 18:23:07.995: V/stdout(4522): Loading crate map symbol "_rust_crate_map_toplevel"
12-17 18:23:07.995: V/stdout(4522): Loaded symbols (0x0 as *())
12-17 18:23:07.995: V/stdout(4522): Crate map point is null!

dl::symbol doesn't work. I will look into it.

[aydinkim]

@larsbergstrom I had checked symbols in libservo. Unfortunately crate_map toplevel is not exist actually. Instead of that, just _rust_crate_map_servo is exist. I'm now trying to rebuild after change to that. Let you know if I would be succeeded.

[aydinkim]

With some modification like below,

--- a/src/librustc/middle/trans/base.rs
+++ b/src/librustc/middle/trans/base.rs
@@ -3044,10 +3044,12 @@ pub fn decl_crate_map(sess: session::Session, mapmeta: LinkMeta,
     let mut n_subcrates = 1;
     let cstore = sess.cstore;
     while cstore.have_crate_data(n_subcrates) { n_subcrates += 1; }
-    let is_top = !sess.building_library.get() || sess.gen_crate_map();
+    let is_top = !sess.building_library.get() || sess.gen_crate_map() || (mapmeta.crateid.name == ~"servo");
     let sym_name = if is_top {
+        println!("This is top level!");
         ~"_rust_crate_map_toplevel"
     } else {
+        println!("This is not top level : {:?}", mapmeta.crateid.name);
         symname(sess, "_rust_crate_map_" + mapmeta.crateid.name, mapmeta.crate_hash,
                 mapmeta.crateid.version_or_default())
     };

I could have a "_rust_crate_map_toplevel" symbol in the libservo.so

But launching was failed with same log.

12-17 23:59:19.260: V/stdout(7155): Loading libservo.so
12-17 23:59:19.260: V/stdout(7155): Loading crate map symbol "_rust_crate_map_toplevel"
12-17 23:59:19.260: V/stdout(7155): Loaded symbols (0x5f259010 as *())
12-17 23:59:19.260: V/stdout(7155): Loading libservo.so
12-17 23:59:19.260: V/stdout(7155): Loading crate map symbol "_rust_crate_map_toplevel"
12-17 23:59:19.260: V/stdout(7155): Loaded symbols (0x5f259010 as *())
12-17 23:59:19.265: V/stderr(7155): task '<unnamed>' failed at 'Unhandled condition: io_error: io::IoError{kind: IoUnavailable, desc: "I/O is unavailable", detail: None}', /home/aydin.kim/servo/mozilla/aaa/src/compiler/rust/src/libstd/condition.rs:139

[aydinkim]

@larsbergstrom In addition, librustuv is not used on android, is it right? I had tried to find but I couldn't. There is no linkage to other libraries at all.. Can you make sure that this is normal or abnormal? Sorry for bother you, but I have fought with this issue all day :( The problem might be from more simple reason that I expected..

[larsbergstrom]

@aydinkim I definitely have the symbol in my binary: https://gist.github.com/larsbergstrom/8478915#file-gistfile1-txt-L707

The only other thing I can think of is that I used the android-18 NDK toolset instead of the android-14 toolset as described in the page that you had created.

Can you get a stack backtrace on where that error is coming from? I have some information on it at: https://github.com/mozilla/servo/wiki/Building-for-Android and then you can use ndk-gdb, set a breakpoint on rust_fail, and you should be able to get a backtrace.

[aydinkim]

@larsbergstrom

without modification of rust, it was like below (no toplevel cate map symbol)
https://gist.github.com/aydinkim/8551546

So I had modified rust for workaround.

diff --git a/src/librustc/middle/trans/base.rs b/src/librustc/middle/trans/base.rs
index ad95344..fe3dad5 100644
--- a/src/librustc/middle/trans/base.rs
+++ b/src/librustc/middle/trans/base.rs
@@ -3044,10 +3044,12 @@ pub fn decl_crate_map(sess: session::Session, mapmeta: LinkMeta,
     let mut n_subcrates = 1;
     let cstore = sess.cstore;
     while cstore.have_crate_data(n_subcrates) { n_subcrates += 1; }
-    let is_top = !sess.building_library.get() || sess.gen_crate_map();
+    let is_top = !sess.building_library.get() || sess.gen_crate_map() || (mapmeta.crateid.name == ~"servo");
     let sym_name = if is_top {
+        println!("This is top level!");
         ~"_rust_crate_map_toplevel"
     } else {
+        println!("This is not top level : {:?}", mapmeta.crateid.name);
         symname(sess, "_rust_crate_map_" + mapmeta.crateid.name, mapmeta.crate_hash,
                 mapmeta.crateid.version_or_default())
     };

Finally I also have that in libservo.
https://gist.github.com/aydinkim/8551533

In addition, I currently use android API-18 and ndk r9.
I need to update the wiki page description. it is out-dated :)

So you mean you don't have any more changes compared to what you posted to me before?
And librustuv is not needed on android, right? (there is no link to other libraries, as I investigated with nm)
Thanks!

[larsbergstrom]

If the _rust_crate_map_toplevel symbol does not appear in your output binary, then maybe your ndk-standalone directory was not made from the android-18 toolset?

[aydinkim]

@larsbergstrom I'm using ndk version below.
lrwxrwxrwx 1 root root 23 11월 19 10:01 /opt/ndk_standalone -> ndk_standalone_r9_api18
lrwxrwxrwx 1 root root 14 10월 25 15:59 /opt/android-ndk -> android-ndk-r9

But I will make sure that again.
You are using api18 & ndk r9(not r9b), is it correct?

[larsbergstrom]

I think I found it - I have a diff here:

diff --git a/Makefile.in b/Makefile.in
index 19a192f..9fa4836 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -328,7 +328,7 @@ servo:      $(DEPS_servo)
 else
 servo:  $(DEPS_servo)
        @$(call E, compile: $@)
-       $(Q)$(RUSTC) $(RFLAGS_servo) -o $@ $< --lib
+       $(Q)$(RUSTC) $(RFLAGS_servo) -o -Z gen-crate-map $@ $< --lib
 endif

 # Darwin app packaging
diff --git a/src/compiler/rust b/src/compiler/rust

[aydinkim]

@larsbergstrom With glancing at rust, it seems similar to my rust code change. Possibly It might be used in llvm.. Anyway I'm trying to rebuild with fix. let you know the result after rebuilded.

[larsbergstrom]

@aydinkim That makefile change is in the servo makefile. It causes rust to emit a crate map in the library libservo-whatever.so.

[aydinkim]

@larsbergstrom Yes. I mean it seems almost same(logically) with my rust modification stuffs before. I couldn't find any other use case of "gen_crate_map". And the result are same. Found _rust_crate_map_toplevel, and crashed with same error logs unfortunately.

aydin.kim@servo64-6:~/servo/mozilla/aaa/build_arm8$ /opt/ndk_standalone/bin/arm-linux-androideabi-nm libservo-c302e9fb-0.0.so | grep crate_map
         U _ZN18_rust_crate_map_js17_7e22fbaef2a15ee84v0.1E
         U _ZN19_rust_crate_map_egl16b98f2d08746627e74v0.1E
         U _ZN19_rust_crate_map_gfx16f3b80f93dfdfd2e44v0.1E
         U _ZN19_rust_crate_map_msg17_4809327a43db67f94v0.1E
         U _ZN19_rust_crate_map_net17_58ba034777e29fcc4v0.1E
         U _ZN19_rust_crate_map_png16ee0d8370bc91809b4v0.1E
         U _ZN19_rust_crate_map_std17_3e5aeb837ae1622e4v0.9E
         U _ZN20_rust_crate_map_geom16b8ee5d95925118a74v0.1E
         U _ZN20_rust_crate_map_glut16f186cebfab0e42f34v0.1E
         U _ZN20_rust_crate_map_http17_9296ff294e0202778v0.1.preE
         U _ZN20_rust_crate_map_util17_825d2436fc9b52d54v0.1E
         U _ZN21_rust_crate_map_alert17_03d2d7b0de894f664v0.1E
         U _ZN21_rust_crate_map_azure17_68f88dae371e90484v0.1E
         U _ZN21_rust_crate_map_extra16fd30a1b1061ba8a94v0.9E
         U _ZN21_rust_crate_map_green17_83b1c0e56a670c004v0.9E
         U _ZN21_rust_crate_map_style17_3177cc7aecc2647d4v0.1E
         U _ZN22_rust_crate_map_hubbub16af0334d01d28bd674v0.1E
         U _ZN22_rust_crate_map_layers16c87f1342bc1891884v0.1E
         U _ZN22_rust_crate_map_native16cf55a53a6897955a4v0.9E
         U _ZN22_rust_crate_map_script17_37d4be8d181582664v0.1E
         U _ZN23_rust_crate_map_sharegl16d1f9f58ddf957e654v0.1E
         U _ZN24_rust_crate_map_encoding17_2b7e299b334784786v0.1.0E
         U _ZN24_rust_crate_map_freetype17_978110fc8119fad24v0.1E
         U _ZN24_rust_crate_map_harfbuzz17_0400c1b13502dbbb4v0.1E
         U _ZN24_rust_crate_map_opengles17_9e9daee33d35560c4v0.1E
         U _ZN25_rust_crate_map_cssparser16b60937539644c8d94v0.1E
         U _ZN25_rust_crate_map_stb_image17_145ba51f85be40a44v0.1E
         U _ZN26_rust_crate_map_fontconfig17_038535be11039ba34v0.1E
00153120 d _crate_map_child_vectors
00153010 D _rust_crate_map_toplevel

and,

12-18 00:47:26.720: V/stderr(2390): task 'task 'Constellation' failed at '<unnamed>' failed at 'Unhandled condition: io_error: io::IoError{kind: IoUnavailable, desc: "I/O is unavailable", detail: None}Unhandled condition: io_error: io::IoError{kind: IoUnavailable, desc: "I/O is unavailable", detail: None}', /home/aydin.kim/servo/mozilla/aaa/src/compiler/rust/src/libstd/condition.rs:139', /home/aydin.kim/servo/mozilla/aaa/src/compiler/rust/src/libstd/condition.rs:139

Should I change the change ndk version to r9b?

[aydinkim]

@larsbergstrom Could you please send me your libservo binary? I tried to build with ndk r9c, I had failed.I want to look into the binary

[aydinkim]

Ok. we need to add some code into servo.rc

• extern mod rustuv;

[larsbergstrom]

The problem is that Rust usually attempts to dynamically inject libuv into the process, but this does not work on Android. This tip was provided in the conversation at:

https://botbot.me/mozilla/rust-internals/msg/9883174/

[larsbergstrom]

@aydinkim are you planning to land these extra changes in the related modules, or should I attempt to do so?

I am trying to get the related Rust bug fixed (about the crate map), but we will need all of these other changes landed as well in order to start working on Android buildbot support for Servo.

[aydinkim]

@larsbergstrom If it doesn't bother you too much, why not? I just have hesitated it because I cannot catch up the rust changes so far. I think we should update rust-mozjs and servo-android-glue.

[larsbergstrom]

@aydinkim OK, if you do not open the PRs yourself, I will create PRs for those two modules tomorrow. Thanks!

[larsbergstrom]

I opened a PR in servo-android-glue with both your prior changes and my changes.

I will check on rust-mozjs once I start in on the Rust upgrade. Hopefully the changes they have been making with libstdc++ and linker flags have fixed the issue we were having.

[larsbergstrom]

Fixed by #2070