New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/mobile: gomobile cause memory issue on android when use with react-native and fresco #20156

Open
narychen opened this Issue Apr 28, 2017 · 16 comments

Comments

Projects
None yet
5 participants
@narychen

narychen commented Apr 28, 2017

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.7.4 darwin/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/nary/Desktop/gopath
GORACE=""
GOROOT="/usr/local/Cellar/go/1.7.4_1/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.7.4_1/libexec/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/f8/73mv3kjn2rxcv3rv_zvs6tn40000gn/T/go-build352177269=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"

What did you do?

use gomobile to create react-native cross platform app

What did you expect to see?

What did you see instead?

It is very easy to crash on android 4.3 when you use gomobile with react-native.
on android 4.4 and 5.0, 6.0 the situation is a little better but it will still lead to memory issue and the pic will never show when you browse a lot.

I create a complete reproducable demo on my github. https://github.com/narychen/ReactNativeListTest

It just print hello call from the gomobile in android/app/src/main/java/com/listtest/MainApplication.java

  public void onCreate() {
    super.onCreate();
    Log.i("gotest", Gotest.greetings("gotest"));  //delete it will make the app work without crash
    SoLoader.init(this, /* native exopackage */ false);
  }

On android 4.3 it will quickly crash and the android log shows

04-28 11:40:42.691 23953-24922/com.listtest D/skia: ---------- mmap failed for imageref_ashmem size=8294400
04-28 11:40:42.901 23953-24890/com.listtest D/dalvikvm: GC_FOR_ALLOC freed 12460K, 30% free 18487K/26372K, paused 18ms, total 19ms
04-28 11:40:42.901 23953-24890/com.listtest I/dalvikvm-heap: Grow heap (frag case) to 23.802MB for 4194320-byte allocation
04-28 11:40:42.931 23953-24922/com.listtest D/dalvikvm: GC_FOR_ALLOC freed 45K, 27% free 22544K/30472K, paused 16ms, total 16ms
04-28 11:40:42.941 23953-24922/com.listtest I/dalvikvm-heap: Grow heap (frag case) to 27.764MB for 4194320-byte allocation
04-28 11:40:42.971 23953-24899/com.listtest D/dalvikvm: GC_FOR_ALLOC freed 135K, 23% free 26645K/34572K, paused 14ms, total 15ms
04-28 11:40:42.981 23953-24899/com.listtest I/dalvikvm-heap: Grow heap (frag case) to 31.770MB for 4194320-byte allocation
04-28 11:40:42.981 23953-24922/com.listtest D/skia: ---------- mmap failed for imageref_ashmem size=8294400
04-28 11:40:42.991 23953-24899/com.listtest D/skia: ---------- mmap failed for imageref_ashmem size=8294400
04-28 11:40:43.021 23953-24917/com.listtest D/dalvikvm: GC_FOR_ALLOC freed 8207K, 26% free 22547K/30468K, paused 16ms, total 16ms
04-28 11:40:43.021 23953-24917/com.listtest I/dalvikvm-heap: Grow heap (frag case) to 27.767MB for 4194320-byte allocation
04-28 11:40:43.031 23953-24917/com.listtest D/skia: ---------- mmap failed for imageref_ashmem size=8294400
04-28 11:40:43.111 23953-24917/com.listtest D/dalvikvm: GC_FOR_ALLOC freed 4380K, 26% free 22556K/30468K, paused 16ms, total 16ms
04-28 11:40:43.121 23953-24917/com.listtest I/dalvikvm-heap: Grow heap (frag case) to 27.776MB for 4194320-byte allocation
04-28 11:40:45.731 23953-24899/com.listtest D/dalvikvm: GC_FOR_ALLOC freed 8307K, 30% free 18594K/26372K, paused 17ms, total 17ms
04-28 11:40:45.741 23953-24899/com.listtest I/dalvikvm-heap: Grow heap (frag case) to 21.907MB for 2097168-byte allocation
04-28 11:40:45.831 23953-24890/com.listtest D/dalvikvm: GC_FOR_ALLOC freed 198K, 28% free 20524K/28424K, paused 15ms, total 15ms
04-28 11:40:45.841 23953-24890/com.listtest I/dalvikvm-heap: Grow heap (frag case) to 23.791MB for 2097168-byte allocation
04-28 11:40:45.941 23953-24922/com.listtest D/dalvikvm: GC_FOR_ALLOC freed 316K, 26% free 22563K/30476K, paused 27ms, total 27ms
04-28 11:40:46.101 23953-23953/com.listtest W/Adreno-GSL: <sharedmem_gpumem_alloc_id:1427>: sharedmem_gpumem_alloc: mmap failed errno 12 Out of memory
04-28 11:40:46.101 23953-23953/com.listtest E/Adreno-GSL: <ioctl_kgsl_sharedmem_alloc:1528>: ioctl_kgsl_sharedmem_alloc: FATAL ERROR : (null)

On android 4.3 and earlier version the facebook fresco module use Ashmem to manage the pic memory. So error shows that 'sharedmem_gpumem_alloc: mmap failed errno 12 Out of memory'.
And if gomobile somewhere conflict with fresco on Ashmem. It may cause this problem.
Nowadays, react-native and fresco is used everywhere. Hope the gomobile team could see this problem and solve it.

When you use the demo, just change the GOPATH in android/gotest/build.gradle and you will compile it. If you are not familiar with react-native, just cd the demo's directory and run command

react-native run-android

or use android studio to compile and run it.
Should be test on real devices.
Thanks.

@narychen

This comment has been minimized.

narychen commented Apr 28, 2017

By the way same code on ios does not has this problem.

@bradfitz bradfitz changed the title from gomobile cause memory issue on android when use with react-native and fresco to x/mobile: gomobile cause memory issue on android when use with react-native and fresco Apr 28, 2017

@gopherbot gopherbot added this to the Unreleased milestone Apr 28, 2017

@eliasnaur

This comment has been minimized.

Contributor

eliasnaur commented Apr 28, 2017

Does the problem disappear on arm64 devices (running with arm64 native code)? You can build an arm64-only binary with gomobile bind -target android/arm64 or by specifying GOARCH="arm64" to the gobind gradle plugin.
If so, I think the problem is that the app runs out of virtual memory because two garbage collected runtimes (Dalvik and Go) and one custom memory allocator (fresco) are fighting over it. It also explains why it works on iOS, whose native runtime is not garbage collected and presumably less demanding of virtual memory.

You could also try to disable the ashmem optimization in fresco (if possible. I don't know fresco myself) and see if that helps.

@narychen

This comment has been minimized.

narychen commented Apr 28, 2017

Got an error when specify GOARCH="arm64"

04-29 00:07:31.911 2927-2927/? D/dalvikvm: Late-enabling CheckJNI
04-29 00:07:31.971 2927-2927/com.listtest W/dalvikvm: Exception Ljava/lang/UnsatisfiedLinkError; thrown while initializing Lgo/LoadJNI;
04-29 00:07:31.971 2927-2927/com.listtest W/GoSeq: LoadJNI class not found
04-29 00:07:31.971 2927-2927/com.listtest W/dalvikvm: No implementation found for native Lgo/Seq;.init:()V
04-29 00:07:31.971 2927-2927/com.listtest W/dalvikvm: Exception Ljava/lang/UnsatisfiedLinkError; thrown while initializing Lgo/Seq;
04-29 00:07:31.971 2927-2927/com.listtest W/dalvikvm: Exception Ljava/lang/UnsatisfiedLinkError; thrown while initializing Lgo/gotest/Gotest;
04-29 00:07:31.971 2927-2927/com.listtest D/AndroidRuntime: Shutting down VM
04-29 00:07:31.971 2927-2927/com.listtest W/dalvikvm: threadid=1: thread exiting with uncaught exception (group=0x41df3898)
04-29 00:07:31.971 2927-2927/com.listtest E/AndroidRuntime: FATAL EXCEPTION: main
                                                            java.lang.UnsatisfiedLinkError: Native method not found: go.Seq.init:()V
                                                                at go.Seq.init(Native Method)
                                                                at go.Seq.<clinit>(Seq.java:38)
                                                                at go.gotest.Gotest.<clinit>(Gotest.java:11)
                                                                at com.listtest.MainApplication.onCreate(MainApplication.java:41)
                                                                at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1010)
                                                                at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4507)
                                                                at android.app.ActivityThread.access$1300(ActivityThread.java:144)
                                                                at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1319)
                                                                at android.os.Handler.dispatchMessage(Handler.java:99)
                                                                at android.os.Looper.loop(Looper.java:137)
                                                                at android.app.ActivityThread.main(ActivityThread.java:5166)
                                                                at java.lang.reflect.Method.invokeNative(Native Method)
                                                                at java.lang.reflect.Method.invoke(Method.java:525)
                                                                at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:768)
                                                                at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:584)
                                                                at dalvik.system.NativeStart.main(Native Method)

The gobind gradle plugin

plugins {
    id "org.golang.mobile.bind" version "0.2.6"
}
gobind {
    pkg = "gotest"
    GOARCH="arm64"
     GOPATH = "/Users/nary/Desktop/entry/qdan/gopath:/Users/nary/Desktop/entry/qdan/react-test/listtest/gopath"
}
@eliasnaur

This comment has been minimized.

Contributor

eliasnaur commented Apr 28, 2017

Are you running on an Android device and Android version that are both capable of running arm64 code? Your log file shows dalvik-related messages and I think only ART is 64-bit capable. Also, you'll need at least Android 5.0, which I believe is the first release that could handle 64-bit code.

Note that this is not a realistic workaround for 4.x devices. It's simply a quick test to see if the problem is related to the limited virtual memory available on 32 bit.

@narychen

This comment has been minimized.

narychen commented Apr 29, 2017

I test it on android 5.0 and 6.0. The starting fail still happens. The logcat is shown below

04-29 10:19:00.433 22993-22993/? I/art: Late-enabling -Xcheck:jni
04-29 10:19:00.471 22993-22993/com.listtest D/QromResources: putQromResourcesInfo|put Resources = android.content.res.Resources@12b4516, size=3
04-29 10:19:00.500 22993-22993/com.listtest D/AndroidRuntime: Shutting down VM
                                                              
                                                              
                                                              --------- beginning of crash
04-29 10:19:00.503 22993-22993/com.listtest E/AndroidRuntime: FATAL EXCEPTION: main
                                                              Process: com.listtest, PID: 22993
                                                              java.lang.UnsatisfiedLinkError: dalvik.system.PathClassLoader[DexPathList[[zip file "/data/app/com.listtest-1/base.apk"],nativeLibraryDirectories=[/data/app/com.listtest-1/lib/arm, /data/app/com.listtest-1/base.apk!/lib/armeabi-v7a, /vendor/lib, /system/lib]]] couldn't find "libgojni.so"
                                                                  at java.lang.Runtime.loadLibrary(Runtime.java:367)
                                                                  at java.lang.System.loadLibrary(System.java:1076)
                                                                  at go.LoadJNI.<clinit>(LoadJNI.java:23)
                                                                  at java.lang.Class.classForName(Native Method)
                                                                  at java.lang.Class.forName(Class.java:324)
                                                                  at java.lang.Class.forName(Class.java:285)
                                                                  at go.Seq.<clinit>(Seq.java:28)
                                                                  at go.Seq.touch(Seq.java:45)
                                                                  at go.gotest.Gotest.<clinit>(Gotest.java:11)
                                                                  at com.listtest.MainApplication.onCreate(MainApplication.java:41)
                                                                  at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1018)
                                                                  at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4914)
                                                                  at android.app.ActivityThread.-wrap1(ActivityThread.java)
                                                                  at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1526)
                                                                  at android.os.Handler.dispatchMessage(Handler.java:111)
                                                                  at android.os.Looper.loop(Looper.java:207)
                                                                  at android.app.ActivityThread.main(ActivityThread.java:5683)
                                                                  at java.lang.reflect.Method.invoke(Native Method)
                                                                  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:901)
                                                                  at com.android.internal.os.ZygoteInit.tos_org_main(ZygoteInit.java:762)
                                                                  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:670)

But as fresco do not use Ashmem on android 5.0+, so the crash seems not happen so easily.
It only quickly crash on android 4.3 and slowly block fresco to show pictures on 4.4 since fresco use different memory management on android 4.3, 4.4 and 5.0+.

@eliasnaur

This comment has been minimized.

Contributor

eliasnaur commented Apr 29, 2017

React Native and/or fresco is probably using 32-bit libraries, blocking ART from running in 64-bit mode. facebook/react-native#2814 seems to indicate that at least reach native doesn't run in 64-bit mode at all.
In any case, the test was only to confirm that the issue is about virtual memory space. And if so, I don't see an (easy) solution for your problem. I'm sorry.

@narychen

This comment has been minimized.

narychen commented May 2, 2017

Maybe it's not caused by running out virtual memory. Because my android phone has 3000MB RAM. If I run it with Go, the app crashes when the app only takes about 200Mb memory. But normally the app will be crashed when the memory is taken above 1000MB.

@bradfitz bradfitz added the mobile label Jul 20, 2017

@eliasnaur

This comment has been minimized.

Contributor

eliasnaur commented Mar 16, 2018

Go tip (to become Go 1.11) now uses fragmented heap maps. If you're feeling adventurous, please retry with go tip. You'll have to download the Go source and run ./make.bash from the src/ directory and use the go command from the bin/ directory.

@narychen

This comment has been minimized.

narychen commented Mar 16, 2018

Are you sure this could let the problem go away ?

@eliasnaur

This comment has been minimized.

Contributor

eliasnaur commented Mar 16, 2018

I'm not sure, that's why I'm asking you to try it.

@narychen

This comment has been minimized.

narychen commented Mar 16, 2018

It is worth a try. Thank you for your advice.

@narychen

This comment has been minimized.

narychen commented Mar 21, 2018

@eliasnaur Just another question: Will the release Go of 1.11 officially uses fragmented heap maps ?
If that will be, or I could just wait for the official release and not resort to compile a adventurous version.

@narychen

This comment has been minimized.

narychen commented Mar 21, 2018

And I couldn't find go 1.11. Google has said nothing about 1.11 and the fragmented heap maps. Where did you get info ?

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Mar 21, 2018

Fragmented heap maps were added to tip in https://golang.org/cl/96779 and https://golang.org/cl/96780. They will be in the future Go 1.11 release.

@narychen

This comment has been minimized.

narychen commented Mar 23, 2018

@ianlancetaylor You mean if I wanna check this I could just git clone the golang repo and switch to this commit then compile ?

@ianlancetaylor

This comment has been minimized.

Contributor

ianlancetaylor commented Mar 23, 2018

Yes.

@eliasnaur eliasnaur removed their assignment Oct 15, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment