Do not include JSC and Hermes debug libraries when building in release mode by maxammann · Pull Request #30303 · react/react-native

maxammann · 2020-11-02T18:48:00Z

Summary

Right now the libhermes-executor-debug.so and libjscexecutor.so get included when doing a release build with Hermes enabled. This can be fixed by deleting the .so files from the correct directories in the gradle build directory. They have to be deleted because they are packaged in the ./node_modules/react-native/android/com/facebook/react/react-native/0.63.2/react-native-0.63.2.aar aar. This actually should be fixed somewhere else properly.

This PR fixes the workaround already implemented:

Execute vmSelectionAction before strip${targetName}DebugSymbols instead of package${targetName}
Remove .so files from merged_native_libs such that they don't get copied to stripped_native_libs

Before this PR multiple stack traces are thrown when starting an app:

11-02 19:46:36.413 24727 24768 W System.err: java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "_ZN8facebook6hermes13HermesRuntime11getDebuggerEv" referenced by "/data/app/~~ZHwF7KVKSeGGWCuyb8_i2Q==/tuerantuer.app.integreat-fSWEEZw7ODD62ZPjepvHlw==/lib/x86_64/libhermes-executor-debug.so"...
11-02 19:46:36.413 24727 24768 W System.err: 	at java.lang.Runtime.load0(Runtime.java:939)
11-02 19:46:36.413 24727 24768 W System.err: 	at java.lang.System.load(System.java:1628)
11-02 19:46:36.413 24727 24768 W System.err: 	at com.facebook.soloader.SoLoader$1.load(SoLoader.java:395)
11-02 19:46:36.413 24727 24768 W System.err: 	at com.facebook.soloader.DirectorySoSource.loadLibraryFrom(DirectorySoSource.java:77)
11-02 19:46:36.413 24727 24768 W System.err: 	at com.facebook.soloader.DirectorySoSource.loadLibrary(DirectorySoSource.java:50)
11-02 19:46:36.413 24727 24768 W System.err: 	at com.facebook.soloader.ApplicationSoSource.loadLibrary(ApplicationSoSource.java:82)
11-02 19:46:36.413 24727 24768 W System.err: 	at com.facebook.soloader.SoLoader.doLoadLibraryBySoName(SoLoader.java:766)
11-02 19:46:36.413 24727 24768 W System.err: 	at com.facebook.soloader.SoLoader.loadLibraryBySoName(SoLoader.java:673)
11-02 19:46:36.414 24727 24768 W System.err: 	at com.facebook.soloader.SoLoader.loadLibrary(SoLoader.java:611)
11-02 19:46:36.414 24727 24768 W System.err: 	at com.facebook.soloader.SoLoader.loadLibrary(SoLoader.java:559)
11-02 19:46:36.414 24727 24768 W System.err: 	at com.facebook.hermes.reactexecutor.HermesExecutor.<clinit>(HermesExecutor.java:22)
11-02 19:46:36.414 24727 24768 W System.err: 	at com.facebook.hermes.reactexecutor.HermesExecutorFactory.create(HermesExecutorFactory.java:29)
11-02 19:46:36.414 24727 24768 W System.err: 	at com.facebook.react.ReactInstanceManager$5.run(ReactInstanceManager.java:1017)
11-02 19:46:36.414 24727 24768 W System.err: 	at java.lang.Thread.run(Thread.java:923)
11-02 19:46:36.414 24727 24768 E SoLoader: couldn't find DSO to load: libhermes-executor-debug.so caused by: dlopen failed: cannot locate symbol "_ZN8facebook6hermes13HermesRuntime11getDebuggerEv" referenced by "/data/app/~~ZHwF7KVKSeGGWCuyb8_i2Q==/tuerantuer.app.integreat-fSWEEZw7ODD62ZPjepvHlw==/lib/x86_64/libhermes-executor-debug.so"... result: 0
11-02 19:46:36.414 24727 24768 D SoLoader: About to load: libhermes-executor-release.so

11-02 19:46:36.356 24727 24727 D SoLoader: About to load: libjscexecutor.so
11-02 19:46:36.356 24727 24727 D SoLoader: libjscexecutor.so not found on /data/user/0/tuerantuer.app.integreat/lib-main
11-02 19:46:36.356 24727 24727 D SoLoader: libjscexecutor.so found on /data/app/~~ZHwF7KVKSeGGWCuyb8_i2Q==/tuerantuer.app.integreat-fSWEEZw7ODD62ZPjepvHlw==/lib/x86_64
11-02 19:46:36.356 24727 24727 D SoLoader: Not resolving dependencies for libjscexecutor.so
11-02 19:46:36.356   307   307 W EmuHWC2 : No layers, exit, buffer 0x7a9456c0e970
11-02 19:46:36.358 24727 24727 W System.err: java.lang.UnsatisfiedLinkError: dlopen failed: library "libjsc.so" not found: needed by /data/app/~~ZHwF7KVKSeGGWCuyb8_i2Q==/tuerantuer.app.integreat-fSWEEZw7ODD62ZPjepvHlw==/lib/x86_64/libjscexecutor.so in namespace classloader-namespace
11-02 19:46:36.359 24727 24727 W System.err: 	at java.lang.Runtime.load0(Runtime.java:939)
11-02 19:46:36.359 24727 24727 W System.err: 	at java.lang.System.load(System.java:1628)
11-02 19:46:36.359 24727 24727 W System.err: 	at com.facebook.soloader.SoLoader$1.load(SoLoader.java:395)
11-02 19:46:36.359 24727 24727 W System.err: 	at com.facebook.soloader.DirectorySoSource.loadLibraryFrom(DirectorySoSource.java:77)
11-02 19:46:36.359 24727 24727 W System.err: 	at com.facebook.soloader.DirectorySoSource.loadLibrary(DirectorySoSource.java:50)
11-02 19:46:36.359 24727 24727 W System.err: 	at com.facebook.soloader.ApplicationSoSource.loadLibrary(ApplicationSoSource.java:82)
11-02 19:46:36.359 24727 24727 W System.err: 	at com.facebook.soloader.SoLoader.doLoadLibraryBySoName(SoLoader.java:766)
11-02 19:46:36.359 24727 24727 W System.err: 	at com.facebook.soloader.SoLoader.loadLibraryBySoName(SoLoader.java:673)
11-02 19:46:36.359 24727 24727 W System.err: 	at com.facebook.soloader.SoLoader.loadLibrary(SoLoader.java:611)
11-02 19:46:36.359 24727 24727 W System.err: 	at com.facebook.soloader.SoLoader.loadLibrary(SoLoader.java:559)
11-02 19:46:36.359 24727 24727 W System.err: 	at com.facebook.react.ReactInstanceManagerBuilder.getDefaultJSExecutorFactory(ReactInstanceManagerBuilder.java:297)
11-02 19:46:36.359 24727 24727 W System.err: 	at com.facebook.react.ReactInstanceManagerBuilder.build(ReactInstanceManagerBuilder.java:270)
11-02 19:46:36.359 24727 24727 W System.err: 	at com.facebook.react.ReactNativeHost.createReactInstanceManager(ReactNativeHost.java:87)
11-02 19:46:36.359 24727 24727 W System.err: 	at com.facebook.react.ReactNativeHost.getReactInstanceManager(ReactNativeHost.java:39)
11-02 19:46:36.359 24727 24727 W System.err: 	at tuerantuer.app.integreat.MainApplication.onCreate(MainApplication.java:45)
11-02 19:46:36.359 24727 24727 W System.err: 	at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1192)
11-02 19:46:36.359 24727 24727 W System.err: 	at android.app.ActivityThread.handleBindApplication(ActivityThread.java:6712)
11-02 19:46:36.359 24727 24727 W System.err: 	at android.app.ActivityThread.access$1300(ActivityThread.java:237)
11-02 19:46:36.359 24727 24727 W System.err: 	at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1913)
11-02 19:46:36.359 24727 24727 W System.err: 	at android.os.Handler.dispatchMessage(Handler.java:106)
11-02 19:46:36.359 24727 24727 W System.err: 	at android.os.Looper.loop(Looper.java:223)
11-02 19:46:36.359 24727 24727 W System.err: 	at android.app.ActivityThread.main(ActivityThread.java:7656)
11-02 19:46:36.360 24727 24727 W System.err: 	at java.lang.reflect.Method.invoke(Native Method)
11-02 19:46:36.360 24727 24727 W System.err: 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
11-02 19:46:36.360 24727 24727 W System.err: 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
11-02 19:46:36.360 24727 24727 E SoLoader: couldn't find DSO to load: libjscexecutor.so caused by: dlopen failed: library "libjsc.so" not found: needed by /data/app/~~ZHwF7KVKSeGGWCuyb8_i2Q==/tuerantuer.app.integreat-fSWEEZw7ODD62ZPjepvHlw==/lib/x86_64/libjscexecutor.so in namespace classloader-namespace result: 0
11-02 19:46:36.360 24727 24727 D SoLoader: init exiting

Afterwards they are gone:

11-02 19:27:08.052 23251 23293 D SoLoader: About to load: libhermes-executor-debug.so
11-02 19:27:08.052 23251 23293 D SoLoader: libhermes-executor-debug.so not found on /data/user/0/tuerantuer.app.integreat/lib-main
11-02 19:27:08.052 23251 23293 D SoLoader: libhermes-executor-debug.so not found on /data/app/~~0mYQ5Jyf4NG9lK86d9nkjw==/tuerantuer.app.integreat-tyc3LBv82Ac1cO7aGgEP3w==/lib/x86_64
11-02 19:27:08.052 23251 23293 D SoLoader: libhermes-executor-debug.so not found on /vendor/lib
11-02 19:27:08.053 23251 23293 D SoLoader: libhermes-executor-debug.so not found on /system/lib

11-02 19:27:07.977 23251 23251 D SoLoader: About to load: libjscexecutor.so
11-02 19:27:07.977 23251 23251 D SoLoader: libjscexecutor.so not found on /data/user/0/tuerantuer.app.integreat/lib-main
11-02 19:27:07.978 23251 23251 D SoLoader: libjscexecutor.so not found on /data/app/~~0mYQ5Jyf4NG9lK86d9nkjw==/tuerantuer.app.integreat-tyc3LBv82Ac1cO7aGgEP3w==/lib/x86_64
11-02 19:27:07.979 23251 23251 D SoLoader: libjscexecutor.so not found on /vendor/lib
11-02 19:27:07.979 23251 23251 D SoLoader: libjscexecutor.so not found on /system/lib

Changelog

Android Fixed - Do not include JSC and Hermes debug libraries when building in release mode

Test Plan

Check for the output of adb logcat for the stacktraces. If they are gone after implementing this fix then it works. Run ./gradlew clean every time before building.

Summary: * Execute vmSelectionAction before strip${targetName}DebugSymbols instead of package${targetName} * Remove .so files from merged_native_libs such that they don't get copied to stripped_native_libs

facebook-github-bot · 2020-11-02T18:48:10Z

Hi @maxammann!

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file.

In order for us to review and merge your code, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks!

maxammann · 2020-11-02T18:51:33Z

Fixes #29064 and facebook/hermes#96

pull-bot · 2020-11-02T18:54:47Z

	Messages
📖	📋 Verify Changelog Format - A changelog entry has the following format: `[CATEGORY] [TYPE] - Message`. Details CATEGORY may be: General iOS Android JavaScript Internal (for changes that do not need to be called out in the release notes) TYPE may be: Added, for new features. Changed, for changes in existing functionality. Deprecated, for soon-to-be removed features. Removed, for now removed features. Fixed, for any bug fixes. Security, in case of vulnerabilities. MESSAGE may answer "what and why" on a feature level. Use this to briefly tell React Native users about notable changes.

Generated by 🚫 dangerJS against 5be02a7

maxammann · 2020-11-02T18:57:53Z

A workaround which works right now is:


 if (enableHermes) {
            def taskName = getGradle().getStartParameter().getTaskRequests().toString()

            exclude "**/libjsc*.so"
            if (taskName.toLowerCase().contains("release")) {
                exclude '**/libhermes-inspector.so'
                exclude '**/libhermes-executor-debug.so'
            } else {
                exclude '**/libhermes-executor-release.so'
            }
        }
}

thanks to @prempalsingh

analysis-bot · 2020-11-02T19:10:16Z

Platform	Engine	Arch	Size (bytes)	Diff
ios	-	universal	n/a	--

Base commit: 00d9dea

analysis-bot · 2020-11-02T19:13:29Z

Platform	Engine	Arch	Size (bytes)	Diff
android	hermes	arm64-v8a	7,507,195	+0
android	hermes	armeabi-v7a	7,098,975	+0
android	hermes	x86	7,945,340	+0
android	hermes	x86_64	7,856,392	+0
android	jsc	arm64-v8a	8,971,898	+0
android	jsc	armeabi-v7a	8,546,808	+0
android	jsc	x86	8,971,545	+0
android	jsc	x86_64	9,522,861	+0

Base commit: 00d9dea

facebook-github-bot · 2020-11-02T19:20:35Z

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!

ecreeth · 2020-11-02T19:25:37Z

cc @fkgozali

facebook-github-bot

@fkgozali has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

fkgozali · 2020-11-05T17:42:27Z

There's a failure in FB internal CI for some reason, I'll need some time to look into that. CircleCI test_android seems green with this PR though.

fkgozali · 2020-11-05T19:25:52Z

@maxammann: it looks like something in this change doesn't play well with multiple flavors of rn-tester build. I think gradle got confused with the cached NDK build artifacts from the previous runs. To repro, we can try building with and without Fabric, and the 2nd build always failed, until you try again. For example:

# OK
./gradlew packages:rn-tester:android:app:packageHermesDebug

# Failed:
USE_FABRIC=1 ./gradlew packages:rn-tester:android:app:packageHermesDebug

# OK (2nd run, the same command as before)
USE_FABRIC=1 ./gradlew packages:rn-tester:android:app:packageHermesDebug

# Failed: if we disable USE_FABRIC again
./gradlew packages:rn-tester:android:app:packageHermesDebug

# OK (2nd run: the same command as before)
./gradlew packages:rn-tester:android:app:packageHermesDebug

The failure:

* What went wrong:
Execution failed for task ':packages:rn-tester:android:app:stripHermesDebugDebugSymbols'.
> 1 exception was raised by workers:
  java.nio.file.NoSuchFileException: /......../packages/rn-tester/android/app/build/intermediates/merged_native_libs/hermesDebug/out/lib/x86_64/libjscexecutor.so

This means, the removal logic in this PR incorrectly notifies gradle of the changes, causing incorrect caching of task outputs. Can you look into how to make each run consistently succeed without having to run the same command twice?

maxammann · 2020-11-06T13:16:15Z

@fkgozali thanks for checking the changes! I replaced the hacky way with a more stable way. I think this should solve problems with compiling multiple times

maxammann · 2020-11-06T13:55:55Z

Updating branch to rerun failed ios tests

fkgozali · 2020-11-06T18:26:58Z

The iOS failures seem unrelated, but test_android had:

1: Task failed with an exception.
-----------
* Where:
Build file '/root/react-native/packages/rn-tester/android/app/build.gradle' line: 86

* What went wrong:
A problem occurred evaluating script.
> No such property: name for class: java.lang.String

maxammann · 2020-11-07T08:09:21Z

@fkgozali The api I need is only available with the gradle plugin 4.2 which is still alpha: https://developer.android.com/reference/tools/gradle-api/4.2/com/android/build/api/variant/ApplicationVariant#packagingOptions(kotlin.Function1)

I think it makes sense to wait with this PR until we are at version 4.2. What do you think?

maxammann · 2020-11-07T08:27:26Z

Alternatively there is this solution: https://github.com/Jween/android-soexcluder/blob/master/app/src/main/groovy/com/jween/gradle/soexcluder/AppPlugin.groovy#L66

fkgozali · 2020-11-11T18:12:33Z

I think it makes sense to wait with this PR until we are at version 4.2. What do you think?

I think this is probably best next step. We tend to upgrade AGP regularly, and if this is a good reason to upgrade, then it will make it easier for us to move forward during upgrade.

Alternatively there is this solution

I'm hesitant to add a custom logic that's supposed to be supported by AGP directly, so unless there's a super urgent reason for excluding the .so right now, let's wait.

What do you think?

maxammann · 2020-11-11T20:04:43Z

Jup lets wait, I can close this and reopen it in the future. What do you think?

fkgozali · 2020-11-11T20:06:15Z

Jup lets wait, I can close this and reopen it in the future. What do you think?

Sure, thanks!

maxammann · 2021-06-05T15:07:45Z

Gradle 4.2 is released now and this could be fixed. I'll also check whether this still happens. Does RN already use 4.2?

maxammann · 2021-06-05T15:11:30Z

Yet it is :) https://github.com/facebook/react-native/blob/master/build.gradle.kts#L15

rkok · 2022-03-24T13:02:52Z

Still happening here. Using Gradle 6.4 and React Native 0.66.4.

My workaround is to put this in build.gradle:

android {
    // ...
    buildTypes {
        release {
            packagingOptions {
                exclude "**/libhermes-inspector.so"
                exclude "**/libhermes*debug.so"
            }
            // ...
        }
    }
}

cortinico · 2022-06-16T21:44:16Z

I'm closing this as we now have a different mechanism to exclude unnecessary libs from being bundled in the final .apk/.aab

rkok · 2022-06-17T06:18:14Z

@cortinico Could you place a link to where that new mechanism is explained? Thanks in advance.

cortinico · 2022-06-17T15:49:59Z

@rkok yup here it is:
https://github.com/facebook/react-native/blob/ea29ae1ceb3845e8116e2e9cf9399ab38523b777/react.gradle#L409-L464

Filter merged_native_libs instead of transforms

ca09548

Summary: * Execute vmSelectionAction before strip${targetName}DebugSymbols instead of package${targetName} * Remove .so files from merged_native_libs such that they don't get copied to stripped_native_libs

maxammann commented Nov 2, 2020

View reviewed changes

Comment thread react.gradle Outdated

Remove debug lines

9c0f41e

maxammann commented Nov 2, 2020

View reviewed changes

Comment thread react.gradle Outdated

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 2, 2020

facebook-github-bot reviewed Nov 3, 2020

View reviewed changes

Replace hacky library exclusion by using the android api

577eb2b

Merge branch 'master' into patch-1

74f29b2

Try to use enableHermesForVariant

ebc683e

maxammann added 2 commits November 6, 2020 20:44

Use variant name instead of whole variant

c40acdc

Merge branch 'master' into patch-1

5be02a7

maxammann closed this Nov 13, 2020

marudy mentioned this pull request Jan 16, 2021

Android application becomes super slow on startup after enabling Hermes #30750

Closed

maxammann reopened this Jun 5, 2021

cortinico closed this Jun 16, 2022

Uh oh!

Conversation

maxammann commented Nov 2, 2020

Summary

Changelog

Test Plan

Uh oh!

facebook-github-bot commented Nov 2, 2020

Uh oh!

Uh oh!

Uh oh!

maxammann commented Nov 2, 2020

Uh oh!

pull-bot commented Nov 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maxammann commented Nov 2, 2020

Uh oh!

analysis-bot commented Nov 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

analysis-bot commented Nov 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Nov 2, 2020

Uh oh!

ecreeth commented Nov 2, 2020

Uh oh!

facebook-github-bot left a comment

Choose a reason for hiding this comment

Uh oh!

fkgozali commented Nov 5, 2020

Uh oh!

fkgozali commented Nov 5, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maxammann commented Nov 6, 2020

Uh oh!

maxammann commented Nov 6, 2020

Uh oh!

fkgozali commented Nov 6, 2020

Uh oh!

maxammann commented Nov 7, 2020

Uh oh!

maxammann commented Nov 7, 2020

Uh oh!

fkgozali commented Nov 11, 2020

Uh oh!

maxammann commented Nov 11, 2020

Uh oh!

fkgozali commented Nov 11, 2020

Uh oh!

maxammann commented Jun 5, 2021

Uh oh!

maxammann commented Jun 5, 2021

Uh oh!

rkok commented Mar 24, 2022

Uh oh!

cortinico commented Jun 16, 2022

Uh oh!

rkok commented Jun 17, 2022

Uh oh!

cortinico commented Jun 17, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

pull-bot commented Nov 2, 2020 •

edited

Loading

analysis-bot commented Nov 2, 2020 •

edited

Loading

analysis-bot commented Nov 2, 2020 •

edited

Loading

fkgozali commented Nov 5, 2020 •

edited

Loading