Enable modules snapshot for Android #1563

Closed
atanasovg opened this Issue Feb 16, 2016 · 17 comments

Projects

None yet

9 participants

@atanasovg
Member
atanasovg commented Feb 16, 2016 edited

The Story

In a nutshell - since in Android application loading time is one of the areas that need improvement,
we've made a POC implementation that takes advantage of V8's startup snapshots feature to see what may be achieved with it. The results are quite promising and we may gain literally more than 1 second ! of an improvement by saving all the modules.

Technical Details

Due to the V8 API specifics, we need to bundle the entire modules JS into one single file and pass it to the V8::CreateSnapshotDataBlob method. What V8 does when making the snapshot is to parse, compile and run this script into a new Context and then to save the state of the heap into a binary representation. Then, upon next application runs, this binary file may be used to load the whole representation of the modules directly within memory.

Here is my proposal for taking advantage of this feature:

Distribute pre-generated BLOBs

The snapshot is CPU-architecture dependent. Hence, if we want to distribute pre-generated version of the snapshot we will need to package three files, saved against the three available architectures - armeabi-v7a, x86 and arm64-v8a. The average size of one file is ~3 MB but it compresses very well and an archived version is about 400 KB.

This is the most efficient way performance-wise. It adds further optimization by skipping the extraction of numerous JS files initially

Tasks:

  • Ensure modules are snapshot-ready. This is already done - #1407

  • Write a custom bundler that puts all the modules content within a single JS file. I've already done a custom Node task for the POC but we may go with webpack for example (as @hdeshev) suggested.

  • Think how to automate snapshot generation against the three CPU architectures.

  • Think how to distribute (package) these three binary files. For example we may have two packages - tns-core-modules and tns-core-modules-snapshot. Or we may package all into one package.

  • Since the debugger does not work with snapshots, we will need to rely on the snapshot for release builds ONLY. Hence, there will be some effort on the CLI side. Plus, for release builds the CLI should not pack the modules JavaScript files but only the BLOBs (ping @teobugslayer, @ligaz).

  • Enable the Android Runtime to consume such a BLOB directly, depending on the current CPU architecture. We may use the same convention as for the native part of the Runtime itself, for example:

    tns_modules
    ├── snapshot
    │   ├── armeabi-v7a
    │   │   ├── snapshot.blob
    │   ├── arm64-v8a
    │   │   ├── snapshot.blob
    │   ├── x86
    │   │   ├── snapshot.blob
    
    
@atanasovg atanasovg added this to the 1.7.0 (Under Review) milestone Feb 16, 2016
@NathanaelA
Contributor

Please allow the CLI to be able to (optionally?) generate the BLOB also; if I am bundling a dozen plugins; their is no reason why I shouldn't be able to get the speed enhancement by creating my own BLOB's with all my modules on Android.

@x4080
x4080 commented Feb 19, 2016

Snapshot generation can be generated on pc or have to be on device? If have to be on device, for start maybe the developer can manually use command or put some kinda function to generate it then send it back for deployment

If this is greatly improve things up, even if this is the only way, i will still be happy :)

@vjoao
Contributor
vjoao commented Feb 20, 2016

Can this be extended to put the actual app code inside the blob?

@fealebenpae
Contributor

I would suggest investigating whether an alternative to v8 with multiple execution tiers such as JavaScriptCore, SpiderMonkey or ChakraCore would obviate the need to jump through heap snapshotting hoops - over in iOS land we are very happy with our startup times since bootstrapping the JavaScriptCore interpreter is way faster than bringing up the optimizing JIT tier. Since v8 lacks an interpreter, it takes longer before it can execute JavaScript code because it has to JIT compile it, right?

@NathanaelA
Contributor

@fealebenpae - v8 has multiple JIT tiers and an interpreter. I don't know for sure, but from what I've seen the issue is not bringing up v8 so much as the android/metadata side of things. This just add an additional speed enhancement. Can @atanasovg or someone post where the startup time is at?

Using JSC "might" (and that is a big might!) be worth while; because it would mean both platforms would be using the same JS engine. However, you would have to re-tool several things like the Developer tools would have to be re-worked to work from JSC rather than v8. Safari isn't supported on Linux or Windows so you have to have something on both those platforms to debug with.

@atanasovg atanasovg removed this from the 1.7.0 (Under Review) milestone Feb 22, 2016
@borovaka

@atanasovg Do you use same approach as Atom's https://github.com/atom/node-mksnapshot?
Can you provide some code sample using this method with NS project?

@jasssonpet
Contributor
jasssonpet commented Apr 11, 2016 edited

Hello, guys. I'm going to update the issue with the results of a more recent research we did on V8 heap snapshots. Feel free to ask any questions, if something is missing or not quite clear to you.

Results

Here are the startup times of {N} Angular on a Nexus 5 device. Only second runs are included, in release configuration.

{N} Angular Hello World App - Source

Run Startup time
Non-bundled 3400ms
Bundled 3000ms
Bundled and snapshotted 2800ms
Bundled and snapshotted and evaluated 1550ms
Bundled and snapshotted and evaluated and static bindings 1450ms

{N} Angular Render Test - Source

Run Startup time
Non-bundled 4000ms
Bundled 3600ms
Bundled and snapshotted 3400ms
Bundled and snapshotted and evaluated 2300ms
Bundled and snapshotted and evaluated and static bindings 2100ms

Snapshot size is about 12-16MB (4-6MB when the script is minified) per architecture.

Tasks - 2.0

  • Update NativeScript modules to be snapshottable:
    • Refer to this patch for the changes.
  • Resolve require calls and JavaScriptImplementation annotations from snapshot:
    • Currently we are using webpack to create the bundle. We should figure out how to map module ids to webpack indices properly.
  • Research what could be appropriate to be evaluated in the snapshot:
    • We should make sure that all modules are included in the snapshot, even those that could get stripped from the bundler dependencies.
  • JavaScript debugging:
    • At the moment feeding the V8 engine with snapshot data conflicts with registering a debug message handler. We should figure out how to start the debugger properly, so that it can work with snapshot data.
  • Static binding generator
  • Currently the Android runtime looks for pre-generated classes from the static binding generator based on filename. When these files are snapshotted, the filename becomes <embedded script> (<embedded> in V8 5.1) and these classes are not found in runtime leaving it to dynamic binding generation.
  • Assets extraction:
    • We should measure the performance of extracting the snapshot data blob at runtime.
    • We should measure the increase in the size of the distributed archive when snapshots are enabled.
    • We should measure the memory overhead when using snapshots, if there is one.
    • Snapshot blobs should be excluded from non-Android builds.
    • Memory map the snapshot blob to reduce startup memory
  • Rebuild V8:
  • Obfuscation:
    • {N} Angular has some issues when using minified scripts, like some styles get stripped.
    • The Android static binding generator does not resolve minified Android namespaces. We should figure out a way to preserve the android and java namespaces when minifying.
  • Warmup API:
    • Test how the new WarmUpSnapshotDataBlob() API is behaving. It seems that it is introducing enum FunctionCodeHandling { CLEAR_FUNCTION_CODE, KEEP_FUNCTION_CODE } in the serializer options. This should probably now include compiled code and would make the snapshots even faster. There is some progress here.
  • Automate snapshot generation:
    • We should figure out how to use the V8 ARM/ARM64 simulator to create a command line tool that generates snapshots during build.

Tasks 2.1

  • Automate bundling and snapshot creation in our build infrastructure
    • At the moment, bundle creation is fully automated, but creating snapshot files for different architectures isn't. Angular and {N} Angular are moving quite fast and we should find a way to automate the creation of these files with our builds. This could mean building the V8 mksnapshot tool or using Android emulators to do this.
  • Merge the nativescript-angular-snapshot prototype with the existing nativescript-angular plugin
    • Currently, the Angular snapshot demo is an unpublished plugin in GitHub. We should make it the default option (maybe for release builds) with the official {N} Angular plugin. This could potentially lead to more unexpected errors when using snapshots for other apps.
  • Decouple the bundler from Angular specifics and integrate it with the tns-core-modules package
    • After the bundling and snapshot creation is fully automated, we can integrate it not only with the {N} Angular plugin, but with the NativeScript core modules, too.
  • Expose all bundled modules
    • Some third-party modules get included in the bundle, but are not exposed in the global require override. The modules included in the bundle resolve to the bundled one, but other external modules resolve the module on the file system. This occurs with the reflect-metadata polyfill when running the ng-todo sample.
  • Properly join path module id and directory name
    • When only some part of a package is included in the snapshotted bundle, other files from the same package resolve to the file system ones. In them there are some relative require('../../') calls that should be normalized in the require override and be resolved from the bundle. This occurs with the extension methods of rxjs when running the groceries sample.
  • Unarchive only the snapshot file for the current architecture
    • The APK file includes snapshot files for all three architectures (ARMv7, ARM64 and x86). This increases the APK file size with 6MB. During the first application launch, all these files are extracted to a total of 24MB. But only the snapshot file for the current architecture is used, so the other ones are useless and should be skipped.
  • Lazily evaluate extending Android application and activity
    • Collaborate with the modules team to figure out the lesser evil way to merge NativeScript#2031.
  • Research if debugging embedded scripts with the VS Code extension is possible
    • At the moment when we step in the snapshotted script when debugging, the VS Code debugger gets detached. We should research what Node Inspector is doing and if there could be an easy fix for this.

Ping @atanasovg, @KristinaKoeva.

P.S. On the regards of encryption, the heap snapshot turns out to be a poor shot, because all of the JavaScript source seems to be included inside the snapshot data blob.

@NathanaelA
Contributor

@jasssonpet Just a FYI, the actual NS runtime (not just angular) has some issues with minifications in the styling system; I ran into this with my NativeScript-Protect; which is one of the reasons I disable minification by default in my encryption system. I suspect the reason why (I haven't traced it fully down yet; since that isn't the purpose of NS-Protect; but because I was curious) and based on some initial tests it looks like the issue is because of the typeName gets set to the wrong name from types.getClass because the minification changes the class names... This might be the same issue you are seeing with Angular...

Good to know that SnapShots won't eliminate the need for my NS-Protect encryption. ;-)

@jasssonpet jasssonpet added a commit to NativeScript/tns-core-modules-widgets that referenced this issue Apr 27, 2016
@jasssonpet jasssonpet Update package.json
There is no `index.js` in the main folder and this breaks bundling.

Related to: NativeScript/NativeScript#1563
d666343
@jasssonpet jasssonpet referenced this issue in NativeScript/tns-core-modules-widgets Apr 27, 2016
Merged

Update package.json #19

@jasssonpet
Contributor
jasssonpet commented Apr 28, 2016 edited
@x4080
x4080 commented May 5, 2016

Is the loading performance improvement already in 2.0?

@jasssonpet
Contributor

@x4080 This is not enabled by default for now, but we are looking for ways to do so in the next release.

@x4080
x4080 commented May 5, 2016

I see, so in the NS demo app it still not using it yet, I guess ?

@jasssonpet
Contributor

@x4080 Not yet.

Once we use it there, you can expect the app to start a whole lot faster 😄

@x4080
x4080 commented May 5, 2016

Alright then

@jasssonpet
Contributor
jasssonpet commented May 13, 2016 edited

I can confirm that the mksnapshot tool from V8 when cross-compiled for ARM successfully generates ARM snapshots with the V8 ARM simulator from the host machine, without the need for any ARM devices or emulators 🎆

@jasssonpet jasssonpet added this to the 2.1 milestone May 25, 2016
@hshristov hshristov closed this Jul 1, 2016
@NathanaelA
Contributor

@jasssonpet Do we have instructions anywhere on how to use this now and/or any gotchas?

@vchimev vchimev added 5 - Done and removed 4 - Ready For Test labels Aug 31, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment