Enable modules snapshot for Android

[atanasovg]

The Story

In a nutshell - since in Android application loading time is one of the areas that need improvement,
we've made a POC implementation that takes advantage of V8's startup snapshots feature to see what may be achieved with it. The results are quite promising and we may gain literally more than 1 second ! of an improvement by saving all the modules.

Technical Details

Due to the V8 API specifics, we need to bundle the entire modules JS into one single file and pass it to the V8::CreateSnapshotDataBlob method. What V8 does when making the snapshot is to parse, compile and run this script into a new Context and then to save the state of the heap into a binary representation. Then, upon next application runs, this binary file may be used to load the whole representation of the modules directly within memory.

Here is my proposal for taking advantage of this feature:

Distribute pre-generated BLOBs

The snapshot is CPU-architecture dependent. Hence, if we want to distribute pre-generated version of the snapshot we will need to package three files, saved against the three available architectures - armeabi-v7a, x86 and arm64-v8a. The average size of one file is ~3 MB but it compresses very well and an archived version is about 400 KB.

This is the most efficient way performance-wise. It adds further optimization by skipping the extraction of numerous JS files initially

Tasks:

• Ensure modules are snapshot-ready. This is already done - Atanasovg/snapshot refactorings #1407

• Write a custom bundler that puts all the modules content within a single JS file. I've already done a custom Node task for the POC but we may go with webpack for example (as @hdeshev) suggested.

• Think how to automate snapshot generation against the three CPU architectures.

• Think how to distribute (package) these three binary files. For example we may have two packages - tns-core-modules and tns-core-modules-snapshot. Or we may package all into one package.

• Since the debugger does not work with snapshots, we will need to rely on the snapshot for release builds ONLY. Hence, there will be some effort on the CLI side. Plus, for release builds the CLI should not pack the modules JavaScript files but only the BLOBs (ping @teobugslayer, @ligaz).

• Enable the Android Runtime to consume such a BLOB directly, depending on the current CPU architecture. We may use the same convention as for the native part of the Runtime itself, for example:

  tns_modules
  ├── snapshot
  │   ├── armeabi-v7a
  │   │   ├── snapshot.blob
  │   ├── arm64-v8a
  │   │   ├── snapshot.blob
  │   ├── x86
  │   │   ├── snapshot.blob
  
  

[NathanaelA]

Please allow the CLI to be able to (optionally?) generate the BLOB also; if I am bundling a dozen plugins; their is no reason why I shouldn't be able to get the speed enhancement by creating my own BLOB's with all my modules on Android.

[x4080]

Snapshot generation can be generated on pc or have to be on device? If have to be on device, for start maybe the developer can manually use command or put some kinda function to generate it then send it back for deployment

If this is greatly improve things up, even if this is the only way, i will still be happy :)

[vjoao]

Can this be extended to put the actual app code inside the blob?

[fealebenpae]

I would suggest investigating whether an alternative to v8 with multiple execution tiers such as JavaScriptCore, SpiderMonkey or ChakraCore would obviate the need to jump through heap snapshotting hoops - over in iOS land we are very happy with our startup times since bootstrapping the JavaScriptCore interpreter is way faster than bringing up the optimizing JIT tier. Since v8 lacks an interpreter, it takes longer before it can execute JavaScript code because it has to JIT compile it, right?

[NathanaelA]

@fealebenpae - v8 has multiple JIT tiers and an interpreter. I don't know for sure, but from what I've seen the issue is not bringing up v8 so much as the android/metadata side of things. This just add an additional speed enhancement. Can @atanasovg or someone post where the startup time is at?

Using JSC "might" (and that is a big might!) be worth while; because it would mean both platforms would be using the same JS engine. However, you would have to re-tool several things like the Developer tools would have to be re-worked to work from JSC rather than v8. Safari isn't supported on Linux or Windows so you have to have something on both those platforms to debug with.

[borovaka]

@atanasovg Do you use same approach as Atom's https://github.com/atom/node-mksnapshot?
Can you provide some code sample using this method with NS project?

[jasssonpet]

Hello, guys. I'm going to update the issue with the results of a more recent research we did on V8 heap snapshots. Feel free to ask any questions, if something is missing or not quite clear to you.

Results

Here are the startup times of {N} Angular on a Nexus 5 device. Only second runs are included, in release configuration.

{N} Angular Hello World App - Source

Run	Startup time
Non-bundled	3400ms
Bundled	3000ms
Bundled and snapshotted	2800ms
Bundled and snapshotted and evaluated	1550ms
Bundled and snapshotted and evaluated and static bindings	1450ms

{N} Angular Render Test - Source

Run	Startup time
Non-bundled	4000ms
Bundled	3600ms
Bundled and snapshotted	3400ms
Bundled and snapshotted and evaluated	2300ms
Bundled and snapshotted and evaluated and static bindings	2100ms

Snapshot size is about 12-16MB (4-6MB when the script is minified) per architecture.

Tasks - 2.0

• Update NativeScript modules to be snapshottable:
  • Refer to this patch for the changes.
• Resolve require calls and JavaScriptImplementation annotations from snapshot:
  • Currently we are using webpack to create the bundle. We should figure out how to map module ids to webpack indices properly.
• Research what could be appropriate to be evaluated in the snapshot:
  • We should make sure that all modules are included in the snapshot, even those that could get stripped from the bundler dependencies.
• JavaScript debugging:
  • At the moment feeding the V8 engine with snapshot data conflicts with registering a debug message handler. We should figure out how to start the debugger properly, so that it can work with snapshot data.
• Static binding generator
• Currently the Android runtime looks for pre-generated classes from the static binding generator based on filename. When these files are snapshotted, the filename becomes <embedded script> (<embedded> in V8 5.1) and these classes are not found in runtime leaving it to dynamic binding generation.
• Assets extraction:
  • We should measure the performance of extracting the snapshot data blob at runtime.
  • We should measure the increase in the size of the distributed archive when snapshots are enabled.
  • We should measure the memory overhead when using snapshots, if there is one.
  • Snapshot blobs should be excluded from non-Android builds.
  • Memory map the snapshot blob to reduce startup memory
• Rebuild V8:
  • Both mksnapshot and nw-snapshot call CPUFeatures::Probe(bool cross_compile) with the cross-compile flag enabled. We should export this class.
  • Measure the possible performance impacts of disabling extra CPU features. We can use the Octane benchmark on NativeScript for this.
  • Use this patch to report errors during snapshot generation. By default the V8 API only returns NULL on error.
  • Make V8 logs print to Android logcat (disable V8_ANDROID_LOG_STDOUT).
• Obfuscation:
  • {N} Angular has some issues when using minified scripts, like some styles get stripped.
  • The Android static binding generator does not resolve minified Android namespaces. We should figure out a way to preserve the android and java namespaces when minifying.
• Warmup API:
  • Test how the new WarmUpSnapshotDataBlob() API is behaving. It seems that it is introducing enum FunctionCodeHandling { CLEAR_FUNCTION_CODE, KEEP_FUNCTION_CODE } in the serializer options. This should probably now include compiled code and would make the snapshots even faster. There is some progress here.
• Automate snapshot generation:
  • We should figure out how to use the V8 ARM/ARM64 simulator to create a command line tool that generates snapshots during build.

Tasks 2.1

• Automate bundling and snapshot creation in our build infrastructure
  • At the moment, bundle creation is fully automated, but creating snapshot files for different architectures isn't. Angular and {N} Angular are moving quite fast and we should find a way to automate the creation of these files with our builds. This could mean building the V8 mksnapshot tool or using Android emulators to do this.
• Merge the nativescript-angular-snapshot prototype with the existing nativescript-angular plugin
  • Currently, the Angular snapshot demo is an unpublished plugin in GitHub. We should make it the default option (maybe for release builds) with the official {N} Angular plugin. This could potentially lead to more unexpected errors when using snapshots for other apps.
• Decouple the bundler from Angular specifics and integrate it with the tns-core-modules package
  • After the bundling and snapshot creation is fully automated, we can integrate it not only with the {N} Angular plugin, but with the NativeScript core modules, too.
• Expose all bundled modules
  • Some third-party modules get included in the bundle, but are not exposed in the global require override. The modules included in the bundle resolve to the bundled one, but other external modules resolve the module on the file system. This occurs with the reflect-metadata polyfill when running the ng-todo sample.
• Properly join path module id and directory name
  • When only some part of a package is included in the snapshotted bundle, other files from the same package resolve to the file system ones. In them there are some relative require('../../') calls that should be normalized in the require override and be resolved from the bundle. This occurs with the extension methods of rxjs when running the groceries sample.
• Unarchive only the snapshot file for the current architecture
  • The APK file includes snapshot files for all three architectures (ARMv7, ARM64 and x86). This increases the APK file size with 6MB. During the first application launch, all these files are extracted to a total of 24MB. But only the snapshot file for the current architecture is used, so the other ones are useless and should be skipped.
• Lazily evaluate extending Android application and activity
  • Collaborate with the modules team to figure out the lesser evil way to merge NativeScript#2031.
• Research if debugging embedded scripts with the VS Code extension is possible
  • At the moment when we step in the snapshotted script when debugging, the VS Code debugger gets detached. We should research what Node Inspector is doing and if there could be an easy fix for this.

Ping @atanasovg, @KristinaKoeva.

P.S. On the regards of encryption, the heap snapshot turns out to be a poor shot, because all of the JavaScript source seems to be included inside the snapshot data blob.

[NathanaelA]

@jasssonpet Just a FYI, the actual NS runtime (not just angular) has some issues with minifications in the styling system; I ran into this with my NativeScript-Protect; which is one of the reasons I disable minification by default in my encryption system. I suspect the reason why (I haven't traced it fully down yet; since that isn't the purpose of NS-Protect; but because I was curious) and based on some initial tests it looks like the issue is because of the typeName gets set to the wrong name from types.getClass because the minification changes the class names... This might be the same issue you are seeing with Angular...

Good to know that SnapShots won't eliminate the need for my NS-Protect encryption. ;-)