Kernel-based VM snapshot size will increase by 4-5x

[mhausner]

When the VM no longer consumes source (or token streams), the core snapshot needs to contain the Kernel binary of the core libs.

A dill file containing a fully linked hello world program is about 4MB today. The file essentially contains the Kernel binary of the core libraries.

The corelib snapshot with token streams today is about 1MB in size. If we remove the token stream objects (say 300kB) and add the Kernel binary instead, we will have a core snapshot that is somewhere between 4.5 to 5MB.

During development, and for debugging, that 4-5MB snapshot still does not contain sources. We'd need to add the size of source code, or keep the token streams.

We should think about how to reduce this size. A portion of the Kernel binary is unnecessary once the snapshot is loaded. For example, once a VM class object is created for a Dart class X, the Kernel representation of X is no longer needed, but it's probably not possible to delete it since it's just a portion of a large blob.

This issue is just a placeholder for us to remember to consider snapshot size.

[a-siva]

We have some hard numbers from sivachandra@ now

main() {
print('Hello, World\n');
}

Run these commands:

$> ./dart --snapshot-kind=script --snapshot=hello.snapshot_script hello.dart
$> ./dart --snapshot-kind=app-jit --snapshot=hello.snapshot_app_jit hello.dart
$> ./dart --dfe=./gen/kernel-service.dart.snapshot --kernel-binaries=patched_sdk --snapshot-kind=app-jit --snapshot=hello.snapshot_app_jit_k hello.dart
$> ./dart --dfe=./gen/kernel-service.dart.snapshot --kernel-binaries=patched_sdk --snapshot-kind=script --snapshot=hello.snapshot_script_k hello.dart

And then "ls -al hello*" gives this:

-rw-r----- 1 sivachandra eng 38 Aug 22 19:05 hello.dart
-rw-r----- 1 sivachandra eng 366112 Aug 23 18:57 hello.snapshot_app_jit
-rw-r----- 1 sivachandra eng 3651136 Aug 23 18:57 hello.snapshot_app_jit_k
-rw-r----- 1 sivachandra eng 5693 Aug 23 18:57 hello.snapshot_script
-rw-r----- 1 sivachandra eng 208577 Aug 23 18:57 hello.snapshot_script_k

[jensjoha]

Up-to-date data:

$ out/ReleaseX64/dart --snapshot-kind=script --snapshot=hello.snapshot_script hello.dart
$ out/ReleaseX64/dart --snapshot-kind=app-jit --snapshot=hello.snapshot_app_jit hello.dart
$ out/ReleaseX64/dart --dfe=pkg/vm/bin/kernel_service.dart --kernel-binaries=out/ReleaseX64 --packages=.packages --snapshot-kind=script --snapshot=hello.snapshot_script_k hello.dart
$ out/ReleaseX64/dart --dfe=pkg/vm/bin/kernel_service.dart --kernel-binaries=out/ReleaseX64 --packages=.packages --snapshot-kind=app-jit --snapshot=hello.snapshot_app_jit_k hello.dart
$ ls -al hello*
-rw-r----- 1 jensj eng      37 Jan  5 13:38 hello.dart
-rw-r----- 1 jensj eng  392256 Jan 10 09:34 hello.snapshot_app_jit
-rw-r----- 1 jensj eng 2473088 Jan 10 09:34 hello.snapshot_app_jit_k
-rw-r----- 1 jensj eng    5656 Jan 10 09:34 hello.snapshot_script
-rw-r----- 1 jensj eng     715 Jan 10 09:34 hello.snapshot_script_k

As I recall though, that comparing the app-jit snapshot size isn't really apples-to-apples - for instance the tokenstreams for the platform aren't included in the "hello.snapshot_app_jit" because it's already present in the core snapshot.

[a-siva]

We are currently not using core snapshots for Dart 2 and instead just linking the platform.dill file into the executable. This task can only be completed when we deprecate Dart 1 and are able to remove the Dart 1 core snapshot.

[dgrove]

Is this required for Dart2Stable?

[dgrove]

Any updates here?

[kmillikin]

This seems unconnected to the Dart 2.1 release.

[a-siva]

The script snapshot of Dart1 has been replaced as kernel files and the sizes of these kernel files without sources is a lot smaller than the script snapshot.
Dart 1 script snapshot size of hello.dart => 5785
Dart 2 kernel file size of hello.dart without source => 893

The core snapshot linked by the VM has the following comparison between Dart 1 and Dart 2

• The snapshot size for Dart1 (vm_isolate + isolate) was (933600 + 278678) = 1.21 MB
• The current snapshot size for Dart2 (vm_isolate + isolate) is (2743618 + 456251) = 3.19 MB
  We should investigate what makes the Dart2 snapshot size almost 3 times bigger.

Will keep this bug open to see if the size of core snapshot can be reduced.

[rmacnak-google]

Fait accompli. Dart 2 is less space efficient.