Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remaining tasks for allowing flutter tools to use dart compile wasm instead of invoking dart2wasm AOT snapshot & wasm-opt directly #54675

Closed
mkustermann opened this issue Jan 19, 2024 · 4 comments
Closed

Remaining tasks for allowing flutter tools to use dart compile wasm instead of invoking dart2wasm AOT snapshot & wasm-opt directly #54675

mkustermann opened this issue Jan 19, 2024 · 4 comments
Assignees
@mkustermann
Labels
area-dart2wasm Issues for the dart2wasm compiler.

Comments

@mkustermann
Copy link
Member

mkustermann commented Jan 19, 2024 •
edited
Loading

Here's a list of remaining things we probably need to do before dart compile wasm can be used by flutter tools (right now it invokes the dart2wasm AOT snapshot burried in our SDK directly and wasm-opt with it's own set of binaryen flags):

  • Performance 16x slower (see below)
  • Support -D defines
  • Support --dart-sdk & --platform
  • Support --import-shared-memory & --shared-memory-max-pages
  • Support --depfile
  • Support passing through arbitrary args to dart2wasm (may also be used to do the above) - similar to (--extra-gen-snapshot-options in VM)

Performance
dart compile wasm is running the dart2wasm compiler in it's implementation (it doesn't use the dart2wasm AOT snapshot). Though the DartDev AppJIT training doesn't include dart2wasm compilation. That means the code will run slow.

It takes 8.5 seconds to run dart compile wasm vs 0.5 seconds by running the AOT snapshot (what flutter does):

% echo 'main() {}' > hello.dart

% time out/ReleaseX64/dart-sdk/bin/dart compile wasm --no-optimize hello.dart
*NOTE*: Compilation to WasmGC is experimental.
The support may change, or be removed, with no advance notice.

Generated wasm module 'hello.wasm', and JS init file 'hello.mjs'.
out/ReleaseX64/dart-sdk/bin/dart compile wasm --no-optimize hello.dart  12.87s user 1.81s system 173% cpu 8.471 total

% time out/ReleaseX64/dart-sdk/bin/dartaotruntime out/ReleaseX64/dart-sdk/bin/snapshots/dart2wasm_product.snapshot --platform=$PWD/out/ReleaseX64/dart-sdk/lib/_internal/dart2wasm_platform.dill --dart-sdk=$PWD/out/ReleaseX64/dart-sdk hello.dart hello.wasm
out/ReleaseX64/dart-sdk/bin/dartaotruntime    hello.dart hello.wasm  0.48s user 0.15s system 111% cpu 0.564 total

It's a little unclear to my why we wouldn't just run the dart2wasm AOT snapshot that we already bundle in the SDK for doing the compilation.

/cc @mit-mit
@mkustermann mkustermann added the area-dart2wasm Issues for the dart2wasm compiler. label Jan 19, 2024
@mit-mit
Copy link
Member

mit-mit commented Jan 19, 2024

About the performance part, I believe @bkonyi and @a-siva have been looking at running all of dartdev as AOT. Any news about that?

@a-siva
Copy link
Contributor

a-siva commented Jan 20, 2024

We could switch the implementation of 'compile dart2wasm' to spawn a process using the AOT snapshot instead of the way it is currently run (included in dartdev appJIT with no training for dart2wasm). This might speed it up. It appears to already spawn a process for the optimize step.

Work on running dartdev itself in AOT mode is progressing, we should not wait for that to complete. Let me take a look at switching the implementation and see how fast it gets.

@mkustermann
Copy link
Member Author

We could switch the implementation of 'compile dart2wasm' to spawn a process using the AOT snapshot instead of the way it is currently run (included in dartdev appJIT with no training for dart2wasm). This might speed it up. It appears to already spawn a process for the optimize step.

Already made cl/347902 to do that.

Though it's still meaningfully slower than running the AOT snapshot directly, seemingly going via DartDev middle man adds 1 second (which seems extraordinary high given that does very little except for shelling out to subprocesses).

@mkustermann mkustermann self-assigned this Jan 23, 2024
copybara-service bot pushed a commit that referenced this issue Jan 24, 2024
@mkustermann
[dart2wasm] Make dart compile wasm use AOT-compiled dart2wasm via s…
6788b73 
…ubprocess

This reduces time for `dart compile wasm` on a hello world in

* `--no-optimize` mode from 8.2 to 1.8 seconds (0.6 sec via [0])
* `--optimize` mode from 9.2 to 3 seconds (1.6 sec via [0])

[0] pkg/dart2wasm/tool/compile_benchmark

Issue #54675

Change-Id: I47093e747f343b542bc7faa34e102c62657c7b81
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/347902
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
copybara-service bot pushed a commit that referenced this issue Jan 24, 2024
@mkustermann
[dart2wasm] Make dart compile wasm pass through -D & --minify flags…
89060a6 
… to dart2wasm

This also fixes some code that was assuming the args parser returns
`Map<String, String>` for the `--define` multi-option, it's rather returning
a `List<String>`.

Forwarding these two flags will allow running some of our configurations
using `dart compile wasm` as opposed to
`pkg/dart2wasm/tool/compile_benchmark` (though the former is still
slower than the ladder).

Though since `dart compile wasm` is still a bit slower then the shell
script, we'll do that only for one configuration.

Issue #54675

Change-Id: I74e9edb2f635f48faade8d843857be2aa7c6066f
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/347903
Reviewed-by: Slava Egorov <vegorov@google.com>
Reviewed-by: Michael Thomsen <mit@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
copybara-service bot pushed a commit that referenced this issue Jan 26, 2024
@mkustermann
[dart2wasm] Align semantics of -O3 with dart2js semantics
b562998 
* Golem is now using -O3
   => remove `--omit-checks` from pkg/dart2wasm/tool/compile_benchmark.

* Dart CI is using -O1/-O2
  => remove `--optimize`/`--no-optimize` from
     pkg/dart2wasm/tool/compile_benchmark and `dart compile wasm`

* Align semantics of -3 with dartjs (enable `--minify`, enable
  `--omit-implicit-checks` disable  `--omit-explicit-checks`).
  => This will make us see changes in benchmarks.

What remains is

* Expose remaining flags in `dart compile wasm` that are needed for
  flutter (or add a generic `--extra-compiler-args` that forwards flags)

* Migrate flutter to use `dart compile wasm`.

* Remove `--omit-type-checks` from pkg/dart2wasm/lib/dart2wasm.dart

Issue #54675

Change-Id: I80654a3ae81bdc5f4c57e3fadccdf5612236102a
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/348500
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
copybara-service bot pushed a commit that referenced this issue Feb 1, 2024
@mkustermann
[dart2wasm] Add --extra-compiler-option to dart compile wasm command
83b7a57 
By allowing a pass-through of arbitrary arguments to the dart2wasm
compiler we allow flutter to use options which

* aren't relevant for end-users of `dart compile wasm`
  (e.g. such as providing different platform dill file)

* we may break at any point in time (though in coordination with
  flutter)

Issue #54675

Change-Id: I7da2b69a8c642f8bfaa6546bf9c1a6c3833d55fa
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/349720
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
copybara-service bot pushed a commit that referenced this issue Feb 8, 2024
@mkustermann
[dart2wasm] Allow --minify/--no-minify to override defaults from -O<l…
a46d59f 
…evel>

* Allow --minify/--no-minify to override defaults from -O<level>

* Remove the unused(!) --name-section option from pkg/dart2wasm/*
  => Stripping it only makes sense at the binaryen step

* Make pkg/dart2wasm/tool/compile_benchmark support -g/--no-minify
  => Only useful for dart2wasm developers

Issue #54675

Change-Id: I695e985897b212fc345dcfd776553a009723d3b6
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/351121
Reviewed-by: Slava Egorov <vegorov@google.com>
Commit-Queue: Martin Kustermann <kustermann@google.com>
@mkustermann mkustermann mentioned this issue Feb 12, 2024
Merged
mkustermann added a commit to flutter/flutter that referenced this issue Feb 14, 2024
@mkustermann
Use dart compile wasm for wasm compilations (#143298)
abadf9f 
* Flags to `dart compile wasm`

Some options are not relevant to a standalone user of `dart compile
wasm` (e.g. specyfing dart-sdk, platform file etc). => Those aren't
offered by the `dart compile wasm` tool directly. => We use the
`--extra-compiler-option=` instead which passes through arbitrary
options to the dart2wasm compiler. => We don't maintain compatibility of
those options, if we update them we'll ensure to also update flutter
tools

* Binaryen optimization passes

This change will mean we use the binaryen flags from Dart SDK which are
slightly different from the ones in flutter.

* Optimization configuration

This change will also start using the more standardized `-O` flag for
determining optimization levels. The meaning of those flags have been
mostly aligned with dart2js (with some differences remaining).

* Minimization

Using the new optimization flags, namely `-O4` for `--wasm-opt=full`,
will automatically enable the new `--minify` support. Minification is
Dart semantics preserving but changes the `<obj>.runtimeType.toString()`
to use minified names (just as in dart2js).

* Code size changes

  Overall this change will reduce wonderous code size by around 10%.

Issue dart-lang/sdk#54675
@mkustermann
Copy link
Member Author

I've just landed flutter/flutter#143298 which makes flutter now use dart compile wasm. That will conclude what I've planned here.

Though I'd like to repeat what I wrote above:

Though it's still meaningfully slower than running the AOT snapshot directly, seemingly going via DartDev middle man adds 1 second (which seems extraordinary high given that does very little except for shelling out to subprocesses).

I think the owners of dartdev should continue optimizing the performance of various dart <command>s. It's not as fast as it should be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mkustermann mkustermann

Labels
area-dart2wasm Issues for the dart2wasm compiler.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mkustermann @a-siva @mit-mit