merge-similar-functions size optimization pass, which merges similar functions #4414

kateinoigakukun · 2021-12-26T08:22:09Z

Merge similar functions that only differs constant values (like immediate
operand of const and call insts) by parameterization.
Performing this pass at post-link time can merge more functions across
objects. Inspired by Swift compiler's optimization which is derived from
LLVM's one:
https://github.com/apple/swift/blob/main/lib/LLVMPasses/LLVMMergeFunctions.cpp
https://github.com/llvm/llvm-project/blob/main/llvm/docs/MergeFunctions.rst

The basic ideas here are constant value parameterization and direct callee
parameterization by indirection.

Constant value parameterization is like below:

;;  Before
(func $big-const-42 (result i32)
  [[many instr 1]]
  (i32.const 44)
  [[many instr 2]]
)
(func $big-const-43 (result i32)
  [[many instr 1]]
  (i32.const 45)
  [[many instr 2]]
)

;; After
(func $byn$mgfn-shared$big-const-42 (result i32)
  [[many instr 1]]
  (local.get $0) ;; parameterized!!
  [[many instr 2]]
)
(func $big-const-42 (result i32)
  (call $byn$mgfn-shared$big-const-42
   (i32.const 42)
  )
)
(func $big-const-43 (result i32)
  (call $byn$mgfn-shared$big-const-42
   (i32.const 43)
  )
)

Direct callee parameterization is similar to the constant value parameterization,
but it parameterizes callee function i by ref.func instead. Therefore it is enabled
only when reference-types and typed-function-references features are enabled.

I saw more 1 ~ 2 % reduction for SwiftWasm binary and Ruby's wasm port
using wasi-sdk, and 3 ~ 4.5% reduction for Unity WebGL binary when Oz.

Oz Benchmark

original: The original .wasm file (dwarf sections and name section are stripped)
old: Optimized by the latest version (104) of wasm-opt
new: Optimized by wasm-opt built from this branch
new-with-features: Optimized by wasm-opt built from this branch and direct callee parameterization enabled
(with --enable-reference-types --enable-typed-function-references)

Code size

	original (B)	old	new	new-with-features
`rust_game_of_life_bg.wasm`	36765	36150 (-1.67%)	35855 (-2.48%)	35855 (-2.48%)
`swift-hello.wasm`	4825211	2968719 (-38.47%)	2928767 (-39.3%)	2907414 (-39.75%)
`chibi-link.wasm` (by Swift)	21873215	9133106 (-58.25%)	8907066 (-59.28%)	8809156 (-59.73%)
`ruby.wasm`	8592839	8428193 (-1.92%)	8302506 (-3.38%)	8270963 (-3.75%)
`unity-3d-webgl-template.wasm`	7986610	7725181 (-3.27%)	7482455 (-6.31%)	7374878 (-7.66%)

unity-3d-webgl-template.wasm is built from Unity's 3D project template targeting WebGL.

This histogram shows the frequency of diff of reduction (%) compared between old and new-with-features, targeting over 1000 npm packages.

Optimizer performance

	old (ms)	new (ms)
`rust_game_of_life_bg.wasm`	59.8 ± 1.2	57.4 ± 1.5
`swift-hello.wasm`	5.176 ± 0.160	4.892 ± 0.074
`ruby.wasm`	13.510 ± 0.278	12.557 ± 0.335
`unity-3d-webgl-template.wasm`	9.801 ± 0.179	9.291 ± 0.064

Measured on Mac mini (M1 2020)

Runtime performance

	old (ms)	new (ms)
`swift-hello.wasm`	55.4 ± 0.7	56.6 ± 0.5
`ruby.wasm`	87.3 ± 0.6	90.8 ± 0.9

Measured on Mac mini (M1 2020) and wasmtime 0.31.0

kripken · 2022-01-04T17:48:20Z

Interesting, thanks for the PR @kateinoigakukun ! I have not looked at the code yet, but some initial thoughts:

Have you seen Explore Similar Function Elimination (SFE) optimization #1142 ? There is some discussion and investigation there of related things. The numbers there are significantly different than here, so I'm curious how the approach (or the data) differs.
Is there data on how good the Swift one is?
This could be done as a change to the existing DuplicateFunctionElimination pass perhaps: extending the hashing and adding more replacement there. I'm not sure if that's better or not, just curious if you've considered it?

kateinoigakukun · 2022-01-05T09:34:43Z

@kripken

Have you seen Explore Similar Function Elimination (SFE) optimization #1142 ?

I haven't seen it yet, thank you for sharing!
I'm not familiar with asm.js, but as far as I read the code of the tool, my approach does almost the same things.
One possible reason for the regression of my approach is that it accepts only the difference of literals and callees, but the SFE tool does for literals and identifiers. So it gets more merge opportunities, I guess. (I'm not sure what kinds of things can appear as identifiers)

Is there data on how good the Swift one is?

According to the PR, which introduces the pass, it gives ~7% reduction in Swift stdlib. apple/swift#2462

This could be done as a change to the existing DuplicateFunctionElimination pass perhaps: extending the hashing and adding more replacement there. I'm not sure if that's better or not, just curious if you've considered it?

Yes, I've considered it before. DuplicateFunctionElimination is always beneficial from performance and code size aspects. but MergeFunctions is not beneficial in the performance aspect due to thunk overhead, so I decided to create a new pass.

kripken · 2022-01-06T18:03:34Z

(I'm not sure what kinds of things can appear as identifiers)

Hmm, I think for asm.js probably function names and global names were relevant.

Yes, I've considered it before. DuplicateFunctionElimination is always beneficial from performance and code size aspects. but MergeFunctions is not beneficial in the performance aspect due to thunk overhead, so I decided to create a new pass.

Makes sense. I agree they should be run at different times/contexts. But I think we can still share code between them if that is useful. See for example the inlining pass which has an "optimize" parameter, and the simplify-locals pass has several. But I'm not sure how much code could be shared, so it's just a thought.

kateinoigakukun · 2022-01-08T10:45:56Z

But I think we can still share code between them if that is useful.

I took a look at DuplicateFunctionElimination again, but I can't see a sharable code because DFE is pretty simple, I think.

kripken

I'm not done reading yet but here are some comments so far.

src/passes/pass.cpp

test/passes/merge-functions.wast

src/passes/MergeFunctions.cpp

kripken

Please measure the speed impact of these hashing changes. Probably fine, but just to be sure. You can try --duplicate-function-elimination on a large wasm file (let me know if you need a link to one) as that will hash a lot.

test/lit/passes/merge-functions.wast

test/lit/passes/merge-functions_all-features.wast

src/passes/MergeFunctions.cpp

Merge similar functions that only differs constant values (like immediate operand of const and call insts) by parameterization. Performing this pass at post-link time can merge more functions across objects. Inspired by Swift compiler's optimization which is derived from LLVM's one: https://github.com/apple/swift/blob/main/lib/LLVMPasses/LLVMMergeFunctions.cpp https://github.com/llvm/llvm-project/blob/main/llvm/docs/MergeFunctions.rst

kateinoigakukun · 2022-02-13T15:29:50Z

@kripken Sorry for late reply.

Please measure the speed impact of these hashing changes. Probably fine, but just to be sure. You can try --duplicate-function-elimination on a large wasm file (let me know if you need a link to one) as that will hash a lot.

OK, I measured DFE performance on chibi-link.wasm (which contains 63135 functions), and doesn't see big regression 👍

Command	Mean [s]	Min [s]	Max [s]	Relative
`wasm-opt.new --duplicate-function-elimination chibi-link.wasm -o /dev/null`	1.595 ± 0.010	1.584	1.618	1.00 ± 0.01
`wasm-opt.old --duplicate-function-elimination chibi-link.wasm -o /dev/null`	1.595 ± 0.012	1.586	1.626	1.00

kripken

How about MergeFunctions => MergeSimilarFunctions? I think that would be less ambiguous as it clarifies this is about similarity (otherwise it's not clear how this differs from the pass for duplicate function elimination, which also merges functions).

src/passes/MergeFunctions.cpp

kripken · 2022-03-02T00:39:01Z

src/passes/MergeFunctions.cpp

+      // Canonicalize locals indices to make comparison easier.
+      PassRunner runner(module);
+      runner.setIsNested(true);
+      runner.add("reorder-locals");


I think this is not needed as the pass is scheduled in the global post passes, so it is after this function pass has been run. I guess it might help if the pass is run manually by itself, but I don't think it's worth the extra wasted work in the normal case for that.

Makes sense. And I confirmed no size regression after removing reorder-locals in this pass.

kripken

Looking very good! Sorry for the late review.

src/passes/MergeFunctions.cpp

kripken · 2022-03-02T01:00:39Z

src/passes/MergeFunctions.cpp

+  if (primaryFunction->imported()) {
+    return false;
+  }
+  DeepValueIterator primaryIt(&primaryFunction->body);


Why gives the first function a special named iterator? Can't we have a vector of iterators for them all, including the primary?

If it is to make the code a little simpler, then I think I can see that. Maybe the comment on 380 could mention that then.

I couldn't figure out a simpler way 🤔

src/passes/MergeFunctions.cpp

kripken

Thanks!

Code looks good to me. The last thing is to fuzz this carefully. Basically you can run ./scripts/fuzz_opt.py for several hours to see if it finds anything. I'll also run some fuzzing locally.

src/passes/MergeSimilarFunctions.cpp

kateinoigakukun · 2022-03-02T18:18:59Z

@kripken Yeah, I've also started fuzz test 👍

src/passes/MergeSimilarFunctions.cpp

kripken · 2022-03-02T22:11:25Z

This is looking pretty good to me. I've fuzzed 40,000 iterations, and also ran the emscripten -Oz and other test suites. Maybe worth leaving the the fuzzer overnight as a last check.

kateinoigakukun · 2022-03-03T11:50:29Z

Update: I've fuzzed 120,000 iterations for now, and no bug related to this was found

kripken · 2022-03-03T22:17:58Z

Thanks, looks good in my fuzzing as well. Landing, thanks @kateinoigakukun ! This is a very nice feature for -Oz builds.

kateinoigakukun · 2022-03-04T03:39:13Z

Thank you @kripken !

kateinoigakukun force-pushed the katei/merge-funcs branch from 193823d to d6647d0 Compare January 8, 2022 07:57

kateinoigakukun force-pushed the katei/merge-funcs branch from 6453e48 to b0cfbc9 Compare January 10, 2022 08:09

kripken reviewed Jan 12, 2022

View reviewed changes

src/passes/MergeFunctions.cpp Outdated Show resolved Hide resolved

src/passes/MergeFunctions.cpp Outdated Show resolved Hide resolved

kateinoigakukun force-pushed the katei/merge-funcs branch 2 times, most recently from aa446d6 to afa90fe Compare January 14, 2022 01:53

kripken reviewed Jan 27, 2022

View reviewed changes

kateinoigakukun force-pushed the katei/merge-funcs branch 2 times, most recently from b6e51b6 to 6beec1b Compare February 13, 2022 15:12

kateinoigakukun added 11 commits February 14, 2022 00:19

Fix help tests

03e59ed

Sort classes by primary function name to produce stable result

4cdb14c

Put merge-funcs after duplicate-function-elimination

8c6aaf4

Follow the comment line convention

dff9728

Reuse FunctionHasher as much as possible while ignoring consts

61374a4

Make non-lit tests to be litted

a660d44

Remove debugging purpose code

e3c76a5

Update test suites for merge-funcs

6966e5f

Add overview description

c084aed

Remove unnecessary monostate

e030503

kateinoigakukun force-pushed the katei/merge-funcs branch from 6beec1b to e030503 Compare February 13, 2022 15:19

kripken reviewed Mar 2, 2022

View reviewed changes

kateinoigakukun changed the title ~~merge-functions size optimization pass, which merges similar functions~~ merge-similar-functions size optimization pass, which merges similar functions Mar 2, 2022

Rename pass name MergeFunctions => MergeSimilarFunctions

59c4c57

kateinoigakukun added 3 commits March 2, 2022 13:11

Address small review points

9020155

Reuse expression as much as possible to avoid extra allocation

05637e9

Skip reorder-locals in merge-similar-functions because it's already done

ab6dc26

kateinoigakukun force-pushed the katei/merge-funcs branch from 68ce94a to ab6dc26 Compare March 2, 2022 04:11

kripken reviewed Mar 2, 2022

View reviewed changes

src/passes/MergeSimilarFunctions.cpp Outdated Show resolved Hide resolved

Fix broken links introduced by find-replace

6c3f068

kripken reviewed Mar 2, 2022

View reviewed changes

src/passes/MergeSimilarFunctions.cpp Outdated Show resolved Hide resolved

Restore isCallIndirectionEnabled condition

1712128

kripken approved these changes Mar 3, 2022

View reviewed changes

kripken merged commit 6247e7c into WebAssembly:main Mar 3, 2022

kateinoigakukun deleted the katei/merge-funcs branch March 4, 2022 03:48

kripken mentioned this pull request Mar 21, 2022

Fix merge-similar-functions test expections #4534

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

merge-similar-functions size optimization pass, which merges similar functions #4414

merge-similar-functions size optimization pass, which merges similar functions #4414

kateinoigakukun commented Dec 26, 2021 •

edited

kripken commented Jan 4, 2022

kateinoigakukun commented Jan 5, 2022

kripken commented Jan 6, 2022

kateinoigakukun commented Jan 8, 2022

kripken left a comment

kripken left a comment

kateinoigakukun commented Feb 13, 2022

kripken left a comment

kripken Mar 2, 2022

kateinoigakukun Mar 2, 2022

kripken left a comment

kripken Mar 2, 2022

kateinoigakukun Mar 2, 2022

kripken left a comment

kateinoigakukun commented Mar 2, 2022

kripken commented Mar 2, 2022

kateinoigakukun commented Mar 3, 2022

kripken commented Mar 3, 2022

kateinoigakukun commented Mar 4, 2022

merge-similar-functions size optimization pass, which merges similar functions #4414

merge-similar-functions size optimization pass, which merges similar functions #4414

Conversation

kateinoigakukun commented Dec 26, 2021 • edited

Oz Benchmark

Code size

Optimizer performance

Runtime performance

kripken commented Jan 4, 2022

kateinoigakukun commented Jan 5, 2022

kripken commented Jan 6, 2022

kateinoigakukun commented Jan 8, 2022

kripken left a comment

Choose a reason for hiding this comment

kripken left a comment

Choose a reason for hiding this comment

kateinoigakukun commented Feb 13, 2022

kripken left a comment

Choose a reason for hiding this comment

kripken Mar 2, 2022

Choose a reason for hiding this comment

kateinoigakukun Mar 2, 2022

Choose a reason for hiding this comment

kripken left a comment

Choose a reason for hiding this comment

kripken Mar 2, 2022

Choose a reason for hiding this comment

kateinoigakukun Mar 2, 2022

Choose a reason for hiding this comment

kripken left a comment

Choose a reason for hiding this comment

kateinoigakukun commented Mar 2, 2022

kripken commented Mar 2, 2022

kateinoigakukun commented Mar 3, 2022

kripken commented Mar 3, 2022

kateinoigakukun commented Mar 4, 2022

kateinoigakukun commented Dec 26, 2021 •

edited