dart2wasm fuzzer: Binary files differ

[dcharkes]

=== FAILURE ===
command: out/ReleaseX64/dart pkg/vm/bin/gen_kernel.dart --platform=out/ReleaseX64C/vm_platform.dill --aot --output=out/dartfuzz/7.dill pkg/dart2wasm/bin/dart2wasm.dart
command: out/ReleaseX64C/gen_snapshot --snapshot_kind=app-aot-elf --elf=out/dartfuzz/7.elf out/dartfuzz/7.dill
command: out/ReleaseX64C/dartaotruntime --profiler --sample_buffer_duration=31 out/dartfuzz/7.elf --platform=out/ReleaseX64/dart2wasm_platform.dill --no-source-maps pkg/compiler/lib/src/util/memory_compiler.dart out/dartfuzz/7.js
command: diff out/dartfuzz/expected.js out/dartfuzz/7.js
exitCode: 1
stdout:
Binary files out/dartfuzz/expected.js and out/dartfuzz/7.js differ

(The diff for binary files is not printed.)

https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket/8711578444112148145/+/u/dartfuzz_-_dart2wasm_shard_25/stdout

[osa1]

I don't understand what this test is doing. I see these commands:

command: out/ReleaseX64/dart pkg/vm/bin/gen_kernel.dart --platform=out/ReleaseX64C/vm_platform.dill --aot --output=out/dartfuzz/7.dill pkg/dart2wasm/bin/dart2wasm.dart
command: out/ReleaseX64C/gen_snapshot --snapshot_kind=app-aot-elf --elf=out/dartfuzz/7.elf out/dartfuzz/7.dill
command: out/ReleaseX64C/dartaotruntime --profiler --sample_buffer_duration=31 out/dartfuzz/7.elf --platform=out/ReleaseX64/dart2wasm_platform.dill --no-source-maps pkg/compiler/lib/src/util/memory_compiler.dart out/dartfuzz/7.js
command: diff out/dartfuzz/expected.js out/dartfuzz/7.js

I think the first command compiles dart2wasm to kernel, but I'm unable to run it locally, it generates errors like:

org-dartlang-untranslatable-uri:dart%3Acore:1:1: Error: Dart library 'dart:cli' is not available on this platform.
import 'dart:_internal';
^
org-dartlang-untranslatable-uri:dart%3Acore:2:8: Error: Dart library 'dart:async' is not available on this platform.
import 'dart:async';
       ^
...

What is this test about?

I thought maybe it's compiling the same program to Wasm multiple times and comparing the outputs, but dart2wasm doesn't generate .js files, it generates a .wasm and a .mjs. So not sure what this .js file is about?

Especially puzzled about this command:

command: out/ReleaseX64C/dartaotruntime --profiler --sample_buffer_duration=31 out/dartfuzz/7.elf --platform=out/ReleaseX64/dart2wasm_platform.dill --no-source-maps pkg/compiler/lib/src/util/memory_compiler.dart out/dartfuzz/7.js

I thought this is invoking dart2wasm, but I don't know what the .../7.js part is about. It's not the output file (that would follow a -o), and not the input (that would be memory_compiler.dart).

[rmacnak-google]

it generates errors like

Those are probably from the platform file not existing. You need to also build build.py -mrelease -ax64c.

What is this test about?

It runs dart2wasm two times with different VM configurations. Usually it catches problems in the form of VM crashes, but if the outputs differ it might indicate non-fatal miscompilation by the VM or non-determinism in dart2wasm. (Basically dartfuzz, but for a real, large program instead of a random, small program.) Given it started failing for a long of configurations all at once, it's more likely to be the latter. (It also runs for dart2js, which why the output suffix is misleadingly .js)

It's not the output file

It is the output. This entry point has a different argument parser than the dart compile wasm entry point.

[osa1]

Thanks.

This can be reproduced with just dart compile wasm:

dart compile wasm pkg/compiler/lib/src/util/memory_compiler.dart -o ...

It looks like symbol names differ when we compile the same file multiple times. Diffs of two compilations with the command above look like this:

Diff

--- 7.wat	2025-06-24 12:29:37.902774149 +0100
+++ 8.wat	2025-06-24 12:29:38.462779840 +0100
@@ -185733,7 +185733,7 @@
   (global $S.- (;3980;) (import "S" "-") externref)
   (global $S.0 (;3981;) (import "S" "0") externref)
   (global $S.compilerCrashed (;3982;) (import "S" "compilerCrashed") externref)
-  (global $S._stackTrace=@933399343 (;3983;) (import "S" "_stackTrace=@933399343") externref)
+  (global $S._stackTrace=@134495353 (;3983;) (import "S" "_stackTrace=@134495353") externref)
   (global $S.NoSuchMethodError:_method_not_found:_' (;3984;) (import "S" "NoSuchMethodError: method not found: '") externref)
   (global $S.'_ (;3985;) (import "S" "'\0a") externref)
   (global $S.Receiver:_ (;3986;) (import "S" "Receiver: ") externref)
@@ -185741,7 +185741,7 @@
   (global $S.Symbol__ (;3988;) (import "S" "Symbol(\22") externref)
   (global $S.__ (;3989;) (import "S" "\22)") externref)
   (global $S.IntegerDivisionByZeroException._stackTrace (;3990;) (import "S" "IntegerDivisionByZeroException._stackTrace") externref)
-  (global $S._stackTrace@933399343 (;3991;) (import "S" "_stackTrace@933399343") externref)
+  (global $S._stackTrace@134495353 (;3991;) (import "S" "_stackTrace@134495353") externref)
   (global $S.generic (;3992;) (import "S" "generic") externref)
   (global $S.#_text_ (;3993;) (import "S" "#{text}") externref)
   (global $S.stringExpected (;3994;) (import "S" "stringExpected") externref)
@@ -188496,10 +188496,10 @@
   (global $S._#wc16#formal (;6743;) (import "S" "_#wc16#formal") externref)
   (global $S.DoubleConstant_ (;6744;) (import "S" "DoubleConstant(") externref)
   (global $S._#wc36#formal (;6745;) (import "S" "_#wc36#formal") externref)
-  (global $S._absSubSetSign@276345231 (;6746;) (import "S" "_absSubSetSign@276345231") externref)
-  (global $S._absAndNotSetSign@276345231 (;6747;) (import "S" "_absAndNotSetSign@276345231") externref)
+  (global $S._absSubSetSign@190867311 (;6746;) (import "S" "_absSubSetSign@190867311") externref)
+  (global $S._absAndNotSetSign@190867311 (;6747;) (import "S" "_absAndNotSetSign@190867311") externref)
   (global $S.isNegative (;6748;) (import "S" "isNegative") externref)
-  (global $S._digits@276345231 (;6749;) (import "S" "_digits@276345231") externref)
+  (global $S._digits@190867311 (;6749;) (import "S" "_digits@190867311") externref)
   (global $S._#wc15#formal (;6750;) (import "S" "_#wc15#formal") externref)
   (global $S.IntConstant_ (;6751;) (import "S" "IntConstant(") externref)
   (global $S.Could_not_parse_BigInt (;6752;) (import "S" "Could not parse BigInt") externref)
@@ -190107,8 +190107,8 @@
   (global $S.DynamicModulo (;8354;) (import "S" "DynamicModulo") externref)
   (global $S.Divide (;8355;) (import "S" "Divide") externref)
   (global $S.Add (;8356;) (import "S" "Add") externref)
-  (global $S._absXorSetSign@276345231 (;8357;) (import "S" "_absXorSetSign@276345231") externref)
-  (global $S._absAddSetSign@276345231 (;8358;) (import "S" "_absAddSetSign@276345231") externref)
+  (global $S._absXorSetSign@190867311 (;8357;) (import "S" "_absXorSetSign@190867311") externref)
+  (global $S._absAddSetSign@190867311 (;8358;) (import "S" "_absAddSetSign@190867311") externref)
   (global $S.subtract (;8359;) (import "S" "subtract") externref)
   (global $S.multiply (;8360;) (import "S" "multiply") externref)
   (global $S.divide (;8361;) (import "S" "divide") externref)
@@ -201473,8 +201473,8 @@
   (global $C4529 _InterfaceType (;19718;) (ref $_InterfaceType) (i32.const 10) (i32.const 0) (i32.const 0) (i32.const 2328) (global.get $C1 WasmArray<_Type>[0]) (struct.new $_InterfaceType))
   (global $C4530 ImmutableList<DiagnosticMessage> (;19719;) (ref $ImmutableList) (i32.const 46) (i32.const 0) (global.get $C4529 _InterfaceType) (i64.const 0) (global.get $C4217 WasmArray<Object>[0]) (struct.new $ImmutableList))
   (global $C4531 _StringStackTrace (;19720;) (ref $_StringStackTrace) (i32.const 82) (i32.const 0) (global.get $C21 "") (struct.new $_StringStackTrace))
-  (global $C4532 "_stackTrace=@933399343" (;19721;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._stackTrace=@933399343) (struct.new $JSStringImpl))
-  (global $C4533 SymbolConstant(#_stackTrace=) (;19722;) (ref $Symbol) (i32.const 427) (i32.const 0) (global.get $C4532 "_stackTrace=@933399343") (struct.new $Symbol))
+  (global $C4532 "_stackTrace=@134495353" (;19721;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._stackTrace=@134495353) (struct.new $JSStringImpl))
+  (global $C4533 SymbolConstant(#_stackTrace=) (;19722;) (ref $Symbol) (i32.const 427) (i32.const 0) (global.get $C4532 "_stackTrace=@134495353") (struct.new $Symbol))
   (global $C4534 _InterfaceType (;19723;) (ref $_InterfaceType) (i32.const 10) (i32.const 0) (i32.const 0) (i32.const 2945) (global.get $C1 WasmArray<_Type>[0]) (struct.new $_InterfaceType))
   (global $C4535 _ConstMap<Symbol, Object> (;19724;) (ref $_ConstMap) (i32.const 16) (i32.const 0) (global.get $C4354 WasmArray<WasmI32>[1]) (i64.const 0) (global.get $C4217 WasmArray<Object>[0]) (i64.const 0) (i64.const 0) (global.get $C4534 _InterfaceType) (global.get $C27 _TopType) (struct.new $_ConstMap))
   (global $C4536 ImmutableList<Object> (;19725;) (ref $ImmutableList) (i32.const 46) (i32.const 0) (global.get $C27 _TopType) (i64.const 0) (global.get $C4217 WasmArray<Object>[0]) (struct.new $ImmutableList))
@@ -201488,8 +201488,8 @@
   (global $C4543 "Symbol("" (;19733;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S.Symbol__) (struct.new $JSStringImpl))
   (global $C4544 "")" (;19734;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S.__) (struct.new $JSStringImpl))
   (global $C4545 "IntegerDivisionByZeroException<...>" (;19735;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S.IntegerDivisionByZeroException._stackTrace) (struct.new $JSStringImpl))
-  (global $C4546 "_stackTrace@933399343" (;19736;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._stackTrace@933399343) (struct.new $JSStringImpl))
-  (global $C4547 SymbolConstant(#_stackTrace) (;19737;) (ref $Symbol) (i32.const 427) (i32.const 0) (global.get $C4546 "_stackTrace@933399343") (struct.new $Symbol))
+  (global $C4546 "_stackTrace@134495353" (;19736;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._stackTrace@134495353) (struct.new $JSStringImpl))
+  (global $C4547 SymbolConstant(#_stackTrace) (;19737;) (ref $Symbol) (i32.const 427) (i32.const 0) (global.get $C4546 "_stackTrace@134495353") (struct.new $Symbol))
   (global $C4548 "generic" (;19738;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S.generic) (struct.new $JSStringImpl))
   (global $C4549 MessageKind (;19739;) (ref $MessageKind) (i32.const 153) (i32.const 0) (i64.const 4) (global.get $C4548 "generic") (struct.new $MessageKind))
   (global $C4550 "#{text}" (;19740;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S.#_text_) (struct.new $JSStringImpl))
@@ -207354,14 +207354,14 @@
   (global $C9497 "DoubleConstant(" (;25599;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S.DoubleConstant_) (struct.new $JSStringImpl))
   (global $IntConstantValue._cachedValues (;25600;) (mut (ref null $DefaultMap)) (ref.null none))
   (global $C9498 "_#wc36#formal" (;25601;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._#wc36#formal) (struct.new $JSStringImpl))
-  (global $C9499 "_absSubSetSign@276345231" (;25602;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._absSubSetSign@276345231) (struct.new $JSStringImpl))
-  (global $C9500 SymbolConstant(#_absSubSetSign) (;25603;) (ref $Symbol) (i32.const 427) (i32.const 0) (global.get $C9499 "_absSubSetSign@276345231") (struct.new $Symbol))
-  (global $C9501 "_absAndNotSetSign@276345231" (;25604;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._absAndNotSetSign@276345231) (struct.new $JSStringImpl))
-  (global $C9502 SymbolConstant(#_absAndNotSetSign) (;25605;) (ref $Symbol) (i32.const 427) (i32.const 0) (global.get $C9501 "_absAndNotSetSign@276345231") (struct.new $Symbol))
+  (global $C9499 "_absSubSetSign@190867311" (;25602;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._absSubSetSign@190867311) (struct.new $JSStringImpl))
+  (global $C9500 SymbolConstant(#_absSubSetSign) (;25603;) (ref $Symbol) (i32.const 427) (i32.const 0) (global.get $C9499 "_absSubSetSign@190867311") (struct.new $Symbol))
+  (global $C9501 "_absAndNotSetSign@190867311" (;25604;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._absAndNotSetSign@190867311) (struct.new $JSStringImpl))
+  (global $C9502 SymbolConstant(#_absAndNotSetSign) (;25605;) (ref $Symbol) (i32.const 427) (i32.const 0) (global.get $C9501 "_absAndNotSetSign@190867311") (struct.new $Symbol))
   (global $C9503 _InterfaceType (;25606;) (ref $_InterfaceType) (i32.const 10) (i32.const 0) (i32.const 0) (i32.const 72) (global.get $C1 WasmArray<_Type>[0]) (struct.new $_InterfaceType))
   (global $C9504 "isNegative" (;25607;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S.isNegative) (struct.new $JSStringImpl))
-  (global $C9505 "_digits@276345231" (;25608;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._digits@276345231) (struct.new $JSStringImpl))
-  (global $C9506 SymbolConstant(#_digits) (;25609;) (ref $Symbol) (i32.const 427) (i32.const 0) (global.get $C9505 "_digits@276345231") (struct.new $Symbol))
+  (global $C9505 "_digits@190867311" (;25608;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._digits@190867311) (struct.new $JSStringImpl))
+  (global $C9506 SymbolConstant(#_digits) (;25609;) (ref $Symbol) (i32.const 427) (i32.const 0) (global.get $C9505 "_digits@190867311") (struct.new $Symbol))
   (global $C9507 "_#wc15#formal" (;25610;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._#wc15#formal) (struct.new $JSStringImpl))
   (global $C9508 "IntConstant(" (;25611;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S.IntConstant_) (struct.new $JSStringImpl))
   (global $C9509 "Could not parse BigInt" (;25612;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S.Could_not_parse_BigInt) (struct.new $JSStringImpl))
@@ -210763,10 +210763,10 @@
   (global $C12273 "Divide" (;29008;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S.Divide) (struct.new $JSStringImpl))
   (global $C12274 "Add" (;29009;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S.Add) (struct.new $JSStringImpl))
   (global $C12275 BitXorOperation (;29010;) (ref $BitXorOperation) (i32.const 594) (i32.const 0) (global.get $C4401 "^") (struct.new $BitXorOperation))
-  (global $C12276 "_absXorSetSign@276345231" (;29011;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._absXorSetSign@276345231) (struct.new $JSStringImpl))
-  (global $C12277 SymbolConstant(#_absXorSetSign) (;29012;) (ref $Symbol) (i32.const 427) (i32.const 0) (global.get $C12276 "_absXorSetSign@276345231") (struct.new $Symbol))
-  (global $C12278 "_absAddSetSign@276345231" (;29013;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._absAddSetSign@276345231) (struct.new $JSStringImpl))
-  (global $C12279 SymbolConstant(#_absAddSetSign) (;29014;) (ref $Symbol) (i32.const 427) (i32.const 0) (global.get $C12278 "_absAddSetSign@276345231") (struct.new $Symbol))
+  (global $C12276 "_absXorSetSign@190867311" (;29011;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._absXorSetSign@190867311) (struct.new $JSStringImpl))
+  (global $C12277 SymbolConstant(#_absXorSetSign) (;29012;) (ref $Symbol) (i32.const 427) (i32.const 0) (global.get $C12276 "_absXorSetSign@190867311") (struct.new $Symbol))
+  (global $C12278 "_absAddSetSign@190867311" (;29013;) (ref $JSStringImpl) (i32.const 4) (i32.const 0) (global.get $S._absAddSetSign@190867311) (struct.new $JSStringImpl))
+  (global $C12279 SymbolConstant(#_absAddSetSign) (;29014;) (ref $Symbol) (i32.const 427) (i32.const 0) (global.get $C12278 "_absAddSetSign@190867311") (struct.new $Symbol))
   (global $C12280 _GvnType (;29015;) (ref $_GvnType) (i32.const 235) (i32.const 0) (i64.const 10) (global.get $C8588 "bitXor") (struct.new $_GvnType))
   (global $C12281 BitOrOperation (;29016;) (ref $BitOrOperation) (i32.const 593) (i32.const 0) (global.get $C5262 "|") (struct.new $BitOrOperation))
   (global $C12282 _GvnType (;29017;) (ref $_GvnType) (i32.const 235) (i32.const 0) (i64.const 8) (global.get $C8590 "bitOr") (struct.new $_GvnType))

If I compile with -O3 --strip-wasm then I get identical .wats but cmp -s still reports changes in .wasm files.

(Note: --strip-wasm requires -O greater than 1, because with -O0 we don't run wasm-opt which is what removes the symbols)

I don't know what's generating those @... suffixes in symbol names.

Do we want the binaries to be identical in all optimization settings/flag combinations?

[mkustermann]

This is probably what I introduced recently, I can fix it.

[mkustermann]

This was probably introduced in my change in 8b495a0, uploaded cl/436623 to fix it.

If I compile with -O3 --strip-wasm then I get identical .wats but cmp -s still reports changes in .wasm files.

After my change it seems stable. But it's important to use the same output filename in multiple runs because we encode the source map url in the wasm file, so if one does dart compile wasm -O a.wasm ... and dart compile wasm -O b.wasm ... - they will always be different because of a different sourceMappingUrl in it.