New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky vm/dart/appjit_determinism_test failure #35092

Open
mkustermann opened this Issue Nov 7, 2018 · 5 comments

Comments

Projects
None yet
3 participants
@mkustermann
Member

mkustermann commented Nov 7, 2018

From this log:

FAILED: dartk-vm product_x64 vm/dart/appjit_determinism_test
Expected: Pass
Actual: RuntimeError

--- Command "vm" (took 02.000488s):
DART_CONFIGURATION=ProductX64 xcodebuild/ProductX64/dart --ignore-unrecognized-flags --packages=/b/s/w/ir/.packages /b/s/w/ir/runtime/tests/vm/dart/appjit_determinism_test.dart

exit code:
255

stdout:
+ xcodebuild/ProductX64/dart --ignore-unrecognized-flags --packages=/b/s/w/ir/.packages --deterministic --snapshot=/b/s/w/it1i5tr1/3f4ZRq/snapshot1 --snapshot-kind=app-jit /b/s/w/ir/runtime/tests/vm/dart/appjit_determinism_test.dart --child
+ xcodebuild/ProductX64/dart --ignore-unrecognized-flags --packages=/b/s/w/ir/.packages --deterministic --snapshot=/b/s/w/it1i5tr1/3f4ZRq/snapshot2 --snapshot-kind=app-jit /b/s/w/ir/runtime/tests/vm/dart/appjit_determinism_test.dart --child

stderr:
Unhandled exception:
Expect.fail('Snapshots are not bitwise equal!')
#0      checkDeterministicSnapshot (file:///b/s/w/ir/runtime/tests/vm/dart/snapshot_test_helper.dart)
#1      _RootZone.runUnary (dart:async/zone.dart:1379:54)
#2      _FutureListener.handleValue (dart:async/future_impl.dart:129:18)
#3      Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:642:45)
#4      Future._propagateToListeners (dart:async/future_impl.dart:671:32)
#5      Future._completeWithValue (dart:async/future_impl.dart:486:5)
#6      Future._asyncComplete.<anonymous closure> (dart:async/future_impl.dart:516:7)
#7      _microtaskLoop (dart:async/schedule_microtask.dart:41:21)
#8      _startMicrotaskLoop (dart:async/schedule_microtask.dart:50:5)
#9      _runPendingImmediateCallback (dart:isolate/runtime/libisolate_patch.dart:115:13)
#10     _RawReceivePortImpl._handleMessage (dart:isolate/runtime/libisolate_patch.dart:172:5)

--- Re-run this test:
python tools/test.py -m product -n dartk-mac-product-x64 vm/dart/appjit_determinism_test

(There were two issues with this test before, in #34638 and #34973, both of which got addressed)

@mraleph
@rmacnak-google Do you know any recent changes which could've caused this?

@rmacnak-google

This comment has been minimized.

Member

rmacnak-google commented Nov 7, 2018

No, the most recent changes to raw_object.h look fine.

I'm not having much luck trying to reproduce. I'll update the test to report the position of the difference so next time we can at least say what type of object is different.

dart-bot pushed a commit that referenced this issue Nov 8, 2018

[test] Debugging aids for low probability flakes in appjit_determinis…
…m_test.

Bug: #35092
Change-Id: I7c487247be600c03e333f4200851da3b00d01515
Reviewed-on: https://dart-review.googlesource.com/c/83521
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>
@sjindel-google

This comment has been minimized.

Contributor

sjindel-google commented Nov 8, 2018

This reproduced again today, and your debugging statement appears to have triggered: https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket.appspot.com/8930438607350824704/+/steps/vm_tests_shard_2/0/stdout

@rmacnak-google

This comment has been minimized.

Member

rmacnak-google commented Nov 8, 2018

The first difference is in an array of length 2050 that starts with

740
0
sentinel
sentinel
sentinel
sentinel
sentinel
sentinel
Type: class 'Iterable', args:[TypeArguments: (@0x7fa48aeeedd1 H136b15d) [TypeParameter: name T; index: 0; function: expand; bound: Object]]
sentinel
sentinel
sentinel
sentinel
sentinel
Type: class 'Iterable', args:[TypeArguments: (@0x7fa48aee4be1 H3b65393c) [TypeParameter: name T; index: 0; function: whereType; bound: Object]]

so probably the canonical type table.

@rmacnak-google

This comment has been minimized.

Member

rmacnak-google commented Nov 9, 2018

The type cluster appears before the array cluster, but the outputs differ first in the array cluster, so I think we've got the same types with a different hashing. The hashes for types are all in terms of CIDs, which are stable, so my suspicion is there's non-determinism in the order the types are created that results in non-determinism in the order of types that have a hash collision appear in the table.

dart-bot pushed a commit that referenced this issue Nov 10, 2018

[test] Check for determinism from the kernel compiler.
Bug: #35092
Change-Id: I405943d47699948665606bc88d3b82e9c899e3d8
Reviewed-on: https://dart-review.googlesource.com/c/84060
Reviewed-by: Siva Annamalai <asiva@google.com>
Commit-Queue: Ryan Macnak <rmacnak@google.com>

dart-bot pushed a commit that referenced this issue Nov 12, 2018

Revert "[test] Check for determinism from the kernel compiler."
This reverts commit a571855.

Reason for revert: ASAN reports memory leak

Original change's description:
> [test] Check for determinism from the kernel compiler.
> 
> Bug: #35092
> Change-Id: I405943d47699948665606bc88d3b82e9c899e3d8
> Reviewed-on: https://dart-review.googlesource.com/c/84060
> Reviewed-by: Siva Annamalai <asiva@google.com>
> Commit-Queue: Ryan Macnak <rmacnak@google.com>

TBR=rmacnak@google.com,asiva@google.com

# Not skipping CQ checks because original CL landed > 1 day ago.

Bug: #35092
Change-Id: I2e494157d7795b3ae3918dfeae66702423217f5d
Reviewed-on: https://dart-review.googlesource.com/c/84029
Reviewed-by: Daco Harkes <dacoharkes@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>

dart-bot pushed a commit that referenced this issue Nov 14, 2018

[test] Check for determinism from the kernel compiler.
Fix leak when generating incremental kernel files.

Bug: #35092
Change-Id: Iaa112150501dd17d943a3747db3cacd2ee4464d2
Reviewed-on: https://dart-review.googlesource.com/c/84301
Commit-Queue: Ryan Macnak <rmacnak@google.com>
Reviewed-by: Siva Annamalai <asiva@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment