Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HeapSnapshot generation flakily fails on a Smi not being present in object id map #50341

Closed
mkustermann opened this issue Oct 31, 2022 · 1 comment
Closed

HeapSnapshot generation flakily fails on a Smi not being present in object id map #50341

mkustermann opened this issue Oct 31, 2022 · 1 comment
Assignees
@sstrickl
Labels
area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. gardening

Comments

@mkustermann
Copy link
Member

See e.g. this log:



/=============================================================================\
| vm/dart_2/heap_snapshot_referrers_test broke (Pass -> Crash, expected Pass) |
\=============================================================================/

--- Command "vm_compile_to_kernel" (took 04.000030s):
DART_CONFIGURATION=DebugX64 /b/s/w/ir/pkg/vm/tool/gen_kernel --aot --platform=out/DebugX64/vm_platform_strong.dill -o /b/s/w/ir/out/DebugX64/generated_compilations/dartkp-linux-debug-x64/runtime_tests_vm_dart_2_heap_snapshot_referrers_test/out.dill /b/s/w/ir/runtime/tests/vm/dart_2/heap_snapshot_referrers_test.dart -Dtest_runner.configuration=dartkp-linux-debug-x64 --packages=/b/s/w/ir/.dart_tool/package_config.json -Ddart.vm.product=false

exit code:
0

--- Command "precompiler" (took 19.000216s):
DART_CONFIGURATION=DebugX64 out/DebugX64/gen_snapshot --snapshot-kind=app-aot-assembly --assembly=/b/s/w/ir/out/DebugX64/generated_compilations/dartkp-linux-debug-x64/runtime_tests_vm_dart_2_heap_snapshot_referrers_test/out.S -Dtest_runner.configuration=dartkp-linux-debug-x64 --ignore-unrecognized-flags --packages=/b/s/w/ir/.dart_tool/package_config.json /b/s/w/ir/out/DebugX64/generated_compilations/dartkp-linux-debug-x64/runtime_tests_vm_dart_2_heap_snapshot_referrers_test/out.dill

exit code:
0

--- Command "remove_kernel_file" (took 8ms):
DART_CONFIGURATION=DebugX64 rm /b/s/w/ir/out/DebugX64/generated_compilations/dartkp-linux-debug-x64/runtime_tests_vm_dart_2_heap_snapshot_referrers_test/out.dill

exit code:
0

--- Command "assemble" (took 01.000024s):
DART_CONFIGURATION=DebugX64 gcc -m64 -Wl,--no-undefined -shared -o /b/s/w/ir/out/DebugX64/generated_compilations/dartkp-linux-debug-x64/runtime_tests_vm_dart_2_heap_snapshot_referrers_test/out.aotsnapshot /b/s/w/ir/out/DebugX64/generated_compilations/dartkp-linux-debug-x64/runtime_tests_vm_dart_2_heap_snapshot_referrers_test/out.S

exit code:
0

--- Command "remove_assembly" (took 12ms):
DART_CONFIGURATION=DebugX64 rm /b/s/w/ir/out/DebugX64/generated_compilations/dartkp-linux-debug-x64/runtime_tests_vm_dart_2_heap_snapshot_referrers_test/out.S

exit code:
0

--- Command "vm" (took 137ms):
DART_CONFIGURATION=DebugX64 out/DebugX64/dart_precompiled_runtime -Dtest_runner.configuration=dartkp-linux-debug-x64 --ignore-unrecognized-flags --packages=/b/s/w/ir/.dart_tool/package_config.json /b/s/w/ir/out/DebugX64/generated_compilations/dartkp-linux-debug-x64/runtime_tests_vm_dart_2_heap_snapshot_referrers_test/out.aotsnapshot

exit code:
-6

stderr:
../../runtime/vm/object_graph.cc: 890: error: expected: id != 0
version=2.19.0-edge.3b2b126f601347beeb2d9dea2391ec28008cf85a (be) (Fri Oct 28 12:45:35 2022 +0000) on "linux_x64"
pid=20326, thread=20331, isolate_group=main(0x5616c8a4a800), isolate=main(0x5616c8a9a000)
os=linux, arch=x64, comp=no, sim=no
isolate_instructions=7fca9c711c00, vm_instructions=7fca9c70c680
  pc 0x00005616c67bed8c fp 0x00007fca9b87d050 dart::Profiler::DumpStackTrace(void*)+0x7c
  pc 0x00005616c645bd64 fp 0x00007fca9b87d130 dart::Assert::Fail(char const*, ...) const+0x84
  pc 0x00005616c67864b6 fp 0x00007fca9b87d160 out/DebugX64/dart_precompiled_runtime+0x9954b6
  pc 0x00005616c678cdf8 fp 0x00007fca9b87d1a0 out/DebugX64/dart_precompiled_runtime+0x99bdf8
  pc 0x00005616c678c82f fp 0x00007fca9b87d210 out/DebugX64/dart_precompiled_runtime+0x99b82f
  pc 0x00005616c687b1e6 fp 0x00007fca9b87d270 dart::Page::VisitObjects(dart::ObjectVisitor*) const+0x116
  pc 0x00005616c688997b fp 0x00007fca9b87d2b0 dart::Scavenger::VisitObjects(dart::ObjectVisitor*) const+0x8b
  pc 0x00005616c686fc17 fp 0x00007fca9b87d2d0 dart::HeapIterationScope::IterateObjects(dart::ObjectVisitor*) const+0x17
  pc 0x00005616c67883c4 fp 0x00007fca9b87d4d0 dart::HeapSnapshotWriter::Write()+0x1844
  pc 0x00005616c65f80f2 fp 0x00007fca9b87d5d0 dart::BootstrapNatives::DN_Internal_writeHeapSnapshotToFile(dart::Thread*, dart::Zone*, dart::NativeArguments*)+0x182
  pc 0x00005616c66c29f6 fp 0x00007fca9b87db00 dart::NativeEntry::BootstrapNativeCallWrapper(_Dart_NativeArguments*, void (*)(_Dart_NativeArguments*))+0xd6
  pc 0x00005616c66c3039 fp 0x00007fca9b87e110 dart::NativeEntry::LinkNativeCall(_Dart_NativeArguments*)+0x3a9
  pc 0x00007fca9c70f5fb fp 0x00007fca9b87e140 _kDartVmSnapshotInstructions+0x2f7b
  pc 0x00007fca9c7e61d4 fp 0x00007fca9b87e160 _kDartIsolateSnapshotInstructions+0xd45d4
  pc 0x00007fca9c7e2e0c fp 0x00007fca9b87e1e8 _kDartIsolateSnapshotInstructions+0xd120c
  pc 0x00007fca9c7e6539 fp 0x00007fca9b87e260 _kDartIsolateSnapshotInstructions+0xd4939
  pc 0x00007fca9c7e63fb fp 0x00007fca9b87e288 _kDartIsolateSnapshotInstructions+0xd47fb
  pc 0x00007fca9c7e67b3 fp 0x00007fca9b87e298 _kDartIsolateSnapshotInstructions+0xd4bb3
  pc 0x00007fca9c7e9737 fp 0x00007fca9b87e2f8 _kDartIsolateSnapshotInstructions+0xd7b37
  pc 0x00007fca9c75cc89 fp 0x00007fca9b87e320 _kDartIsolateSnapshotInstructions+0x4b089
  pc 0x00007fca9c7e9a37 fp 0x00007fca9b87e390 _kDartIsolateSnapshotInstructions+0xd7e37
  pc 0x00007fca9c7735e3 fp 0x00007fca9b87e3b8 _kDartIsolateSnapshotInstructions+0x619e3
  pc 0x00007fca9c70f374 fp 0x00007fca9b87e430 _kDartVmSnapshotInstructions+0x2cf4
  pc 0x00005616c6666c05 fp 0x00007fca9b87e4c0 dart::DartEntry::InvokeCode(dart::Code const&, unsigned long, dart::Array const&, dart::Array const&, dart::Thread*)+0xf5
  pc 0x00005616c6666893 fp 0x00007fca9b87e530 dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long)+0x153
  pc 0x00005616c66697ab fp 0x00007fca9b87e590 dart::DartLibraryCalls::HandleMessage(long, dart::Instance const&)+0x10b
  pc 0x00005616c668dfb2 fp 0x00007fca9b87eb30 dart::IsolateMessageHandler::HandleMessage(std::__2::unique_ptr<dart::Message, std::__2::default_delete<dart::Message>>)+0x3b2
  pc 0x00005616c66a2864 fp 0x00007fca9b87ebc0 dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)+0x214
  pc 0x00005616c66a35e3 fp 0x00007fca9b87ec40 dart::MessageHandler::TaskCallback()+0x333
  pc 0x00005616c684ea00 fp 0x00007fca9b87ecd0 dart::ThreadPool::WorkerLoop(dart::ThreadPool::Worker*)+0x1e0
  pc 0x00005616c684f37a fp 0x00007fca9b87ed20 dart::ThreadPool::Worker::Main(unsigned long)+0x14a
  pc 0x00005616c67b9d02 fp 0x00007fca9b87ee70 out/DebugX64/dart_precompiled_runtime+0x9c8d02
-- End of DumpStackTrace
../../runtime/vm/stack_frame.cc: 356: error: expected: code != Code::null()
Aborting re-entrant request for stack trace.

--- Re-run this test:
python3 tools/test.py -n dartkp-linux-debug-x64 vm/dart_2/heap_snapshot_referrers_test

or this log


/=================================================================================\
| vm/dart_2/heap_snapshot_regress_49710_test broke (Pass -> Crash, expected Pass) |
\=================================================================================/

--- Command "vm_compile_to_kernel" (took 33.000245s):
DART_CONFIGURATION=DebugSIMARM /b/s/w/ir/pkg/vm/tool/gen_kernel --aot --platform=out/DebugSIMARM/vm_platform_strong.dill -o /b/s/w/ir/out/DebugSIMARM/generated_compilations/dartkp-linux-debug-simarm-crossword/runtime_tests_vm_dart_2_heap_snapshot_regress_49710_test/out.dill /b/s/w/ir/runtime/tests/vm/dart_2/heap_snapshot_regress_49710_test.dart -Dtest_runner.configuration=dartkp-linux-debug-simarm-crossword --packages=/b/s/w/ir/.dart_tool/package_config.json -Ddart.vm.product=false

exit code:
0

--- Command "precompiler" (took 17.000349s):
DART_CONFIGURATION=DebugSIMARM out/DebugSIMARM_X64/gen_snapshot --snapshot-kind=app-aot-elf --elf=/b/s/w/ir/out/DebugSIMARM/generated_compilations/dartkp-linux-debug-simarm-crossword/runtime_tests_vm_dart_2_heap_snapshot_regress_49710_test/out.aotsnapshot --loading-unit-manifest=/b/s/w/ir/out/DebugSIMARM/generated_compilations/dartkp-linux-debug-simarm-crossword/runtime_tests_vm_dart_2_heap_snapshot_regress_49710_test/ignored.json -Dtest_runner.configuration=dartkp-linux-debug-simarm-crossword --ignore-unrecognized-flags --packages=/b/s/w/ir/.dart_tool/package_config.json /b/s/w/ir/out/DebugSIMARM/generated_compilations/dartkp-linux-debug-simarm-crossword/runtime_tests_vm_dart_2_heap_snapshot_regress_49710_test/out.dill

exit code:
0

--- Command "remove_kernel_file" (took 30ms):
DART_CONFIGURATION=DebugSIMARM rm /b/s/w/ir/out/DebugSIMARM/generated_compilations/dartkp-linux-debug-simarm-crossword/runtime_tests_vm_dart_2_heap_snapshot_regress_49710_test/out.dill

exit code:
0

--- Command "vm" (took 237ms):
DART_CONFIGURATION=DebugSIMARM out/DebugSIMARM/dart_precompiled_runtime -Dtest_runner.configuration=dartkp-linux-debug-simarm-crossword --ignore-unrecognized-flags --packages=/b/s/w/ir/.dart_tool/package_config.json /b/s/w/ir/out/DebugSIMARM/generated_compilations/dartkp-linux-debug-simarm-crossword/runtime_tests_vm_dart_2_heap_snapshot_regress_49710_test/out.aotsnapshot

exit code:
-6

stderr:
../../runtime/vm/object_graph.cc: 890: error: expected: id != 0
version=2.19.0-edge.21039dc83b6594411d1e6a26a411466fab4ec625 (be) (Wed Oct 26 23:27:40 2022 +0000) on "linux_simarm"
pid=3087, thread=3093, isolate_group=main(0x58728480), isolate=main(0x58736500)
os=linux, arch=arm, comp=no, sim=yes
isolate_instructions=f7b58880, vm_instructions=f7b53000
  pc 0x56f2f16b fp 0xf6d7d858 dart::Profiler::DumpStackTrace(void*)+0x9b
  pc 0x572eaa81 fp 0xf6d7d878 Dart_DumpNativeStackTrace+0x21
  pc 0x56bb3f66 fp 0xf6d7d8a8 dart::Assert::Fail(char const*, ...) const+0x36
  pc 0x56ef0eec fp 0xf6d7d8f8 out/DebugSIMARM/dart_precompiled_runtime+0x905eec
  pc 0x56ef7e36 fp 0xf6d7d958 out/DebugSIMARM/dart_precompiled_runtime+0x90ce36
  pc 0x56ef77a1 fp 0xf6d7da08 out/DebugSIMARM/dart_precompiled_runtime+0x90c7a1
  pc 0x56ffde93 fp 0xf6d7da68 dart::Page::VisitObjects(dart::ObjectVisitor*) const+0x123
  pc 0x5700cc3a fp 0xf6d7daa8 dart::Scavenger::VisitObjects(dart::ObjectVisitor*) const+0x8a
  pc 0x56ff189e fp 0xf6d7dae8 dart::HeapIterationScope::IterateObjects(dart::ObjectVisitor*) const+0x2e
  pc 0x56ef30b9 fp 0xf6d7dc78 dart::HeapSnapshotWriter::Write()+0x1a49
  pc 0x56d48bc0 fp 0xf6d7dd28 dart::BootstrapNatives::DN_Internal_writeHeapSnapshotToFile(dart::Thread*, dart::Zone*, dart::NativeArguments*)+0x190
  pc 0x56e2358f fp 0xf6d7e038 dart::NativeEntry::BootstrapNativeCallWrapper(_Dart_NativeArguments*, void (*)(_Dart_NativeArguments*))+0xaf
  pc 0x56e23b6f fp 0xf6d7e3d8 dart::NativeEntry::LinkNativeCall(_Dart_NativeArguments*)+0x3ef
  pc 0x56e2363d fp 0xf6d7e6e8 dart::NativeEntry::BootstrapNativeCallWrapper(_Dart_NativeArguments*, void (*)(_Dart_NativeArguments*))+0x15d
  pc 0x56fa9be7 fp 0xf6d7e808 dart::Simulator::SupervisorCall(dart::Instr*)+0x387
  pc 0x56faaeb0 fp 0xf6d7e8b8 dart::Simulator::DecodeType7(dart::Instr*)+0xd0
  pc 0x56fad43e fp 0xf6d7e9a8 dart::Simulator::Execute()+0x35e
  pc 0x56fae872 fp 0xf6d7ea68 dart::Simulator::Call(int, int, int, int, int, bool, bool)+0x162
  pc 0x56dbf218 fp 0xf6d7ead8 dart::DartEntry::InvokeCode(dart::Code const&, unsigned int, dart::Array const&, dart::Array const&, dart::Thread*)+0x108
  pc 0x56dbeff3 fp 0xf6d7eb38 dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned int)+0x1a3
  pc 0x56dbedf7 fp 0xf6d7eb88 dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&)+0xa7
  pc 0x56dc22c5 fp 0xf6d7ebf8 dart::DartLibraryCalls::HandleMessage(long long, dart::Instance const&)+0x125
  pc 0x56dea7e1 fp 0xf6d7ef78 dart::IsolateMessageHandler::HandleMessage(std::__2::unique_ptr<dart::Message, std::__2::default_delete<dart::Message>>)+0x421
  pc 0x56e006b4 fp 0xf6d7eff8 dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)+0x244
  pc 0x56e014f3 fp 0xf6d7f068 dart::MessageHandler::TaskCallback()+0x353
  pc 0x56e028a9 fp 0xf6d7f0a8 out/DebugSIMARM/dart_precompiled_runtime+0x8178a9
  pc 0x56fcdfa5 fp 0xf6d7f128 dart::ThreadPool::WorkerLoop(dart::ThreadPool::Worker*)+0x205
  pc 0x56fce91e fp 0xf6d7f178 dart::ThreadPool::Worker::Main(unsigned int)+0x14e
  pc 0x56f29e7b fp 0xf6d7f2d8 out/DebugSIMARM/dart_precompiled_runtime+0x93ee7b
  pc 0xf7f4e3bd fp 0xf6d7f3a8 /lib/i386-linux-gnu/libpthread.so.0+0x63bd
-- End of DumpStackTrace
../../runtime/vm/stack_frame.cc: 356: error: expected: code != Code::null()
Aborting re-entrant request for stack trace.

--- Re-run this test:
python3 tools/test.py -n dartkp-linux-debug-simarm-crossword vm/dart_2/heap_snapshot_regress_49710_test

This may have started with 2d6037a which introduced Smis into the heapsnapshot.

The phase-1 visitor should assign object-ids to all needed Smis and phase-2 visitor will get the object-ids. Though it seems it flakily hits a Smi which wasn't visited in phase-1 visitor.

The failures above are flakily reproducable for me using rr record -h of the AOT runtime command of the test.

/cc @rmacnak-google Could you have a look?
@mkustermann mkustermann added area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. gardening labels Oct 31, 2022
@sstrickl sstrickl self-assigned this Nov 3, 2022
@sstrickl
Copy link
Contributor

sstrickl commented Nov 3, 2022

Didn't realize this bug existed when I made CL 267580 while gardening today, so going ahead and assigning to me since I think that CL should fix this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sstrickl sstrickl

Labels
area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. gardening
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sstrickl @mkustermann