CheckStackOverflow triggers crash in FFI

[sjindel-google]

Migrated from #37606:

The following code leads to a segmentation fault when executed with the option --stacktrace_every=100. This happens on multiple builds e.g ReleaseX64.

import 'dart:ffi' as ffi;

typedef fooFfi1Type = ffi.Int32 Function();
int fooFfi1() {
  int a = 0;
  for(int i=0; i<1000; ++i) {                                                                                                            
    a += i;
  }
  return a;
}
int Function() foo1 = ffi.Pointer.fromFunction<fooFfi1Type>(fooFfi1, 0).cast<ffi.NativeFunction<fooFfi1Type>>().asFunction();
main() {
  foo1();
} 

Fault:

../../sdk/out/ReleaseX64/dart --stacktrace_every=100 ffitest.dart

===== CRASH =====
si_signo=Segmentation fault(11), si_code=1, si_addr=0x7ff4c01c0000
version=2.5.0-edge.92562dc5a50a822656d76ce1b9b49da560b4327c (Mon Aug 12 17:01:06 2019 +0000) on "linux_x64"
thread=246184, isolate=main(0x56082d5b0100)
  pc 0x000056082bc731b3 fp 0x00007ff4beffe130 dart::PcDescriptors::DecodeInteger(long*) const
  pc 0x000056082bc7db18 fp 0x00007ff4beffe1a0 dart::Code::GetPcForDeoptId(long, dart::RawPcDescriptors::Kind) const
  pc 0x000056082bd4152b fp 0x00007ff4beffe1e0 dart::InlinedFunctionsIterator::InlinedFunctionsIterator(dart::Code const&, unsigned long)
  pc 0x000056082bbb51e1 fp 0x00007ff4beffe2a0 dart::Debugger::AppendCodeFrames(dart::Thread*, dart::Isolate*, dart::Zone*, dart::DebuggerStackTrace*, dart::StackFrame*, dart::Code*, dart::Code*, dart::Array*)
  pc 0x000056082bba9cb5 fp 0x00007ff4beffe400 dart::Debugger::CollectStackTrace()
  pc 0x000056082bd19321 fp 0x00007ff4beffe610 dart::DRT_StackOverflow(dart::NativeArguments)
  pc 0x00007ff4bf001108 fp 0x00007ff4beffe650 Unknown symbol
  pc 0x00007ff4bbf6cf82 fp 0x00007ff4beffe688 Unknown symbol
  pc 0x00007ff4bbf6c9a4 fp 0x00007ff4beffe6e0 Unknown symbol
  pc 0x00007ff4bf0015e0 fp 0x00007ff4beffe740 Unknown symbol
  pc 0x0000000000000000 fp 0x00007ff4beffe750 Unknown symbol
  pc 0x00007ff4bf0041ee fp 0x00007ff4beffe770 Unknown symbol
  pc 0x00007ff4bbf6cd4d fp 0x00007ff4beffe798 Unknown symbol
  pc 0x00007ff4bbf6c461 fp 0x00007ff4beffe7d0 Unknown symbol
  pc 0x00007ff4bbf6c350 fp 0x00007ff4beffe810 Unknown symbol
  pc 0x00007ff4bbf6c17c fp 0x00007ff4beffe848 Unknown symbol
  pc 0x00007ff4bbf6b3a4 fp 0x00007ff4beffe890 Unknown symbol
  pc 0x00007ff4bbf4a0ff fp 0x00007ff4beffe8d0 Unknown symbol
  pc 0x00007ff4bbf6b073 fp 0x00007ff4beffe908 Unknown symbol
  pc 0x00007ff4bf00166c fp 0x00007ff4beffe978 Unknown symbol
  pc 0x000056082bba4436 fp 0x00007ff4beffea20 dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long)
  pc 0x000056082bba78e6 fp 0x00007ff4beffea80 dart::DartLibraryCalls::HandleMessage(dart::Object const&, dart::Instance const&)
  pc 0x000056082bbe57b2 fp 0x00007ff4beffec80 dart::IsolateMessageHandler::HandleMessage(std::__2::unique_ptr<dart::Message, std::__2::default_delete<dart::Message> >)
  pc 0x000056082bc190fd fp 0x00007ff4beffecf0 dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool)
  pc 0x000056082bc19946 fp 0x00007ff4beffed50 dart::MessageHandler::TaskCallback()
  pc 0x000056082bd51d97 fp 0x00007ff4beffed80 dart::ThreadPool::Worker::Loop()
  pc 0x000056082bd51c35 fp 0x00007ff4beffedc0 dart::ThreadPool::Worker::Main(unsigned long)
  pc 0x000056082bcb9569 fp 0x00007ff4beffee70 ../../sdk/out/ReleaseX64/dart+0x192b569
-- End of DumpStackTrace
[1]    246182 abort      ../../sdk/out/ReleaseX64/dart --stacktrace_every=100 ffitest.dart

/cc @feli-citas

[sjindel-google]

This looks like an issue with the ForceOptimize() technology. I'll investigate.

[sjindel-google]

There turns out to be two issues here:

1. The profiler attempts to read the deopt info for optimized frames in order to recover inlined frames. There is no deopt info for force-optimized functions, so it crashes.
2. The debugger attempts to deoptimize any frame on the stack in order to debug the frame. Force-optimized functions cannot deoptimize, so the debugger crashes.

I've addressed both of these in https://dart-review.googlesource.com/c/sdk/+/113999