Segmentation fault while deconstructing isolate on dart console app exit

[timsneath]

Running the dart_console demo example crashes on the latest Dart build checked into the flutter master channel, running on an M1 MacBook Air:

===== CRASH =====
si_signo=Segmentation fault: 11(11), si_code=2, si_addr=0x1e002bc00640026
version=2.19.0-3.0.dev (dev) (Fri Jul 15 07:03:13 2022 -0700) on "macos_arm64"
pid=21167, thread=259, isolate_group=(nil)(0x0), isolate=(nil)(0x0)
isolate_instructions=0, vm_instructions=100e077e0
  pc 0x0000000100f4a568 fp 0x000000016f04a480 dart::Isolate::~Isolate()+0x88
  pc 0x0000000100f4aed8 fp 0x000000016f04a500 dart::Isolate::LowLevelCleanup(dart::Isolate*)+0xe0
  pc 0x0000000100f4c6c0 fp 0x000000016f04af50 dart::Isolate::Shutdown()+0x184
  pc 0x00000001014a5dbc fp 0x000000016f04b4a0 Dart_ShutdownIsolate+0xac
  pc 0x0000000100de2060 fp 0x000000016f04b550 dart::bin::RunMainIsolate(char const*, char const*, dart::bin::CommandLineOptions*)+0x2cc
  pc 0x0000000100de2cf4 fp 0x000000016f04b6c0 dart::bin::main(int, char**)+0x60c
  pc 0x0000000100de3b08 fp 0x000000016f04b6d0 main+0xc
  pc 0x000000010355908c fp 0x000000016f04b820 Unknown symbol
-- End of DumpStackTrace
[1]    21167 abort      dart example/demo.dart

dart --version reports:

Dart SDK version: 2.19.0-3.0.dev (dev) (Fri Jul 15 07:03:13 2022 -0700) on "macos_arm64"

I tested this on various other versions of Dart:

• [ERROR] Dart SDK version: 2.19.0-edge.ea7f3d4d967be60ff78d3abbc12bcd67c7a28af9 (be) (Sat Jul 16 06:59:02 2022 +0000) on "macos_arm64"
• [ERROR] Dart SDK version: 2.18.0-165.1.beta (beta) (Mon Jun 13 15:34:45 2022 +0200) on "macos_arm64"
• [NO ERROR] Dart SDK version: 2.17.6 (stable) (Tue Jul 12 12:54:37 2022 +0200) on "macos_arm64"
• [NO ERROR] Dart SDK version: 2.19.0-edge.ea7f3d4d967be60ff78d3abbc12bcd67c7a28af9 (be) (Sat Jul 16 06:59:02 2022 +0000) on "macos_x64" (running on Rosetta)

So it looks like something regressed on ARM64 builds between stable and beta.

Repro by cloning https://github.com/timsneath/dart_console/tree/f88ab598bf896878b347013400049d0702eecd0c and running dart example/demo.dart from the top-level directory.

[aam]

It looks like culprit is the way how variadic ioctl function is defined in dart https://github.com/timsneath/dart_console/blob/f88ab598bf896878b347013400049d0702eecd0c/lib/src/ffi/unix/ioctl.dart#L35

// int ioctl(int, unsigned long, ...);
typedef IOCtlNative = Int32 Function(Int32, Int64, Pointer<Void>);
typedef IOCtlDart = int Function(int, int, Pointer<Void>);

Such ioctl invocation ends up corrupting vm isolate structure causing a crashes.

thanks @rmacnak-google for lending a hand with this!

cc @dcharkes

[aam]

see dart-lang/native#238 that tracks ffi support for variadic functions

[dcharkes]

see dart-lang/native#238 that tracks ffi support for variadic functions

This tracks package:ffigens support for variadic functions.

Tracking bug for dart:ffi variadic function support:

• dart:ffi: Support variadic C functions #38578

[Sunbreak]

• [ERROR] Dart SDK version: 2.19.0-32.0.dev (dev) (Sun Jul 24 15:18:55 2022 -0700) on "macos_arm64"
• [NO ERROR] Dart SDK version: 2.18.0-271.4.beta (beta) (Tue Jul 26 10:14:06 2022 +0200) on "macos_arm64"

// int ioctl(int, unsigned long, ...);
import 'dart:ffi';
import 'dart:io';

import 'package:ffi/ffi.dart';

typedef IOCtlNative = Int32 Function(Int32, Int64, Pointer<Void>);
typedef IOCtlDart = int Function(int, int, Pointer<Void>);

final TIOCGWINSZ = Platform.isMacOS ? 0x40087468 : 0x5413;
const STDIN_FILENO = 0;
const STDOUT_FILENO = 1;
const STDERR_FILENO = 2;

// struct winsize {
//      unsigned short  ws_row;         /* rows, in characters */
//      unsigned short  ws_col;         /* columns, in characters */
//      unsigned short  ws_xpixel;      /* horizontal size, pixels */
//      unsigned short  ws_ypixel;      /* vertical size, pixels */
// };
class WinSize extends Struct {
  @Int16()
  external int ws_row;

  @Int16()
  external int ws_col;

  @Int16()
  external int ws_xpixel;

  @Int16()
  external int ws_ypixel;
}

void main() {
  final ioctl = DynamicLibrary.process().lookupFunction<IOCtlNative, IOCtlDart>('ioctl');

  final winSizePointer = calloc<WinSize>();
  final result = ioctl(STDOUT_FILENO, TIOCGWINSZ, winSizePointer.cast());
  print('result is $result');

  final winSize = winSizePointer.ref;
  print('Per ioctl, this console window has ${winSize.ws_col} cols and '
      '${winSize.ws_row} rows.');

  calloc.free(winSizePointer);
}