Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[vm/ffi] Add support for MacOS arm64 #46305

Closed
sstrickl opened this issue Jun 9, 2021 · 3 comments
Closed

[vm/ffi] Add support for MacOS arm64 #46305

sstrickl opened this issue Jun 9, 2021 · 3 comments
Labels
area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. library-ffi

Comments

@sstrickl
Copy link
Contributor

sstrickl commented Jun 9, 2021

There are new test failures on [infra] Add xcodebuild/...XARM64 directories to the vm-kernel fileset.

The tests

ffi_2/function_callbacks_structs_by_value_generated_test/0 RuntimeError (expected Pass)
ffi_2/function_callbacks_structs_by_value_generated_test/1 RuntimeError (expected Pass)
ffi_2/function_callbacks_structs_by_value_generated_test/2 RuntimeError (expected Pass)
ffi_2/function_callbacks_structs_by_value_generated_test/3 RuntimeError (expected Pass)
ffi_2/function_callbacks_test/0 RuntimeError (expected Pass)
ffi_2/function_callbacks_test/1 RuntimeError (expected Pass)
ffi_2/function_callbacks_test/2 RuntimeError (expected Pass)
ffi_2/function_callbacks_test/3 RuntimeError (expected Pass)
ffi_2/function_callbacks_test/4 RuntimeError (expected Pass)
ffi_2/function_callbacks_test/5 RuntimeError (expected Pass)
ffi_2/function_callbacks_test/6 RuntimeError (expected Pass)
ffi_2/function_callbacks_test/7 RuntimeError (expected Pass)
ffi_2/function_callbacks_test/8 RuntimeError (expected Pass)
ffi_2/function_callbacks_very_many_test/0 RuntimeError (expected Pass)
ffi_2/function_callbacks_very_many_test/1 RuntimeError (expected Pass)
ffi_2/function_callbacks_very_many_test/2 RuntimeError (expected Pass)
ffi_2/function_callbacks_very_many_test/3 RuntimeError (expected Pass)
ffi_2/function_callbacks_very_many_test/4 RuntimeError (expected Pass)
ffi_2/function_callbacks_very_many_test/5 RuntimeError (expected Pass)
ffi_2/function_callbacks_very_many_test/6 RuntimeError (expected Pass)
ffi_2/function_callbacks_very_many_test/7 RuntimeError (expected Pass)
ffi_2/function_structs_by_value_generated_test/0 RuntimeError (expected Pass)
ffi_2/function_structs_by_value_generated_test/1 RuntimeError (expected Pass)
ffi_2/function_structs_by_value_generated_test/2 RuntimeError (expected Pass)
ffi_2/function_structs_by_value_generated_test/3 RuntimeError (expected Pass)
ffi_2/function_test/0 RuntimeError (expected Pass)
ffi_2/function_test/1 RuntimeError (expected Pass)
ffi_2/function_test/2 RuntimeError (expected Pass)
    and 22 more tests

are failing on configurations

dartk-mac-release-arm64

The vm_specific_function_callbacks tests are failing due to symbol lookup failures. Not sure about the others, but guessing it's just needing to adjust stuff for the possibility of ARM64.

/cc @dcharkes
@sstrickl sstrickl added area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. gardening labels Jun 9, 2021
@dcharkes dcharkes changed the title Failures on [infra] Add xcodebuild/...XARM64 directories to the vm-kernel fileset [vm/ffi] Add support for MacOS arm64 Jun 9, 2021
@dcharkes
Copy link
Contributor

dcharkes commented Jun 9, 2021 •
edited
Loading

Link to approved failures on bot: https://dart-ci.firebaseapp.com/#configurations=dartk-mac-release-arm64

Edit: current results: https://dart-ci.firebaseapp.com/current_results/#/filter=dartk-mac-release-arm64,ffi

Quick triage:

--- Command "vm" (took 794ms):
DART_CONFIGURATION=ReleaseXARM64 xcodebuild/ReleaseXARM64/dart --enable-isolate-groups --experimental-enable-isolate-groups-jit --ignore-unrecognized-flags --packages=/opt/s/w/ir/.packages /opt/s/w/ir/tests/ffi_2/vmspecific_function_callbacks_test.dart

exit code:
-6

stderr:
dyld: lazy symbol binding failed: Symbol not found: _ClobberAndCall
  Referenced from: /opt/s/w/ir/xcodebuild/ReleaseXARM64/./libffi_test_functions.dylib
  Expected in: flat namespace

dyld: Symbol not found: _ClobberAndCall
  Referenced from: /opt/s/w/ir/xcodebuild/ReleaseXARM64/./libffi_test_functions.dylib
  Expected in: flat namespace

--- Re-run this test:
python3 tools/test.py -n dartk-mac-release-arm64 ffi_2/vmspecific_function_callbacks_test/0

A linking error.

https://dart-ci.appspot.com/log/vm-kernel-mac-release-arm64/dartk-mac-release-arm64/2/ffi_2/vmspecific_function_callbacks_test/0

==== Running PassStruct12BytesHomogeneousFloatx6
Calling TestPassStruct12BytesHomogeneousFloatx6(((-1, 2, -3), (4, -5, 6), (-7, 8, -9), (10, -11, 12), (-13, 14, -15), (16, -17, 18)))
passStruct12BytesHomogeneousFloatx6((-1.0, 2.0, -3.0), (4.0, -5.0, 6.0), (-7.0, 8.0, -9.0), (-11.0, 12.0, -13.0), (-15.0, 16.0, -17.0), (0.0, 0.0, 0.0))
result = -33.0
result = -33

stderr:
Check failed: ((9.0 * 0.99) <= (result) && (9.0 * 1.01) >= (result)) || ((9.0 * 0.99) >= (result) && (9.0 * 1.01) <= (result))
Unhandled exception:
Expect.fail('Test PassStruct12BytesHomogeneousFloatx6 failed.')
#0      Expect._fail (package:expect/expect.dart:731:5)
#1      Expect.fail (package:expect/expect.dart:300:5)
#2      CallbackTest.run (file:///opt/s/w/ir/tests/ffi_2/callback_tests_utils.dart:39:14)
#3      main.<anonymous closure> (file:///opt/s/w/ir/tests/ffi_2/function_callbacks_structs_by_value_generated_test.dart:28:7)
#4      List.forEach (dart:core-patch/growable_array.dart:403:8)
#5      main (file:///opt/s/w/ir/tests/ffi_2/function_callbacks_structs_by_value_generated_test.dart:26:13)
#6      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:283:19)
#7      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)

--- Re-run this test:
python3 tools/test.py -n dartk-mac-release-arm64 ffi_2/function_callbacks_structs_by_value_generated_test/0

Mangled arguments in callbacks

10.0, 14.0, and 18.0 are passed "in between" the structs leaving the last struct 0.0, 0.0, 0.0.

https://dart-ci.appspot.com/log/vm-kernel-mac-release-arm64/dartk-mac-release-arm64/2/ffi_2/function_callbacks_test/5

PassStruct12BytesHomogeneousFloatx6((-1, 2, -3), (4, -5, 6), (-7, 8, -9), (0, 10, -11), (12, 0, -13), (14, -15, 0))
result = -8
result = -8.0

stderr:
Unhandled exception:
Expect.approxEquals(expected:<9.0>, actual:<-8.0>, tolerance:<0.0009>) fails
#0      Expect._fail (package:expect/expect.dart:731:5)
#1      Expect.approxEquals (package:expect/expect.dart:317:5)
#2      testPassStruct12BytesHomogeneousFloatx6 (file:///opt/s/w/ir/tests/ffi_2/function_structs_by_value_generated_test.dart:2636:10)
#3      main (file:///opt/s/w/ir/tests/ffi_2/function_structs_by_value_generated_test.dart:36:5)
#4      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:283:19)
#5      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)

--- Re-run this test:
python3 tools/test.py -n dartk-mac-release-arm64 ffi_2/function_structs_by_value_generated_test/0

Mangled arguments the other way around in calls.

Expected 11 where 10 is passed.

https://dart-ci.appspot.com/log/vm-kernel-mac-release-arm64/dartk-mac-release-arm64/2/ffi_2/function_structs_by_value_generated_test/0

dart-bot pushed a commit that referenced this issue Jun 9, 2021
@dcharkes
[vm/ffi] Fix MacOS arm64 homogenous float alignment
a2ac423 
Fix the alignment of structs on MacOS arm64 so that it is the same as on
iOS arm64.

This fixes some of the ffi test failures, but not all. I'll follow up
with further CLs. (Having this landed will allow us to see further
failures on the bots.)

Bug: #46305

TEST=runtime/vm/compiler/ffi/unit_tests/struct12bytesFloatx6/arm64_macos.expect
TEST=tests/ffi(_2)/(.*)function(.*)_test.dart

Change-Id: I66d214ecf672734c685ff97316b1666a2859b106
Cq-Include-Trybots: luci.dart.try:vm-kernel-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/202940
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
@dcharkes
Copy link
Contributor

dcharkes commented Jun 10, 2021 •
edited
Loading

Having the previous fix landed enables us to see further failure on the M1 devices.

Log of current results: https://dart-ci.firebaseapp.com/current_results/#/filter=dartk-mac-release-arm64,ffi&showAll

ABI

Further triage:

PassStruct9BytesPackedMixedx10DoubleInt32((1, 2), (3, 4), (5, 6), (7, 8), (9, 10), (11, 12), (13, 14), (15, 16), (17, 18), (19, 20), -21, 392336737)
result = 3.92337e+08
result = 392336926.0

stderr:
Unhandled exception:
Expect.approxEquals(expected:<211.0>, actual:<392336926.0>, tolerance:<0.0211>) fails
#0      Expect._fail (package:expect/expect.dart:731:5)
#1      Expect.approxEquals (package:expect/expect.dart:317:5)
#2      testPassStruct9BytesPackedMixedx10DoubleInt32 (file:///opt/s/w/ir/tests/ffi_2/function_structs_by_value_generated_test.dart:5980:10)
#3      main (file:///opt/s/w/ir/tests/ffi_2/function_structs_by_value_generated_test.dart:77:5)
#4      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:283:19)
#5      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)

--- Re-run this test:
python3 tools/test.py -n dartk-mac-release-arm64 ffi_2/function_structs_by_value_generated_test/0

Mangled argument in FFI call again.

https://dart-ci.appspot.com/log/any/dartk-mac-release-arm64/32/ffi_2/function_structs_by_value_generated_test/0

==== Running PassStruct9BytesPackedMixedx10DoubleInt32
Calling TestPassStruct9BytesPackedMixedx10DoubleInt32(((1, 2), (3, 4), (5, 6), (7, 8), (9, 10), (11, 12), (13, 14), (15, 16), (17, 18), (19, 20), -21, 22))
passStruct9BytesPackedMixedx10DoubleInt32((1, 2.0), (3, 4.0), (5, 6.0), (7, 8.0), (9, 10.0), (11, 12.0), (13, 14.0), (15, 16.0), (17, 18.0), (19, 20.0), -21.0, 0)
result = 189.0
result = 189

stderr:
Check failed: ((211.0 * 0.99) <= (result) && (211.0 * 1.01) >= (result)) || ((211.0 * 0.99) >= (result) && (211.0 * 1.01) <= (result))
Unhandled exception:
Expect.fail('Test PassStruct9BytesPackedMixedx10DoubleInt32 failed.')
#0      Expect._fail (package:expect/expect.dart:731:5)
#1      Expect.fail (package:expect/expect.dart:300:5)
#2      CallbackTest.run (file:///opt/s/w/ir/tests/ffi_2/callback_tests_utils.dart:39:14)
#3      main.<anonymous closure> (file:///opt/s/w/ir/tests/ffi_2/function_callbacks_structs_by_value_generated_test.dart:28:7)
#4      List.forEach (dart:core-patch/growable_array.dart:403:8)
#5      main (file:///opt/s/w/ir/tests/ffi_2/function_callbacks_structs_by_value_generated_test.dart:26:13)
#6      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:283:19)
#7      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)

--- Re-run this test:
python3 tools/test.py -n dartk-mac-release-arm64 ffi_2/function_callbacks_structs_by_value_generated_test/0

Same mangled arguments in callbacks.

https://dart-ci.appspot.com/log/any/dartk-mac-release-arm64/32/ffi_2/function_structs_by_value_generated_test/0

double PassStruct9BytesPackedMixedx10DoubleInt32(
    Struct9BytesPackedMixed a0,
    Struct9BytesPackedMixed a1,
    Struct9BytesPackedMixed a2,
    Struct9BytesPackedMixed a3,
    Struct9BytesPackedMixed a4,
    Struct9BytesPackedMixed a5,
    Struct9BytesPackedMixed a6,
    Struct9BytesPackedMixed a7,
    Struct9BytesPackedMixed a8,
    Struct9BytesPackedMixed a9,
    double a10,
    int32_t a11) {

a11 is not in the place where it is expected to be.

Linking

_ClobberAndCall is also not linked in.

https://dart-ci.appspot.com/log/any/dartk-mac-release-arm64/32/ffi_2/vmspecific_function_callbacks_test/0

dart-bot pushed a commit that referenced this issue Jun 11, 2021
@dcharkes
[vm/ffi] Fix _ClobberAndCall linking on MacOS arm64
a9f08a7 
Bug: #46305

TEST=tests/ffi_2/vmspecific_function_callbacks_test.dart

Change-Id: I82af08c5123b9bf637b9dc6e1983dde3fa155291
Cq-Include-Trybots: luci.dart.try:vm-kernel-mac-release-arm64-try,vm-ffi-android-debug-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/202967
Reviewed-by: Tess Strickland <sstrickl@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
@dcharkes
Copy link
Contributor

After https://dart-review.googlesource.com/c/sdk/+/203305, we still have 2 tests failing which use --use-slow-path. I'm assuming they also fail on arm64 iOS at the moment. I've filed #46349 to track that failure.

dart-bot pushed a commit that referenced this issue Jun 14, 2021
@dcharkes
[vm/ffi] Fix MacOS arm64 register extension
00d39e0 
Makes (u)int8 and (u)int16 extended to 32 bits in registers on arm64
MacOS. This makes it consistent with iOS. (Now the calling convention
on arm64 MacOS is fully identical to arm64 iOS.)

TEST=runtime/vm/compiler/ffi/native_calling_convention_test.cc
TEST=tests/ffi_2/function_test.dart

Bug: #46305

This CL makes all remaining failing tests pass listed in the bug,
except for tests using `--use-slow-path`.

Change-Id: If0d93bfa774e561f39468d2b63fc9381b7646b18
Cq-Include-Trybots: luci.dart.try:vm-kernel-mac-release-arm64-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/203305
Reviewed-by: Clement Skau <cskau@google.com>
Commit-Queue: Daco Harkes <dacoharkes@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area-vm Use area-vm for VM related issues, including code coverage, and the AOT and JIT backends. library-ffi
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sstrickl @dcharkes