Cross compile for m1 in ci #958

Ninja3047 · 2022-08-04T17:21:34Z

build_dependencies.sh works locally for when cross compiling to x64 from an M1
hopefully it just works in the other direction as well

update workflow to cross compile for m1 add arm64-osx-rel triplet temporarily remove aws runners

Host triplet is really only used for providing tools that need to be run on the host, so Debug doesn't really make sense. This also saves a lot of disk space and compile time

Has fixes for grpc and macOS cross compiling

ekilmer · 2022-08-24T17:41:53Z

@Ninja3047 Have you tested whether this PR works with the latest commit using my fixes for protobuf cross compilation?

I have an x86 mac but I don't have an ARM mac to test the built product.

If this works for you, then I want to submit the fixes upstream to vcpkg.

This reverts commit 49f015e.

Ninja3047 · 2022-08-25T19:32:40Z

cat ~/Projects/cxx-export/vcpkg_macos-11_llvm-14_xcode-13.0_arm64/installed/arm64-osx-rel/share/flatbuffers/FlatbuffersConfig.cmake

include("${CMAKE_CURRENT_LIST_DIR}/FlatbuffersTargets.cmake" OPTIONAL)
include("${CMAKE_CURRENT_LIST_DIR}/FlatcTargets.cmake" OPTIONAL)
include("${CMAKE_CURRENT_LIST_DIR}/FlatbuffersSharedTargets.cmake" OPTIONAL)

include("${CMAKE_CURRENT_LIST_DIR}/../../../x64-osx-rel/share/flatbuffers/FlatcTargets.cmake")

hrmm flatbuffers appears to have an issue

Ninja3047 · 2022-08-25T19:39:53Z

weird, looks like it was done intentionally
https://github.com/microsoft/vcpkg/blob/8a5bbf44092a10d75b159b7e474eb033b8c0df39/ports/flatbuffers/portfile.cmake#L44

Ninja3047 · 2022-08-25T20:02:59Z

  Target "Runtime" links to:

    gRPC::upb

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.

get this error when building a library that links grpc using the exported artifacts
works fine with the natively compiled install directory

ekilmer · 2022-08-29T13:23:30Z

Thanks for testing and investigating... I believe we're in uncharted territory, so there will be bugs like this.

I'll remove the line you mentioned here #958 (comment)

ekilmer · 2022-08-29T13:32:49Z

  Target "Runtime" links to:

    gRPC::upb

  but the target was not found.  Possible reasons include:

    * There is a typo in the target name.
    * A find_package call is missing for an IMPORTED target.
    * An ALIAS target is missing.
get this error when building a library that links grpc using the exported artifacts works fine with the natively compiled install directory

Hmmmm. I'll have to investigate this a bit more. gRPC is patched to use a separate upb package but I haven't dug into much yet. The patch is likely missing some additional fixes.

…ebug artifacts

Ninja3047 · 2022-09-01T16:41:10Z

tested on M1 with Rellic/Anvill and Remill

ekilmer

Looks good aside from using the vcpkg fork!

.github/workflows/vcpkg_ci_mac.yml

vcpkg_info.txt

…in-ci

CLAassistant · 2022-09-29T20:04:29Z

All committers have signed the CLA.

they may be needed for protoc, in theory they should run with rosetta2

* master: Remove AWS CI runners (#978)

ekilmer · 2022-10-12T17:51:51Z

The LLVM 15 error is coming due to LLVM using the wrong libzstd. I reproduced locally and it's not using zstd from vcpkg, so it produces an error

ld: warning: ignoring file /usr/local/lib/libzstd.1.5.2.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
Undefined symbols for architecture arm64:

ekilmer

LGTM! Waiting on CI

Ninja3047 linked an issue Aug 4, 2022 that may be closed by this pull request

Cross compile for M1 in CI #957

Closed

Ninja3047 force-pushed the will/cross-compile-for-m1-in-ci branch 6 times, most recently from 46f4dfa to a87a94c Compare August 4, 2022 18:07

Ninja3047 marked this pull request as draft August 4, 2022 20:10

Ninja3047 added the ci Continuous integration label Aug 4, 2022

Ninja3047 force-pushed the will/cross-compile-for-m1-in-ci branch 3 times, most recently from 3d3065a to 400f9a9 Compare August 5, 2022 17:53

Ninja3047 and others added 4 commits August 8, 2022 11:59

add cross compilation support to build_dependencies.sh script

3a7a729

update workflow to cross compile for m1 add arm64-osx-rel triplet temporarily remove aws runners

Update vcpkg to fix grpc cross compile

4a4243f

Make host triplet the release-only triplet

b64da00

Host triplet is really only used for providing tools that need to be run on the host, so Debug doesn't really make sense. This also saves a lot of disk space and compile time

specify system name for cross compilation

4bf4602

Ninja3047 force-pushed the will/cross-compile-for-m1-in-ci branch 2 times, most recently from 4960800 to 5192b45 Compare August 8, 2022 20:53

bump remill/rellic to fix cross compilation bug with z3

49f015e

Ninja3047 force-pushed the will/cross-compile-for-m1-in-ci branch from 5192b45 to 49f015e Compare August 8, 2022 21:06

Ninja3047 and others added 4 commits August 8, 2022 18:37

disable remill/rellic/anvil build on non x64

64c4e6f

remove explicit CMAKE_SYSTEM_NAME to avoid triggering cross compilation

9786f96

Use latest vcpkg commit

a32a0be

Has fixes for grpc and macOS cross compiling

Fix protobuf cross compile (ekilmer branch)

ffc7498

Ninja3047 added 2 commits August 24, 2022 13:55

Revert "bump remill/rellic to fix cross compilation bug with z3"

da33c57

This reverts commit 49f015e.

debug artifacts

6bc9013

Fix flatc targets. debug artifacts

4f31b74

remove host compiled artifacts if the target and host do not match, d…

13391c2

…ebug artifacts

Ninja3047 force-pushed the will/cross-compile-for-m1-in-ci branch from 1e78ff6 to 13391c2 Compare September 1, 2022 15:23

add back aws to ci

e87dfbe

Ninja3047 marked this pull request as ready for review September 1, 2022 18:34

Ninja3047 requested a review from ekilmer September 1, 2022 18:35

ekilmer requested changes Sep 1, 2022

View reviewed changes

.github/workflows/vcpkg_ci_mac.yml Show resolved Hide resolved

vcpkg_info.txt Outdated Show resolved Hide resolved

Merge remote-tracking branch 'origin' into will/cross-compile-for-m1-…

3d8a9f2

…in-ci

Ninja3047 requested a review from ekilmer September 30, 2022 14:15

Ninja3047 and others added 6 commits September 30, 2022 10:56

debug artifacts

a9c0350

revert remove host artifacts, debug artifacts

c43e281

they may be needed for protoc, in theory they should run with rosetta2

Merge branch 'master' into will/cross-compile-for-m1-in-ci

2f8601e

Fix conditional for remill test on mac

4070ac1

Fix anvill test conditional

3c1babd

Merge branch 'master' into will/cross-compile-for-m1-in-ci

0d1fc65

* master: Remove AWS CI runners (#978)

Fix LLVM-15 for cross-compilation

393597a

ekilmer approved these changes Oct 12, 2022

View reviewed changes

ekilmer merged commit 3c1932d into master Oct 13, 2022

ekilmer deleted the will/cross-compile-for-m1-in-ci branch October 13, 2022 00:35

Cross compile for m1 in ci #958

Cross compile for m1 in ci #958

Uh oh!

Conversation

Ninja3047 commented Aug 4, 2022

Uh oh!

ekilmer commented Aug 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Ninja3047 commented Aug 25, 2022

Uh oh!

Ninja3047 commented Aug 25, 2022

Uh oh!

Ninja3047 commented Aug 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ekilmer commented Aug 29, 2022

Uh oh!

ekilmer commented Aug 29, 2022

Uh oh!

Ninja3047 commented Sep 1, 2022

Uh oh!

ekilmer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

CLAassistant commented Sep 29, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ekilmer commented Oct 12, 2022

Uh oh!

ekilmer left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ekilmer commented Aug 24, 2022 •

edited

Loading

Ninja3047 commented Aug 25, 2022 •

edited

Loading

CLAassistant commented Sep 29, 2022 •

edited

Loading