Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[flutter_tools] Add ARM64 Linux host and cross-building option support #61221

Conversation

HidenoriMatsubayashi
Copy link
Member

@HidenoriMatsubayashi HidenoriMatsubayashi commented Jul 10, 2020

1. Description

I want to support following functions.

  • The Flutter SDK works on Arm64 Linux Host (e.g. flutter run -d linux, flutter doctor)
  • Flutter apps can be built on Arm64 Linux Host.
  • Flutter apps can be cross-build on x64 Linux Host for Arm64 target devices.

Design documents

https://docs.google.com/document/d/19tzWySgtgtTA99XQsjx5Pg0SFJeZKXyUlYavR0EXv8c/edit?usp=sharing

Use Case

Running the flutter app on Arm64 Linux Host.

$ flutter doctor
$ flutter run -d linux

Screenshot from 2020-08-05 12-52-42

Cross-building a Flutter app for Arm64 on x64 Linux Host.

If you want to cross-build for Arm64, you need to prepare a target sysroot. You can specify it as a command line argument.

$ flutter build linux --target-platform=linux-arm64 --target-sysroot=/target-sysroot-path

2. Related PRs

3. Related Issues

4. Tests

I have tested that it can be built and run for x84_64 Host and built for aarch64 to work with Jeston Nano.

5. Checklist

Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I signed the CLA.
  • I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I updated/added relevant documentation (doc comments with ///).
  • All existing and new tests are passing.
  • The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
  • I am willing to follow-up on review comments in a timely manner.

Breaking Change

Did any tests fail when you ran them? Please read Handling breaking changes.

@fluttergithubbot fluttergithubbot added the tool Affects the "flutter" command-line tool. See also t: labels. label Jul 10, 2020
@fluttergithubbot
Copy link
Contributor

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption to this rule, contact Hixie on the #hackers channel in Chat.

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

@christopherfujino christopherfujino added a: desktop Running on desktop platform-linux Building on or for Linux specifically labels Jul 10, 2020
@@ -299,6 +299,7 @@ class AOTSnapshotter {
TargetPlatform.ios,
TargetPlatform.darwin_x64,
TargetPlatform.linux_x64,
TargetPlatform.linux_arm64,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're trying to move away from the giant switch of platforms. The problem this causes is that it makes a lot of invalid states easily representable.

Instead this should be factored out into a LinuxArch enum. This could be done before any of the work to support arm

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonahwilliams
Okay. I will create a LinuxArch enum like Android/iOS and modify this PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the cache APIs we have today, it doesn't seem easy to do this. I wonder if you've had any progress? Perhaps we should reconsider for now

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonahwilliams
I'm reading around the source code of cache.dart. I'm also considering a fix to support ARM64 Host. Please give me some time.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonahwilliams
By the way, I think TargetPlatform should be separated into OS and Arch. Are you already considering some kind of refactoring? It may take some time, but is it okay for me to consider it? However, there are many changes. The timing of merging is also difficult.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I missed this - that is probably a good change but its a bit out of scope

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. Thank you.

@christopherfujino
Copy link
Member

@HidenoriMatsubayashi did you actually build the binaries and get this working, or is this a proof of concept? When I said I could help you with infra, I misunderstood and thought you had a working linux arm host setup. I can point you in the right direction, but I don't really have bandwidth to work on this at the moment.

@HidenoriMatsubayashi
Copy link
Member Author

HidenoriMatsubayashi commented Jul 14, 2020

@christopherfujino

did you actually build the binaries and get this working, or is this a proof of concept?

Yes, I did. I use my customized sysrootfs for aarch64, but here is a binary for sample flutter app built with Flutter SDK with this patch. Works with ARM64 platforms such as JetsonNao. Of course, I also use a locally built version of Flutter Engine.
https://github.com/HidenoriMatsubayashi/flutter-examples/blob/master/flutter_aarch64_sample.zip

Screenshot from 2020-06-11 17-32-47

I misunderstood and thought you had a working linux arm host setup.

I see. No problem at all.
By the way, I've also tried it partially. I did dart-sdk cross build for aarch64.

@christopherfujino
Copy link
Member

@HidenoriMatsubayashi That's awesome to hear!

Did you cross-build Dart directly from the https://github.com/dart-lang/sdk repo, or via Flutter's engine environment? I was able to do the former, but got stuck on the latter before I had to move on to other things.

@HidenoriMatsubayashi
Copy link
Member Author

HidenoriMatsubayashi commented Jul 15, 2020

@christopherfujino

Did you cross-build Dart directly from the https://github.com/dart-lang/sdk repo, or via Flutter's engine environment?

A while ago, I tried to cross-build Dart directly from the https://github.com/dart-lang/sdk repo for running Flutter SDK (flutter commands) on ARM64 Host. I basically tried the steps here. I think the build result worked on Raspberry Pi 4.
(By the way, I use Flutter Engine environment for building Flutter Engine (so files etc.).)

I was able to do the former, but got stuck on the latter before I had to move on to other things.

What exactly was the problem? If you want to support running flutter sdk on ARM64 Host, I may be able to help you..

@nhjk
Copy link

nhjk commented Jul 30, 2020

Any updates on this?

@stuartmorgan
Copy link
Contributor

Addressing some of the questions/comments in the initial description:

Should the result of the x86_64 and aarch64 builds be in separate directories?

Yes, definitely (unless there's a fat/universal build option for Linux that I'm not aware of). We should not stomp builds with different target outputs.

Should we include aarch64 rootfs in the Flutter SDK?

This seems out of scope for Flutter; we don't ship and use a sysroot for the current builds.

We will need infra updates so that you can download artifacts (Flutter Eingine build results, etc.) to the following locations.

This would be the first place to start, IMO. We also need confirmation/commitment from Dart folks that cross-compile tooling (e.g., for gen_snapshot) is a supported configuration on Linux.

@googlebot
Copy link

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

@HidenoriMatsubayashi
Copy link
Member Author

@nikiask
I'm a bit busy. But I'm still working.

@stuartmorgan
Thank you for your reply.

@christopherfujino
FYI. Here is the procedure for cross-build and running dart-sdk I tried. Basically cross-build procedure is as per the official wiki.

  1. Build dart-sdk on Host PC (Ubuntu x64)
$ mkdir dart-sdk
$ cd dart-sdk
$ git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
$ export PATH="$PATH:$PWD/depot_tools"
$ fetch dart
$ cd dart
$ ./tools/build.py --no-goma -m release -a arm64
$ cd out/ReleaseXARM64/
$ tar zcf /tmp/dart-sdk.tar.gz dart-sdk
  1. run Flutte SDK on Jetson Nano (Ubuntu arm64)
$ git clone https://github.com/flutter/flutter.git
$ export PATH=$PATH:./flutter/bin
$ tar xf dart-sdk.tar.gz

Replace dart-sdk after the error.

$ flutter doctor
  build error!!

$ rm -rf flutter/bin/cache/dart-sdk
$ cp -rf dart-sdk flutter/bin/cache
$ flutter doctor
Building flutter tool...

  ╔════════════════════════════════════════════════════════════════════════════╗
  ║                 Welcome to Flutter! - https://flutter.dev                  ║
  ║                                                                            ║
  ║ The Flutter tool uses Google Analytics to anonymously report feature usage ║
  ║ statistics and basic crash reports. This data is used to help improve      ║
  ║ Flutter tools over time.                                                   ║
  ║                                                                            ║
  ║ Flutter tool analytics are not sent on the very first run. To disable      ║
  ║ reporting, type 'flutter config --no-analytics'. To display the current    ║
  ║ setting, type 'flutter config'. If you opt out of analytics, an opt-out    ║
  ║ event will be sent, and then no further information will be sent by the    ║
  ║ Flutter tool.                                                              ║
  ║                                                                            ║
  ║ By downloading the Flutter SDK, you agree to the Google Terms of Service.  ║
  ║ Note: The Google Privacy Policy describes how data is handled in this      ║
  ║ service.                                                                   ║
  ║                                                                            ║
  ║ Moreover, Flutter includes the Dart SDK, which may send usage metrics and  ║
  ║ crash reports to Google.                                                   ║
  ║                                                                            ║
  ║ Read about data we send with crash reports:                                ║
  ║ https://flutter.dev/docs/reference/crash-reporting                         ║
  ║                                                                            ║
  ║ See Google's privacy policy:                                               ║
  ║ https://policies.google.com/privacy                                        ║
  ╚════════════════════════════════════════════════════════════════════════════╝


Downloading Material fonts...                                       1.3s
Downloading Gradle Wrapper...                                       0.2s
Downloading package sky_engine...                                   0.5s
Downloading flutter_patched_sdk tools...                            1.5s
Downloading flutter_patched_sdk_product tools...                    1.3s
Downloading linux-x64 tools...                                      2.4s
Downloading linux-x64/font-subset tools...                          0.6s
Downloading android-arm-profile/linux-x64 tools...                  0.6s
Downloading android-arm-release/linux-x64 tools...                  0.6s
Downloading android-arm64-profile/linux-x64 tools...                0.7s
Downloading android-arm64-release/linux-x64 tools...                0.6s
Downloading android-x64-profile/linux-x64 tools...                  0.6s
Downloading android-x64-release/linux-x64 tools...                  0.6s
Doctor summary (to see all details, run flutter doctor -v):
[!] Flutter (Channel master, 1.21.0-6.0.pre.129, on Linux, locale en_US.UTF-8)
    ✗ Downloaded executables cannot execute on host.
      See https://github.com/flutter/flutter/issues/6207 for more information
      On Debian/Ubuntu/Mint: sudo apt-get install lib32stdc++6
      On Fedora: dnf install libstdc++.i686
      On Arch: pacman -S lib32-gcc-libs

[✗] Android toolchain - develop for Android devices
    ✗ Unable to locate Android SDK.
      Install Android Studio from: https://developer.android.com/studio/index.html
      On first launch it will assist you in installing the Android SDK components.
      (or visit https://flutter.dev/docs/get-started/install/linux#android-setup for detailed instructions).
      If the Android SDK has been installed to a custom location, set ANDROID_SDK_ROOT to that location.
      You may also want to add it to your PATH environment variable.

[!] Android Studio (not installed)
[!] Connected device
    ! No devices available

! Doctor found issues in 4 categories.

@christopherfujino
Copy link
Member

@christopherfujino
FYI. Here is the procedure for cross-build and running dart-sdk I tried. Basically cross-build procedure is as per the official wiki.

Thanks for the response. Sorry I didn't respond, I've been busy putting out fires. I was also able to build the Dart SDK via the Dart repo directly. However, when I tried building it via the Flutter engine environment (link, this is how the Dart SDK that the Flutter SDK uses gets built) I got missing headers errors. I'm guessing I need to provide a sysroot? @HidenoriMatsubayashi

@HidenoriMatsubayashi
Copy link
Member Author

HidenoriMatsubayashi commented Aug 3, 2020

@christopherfujino

I'm guessing I need to provide a sysroot?

Yes. There are two ways. One is to use an external sysroot. Another way is to modify buildroot to use debian_sid_arm64-sysroot. I have tried both, and built Flutter Engine.

I could cross-build and pass the build.. I am going to send the PR.
https://github.com/HidenoriMatsubayashi/engine/tree/topic-support-linux-aarch64-cross-build
https://github.com/HidenoriMatsubayashi/buildroot/tree/topic-support-linux-aarch64-cross-build

.gclient file that I used is following.

solutions = [
  {
    "managed": False,
    "name": "src/flutter",
    "url": "https://github.com/HidenoriMatsubayashi/engine.git@9158ef84c326dc7260bfe39dd65189dd033d0b29",
    "custom_deps": {},
    "deps_file": "DEPS",
    "safesync_url": "",
  },
]

You need to modify flutter/DEPS file by yourself.

  'src': 'https://github.com/flutter/buildroot.git' + '@' + 'fe3b46e595e7ce1350e11aa0c90365976051f4a3',
↓
  'src': 'https://github.com/HidenoriMatsubayashi/buildroot.git' + '@' + '5658a5f9168a0025d25eb700b607abc4b1e247af',

Build Flutter Engine. The build is successful. But it is necessary to be careful about version combinations when actually run a Flutter app because of version check of DartVM.

$ ./flutter/tools/gn --target-os linux --linux-cpu arm64 --runtime-mode release
$ ninja -C out/linux_release_arm64

$ ./flutter/tools/gn --target-os linux --linux-cpu arm64 --runtime-mode debug
$ ninja -C out/linux_debug_arm64

@HidenoriMatsubayashi
Copy link
Member Author

I confirmed that the dart-sdk built above works on ARM64 Host. However, the flutter app has not been executed yet due to the error of dart-sdk. I am currently investigating..

[        ] [   +3 ms] executing: /opt/flutter/bin/cache/artifacts/engine/linux-arm64-release/gen_snapshot
--deterministic --snapshot_kind=app-aot-elf
--elf=/root/sample/.dart_tool/flutter_build/923ab164b9c4e4e2cb3229e9cc28c21c/app.so --strip
--no-causal-async-stacks --lazy-async-stacks
/root/sample/.dart_tool/flutter_build/923ab164b9c4e4e2cb3229e9cc28c21c/app.dill
[        ] [  +19 ms] Can't load Kernel binary: Invalid SDK hash.
[        ] [   +3 ms] Dart snapshot generator failed with exit code 254

@christopherfujino
Copy link
Member

@christopherfujino

I'm guessing I need to provide a sysroot?

Yes. There are two ways. One is to use an external sysroot. Another way is to modify buildroot to use debian_sid_arm64-sysroot. I have tried both, and built Flutter Engine.

I could cross-build and pass the build.. I am going to send the PR.
https://github.com/HidenoriMatsubayashi/engine/tree/topic-support-linux-aarch64-cross-build
https://github.com/HidenoriMatsubayashi/buildroot/tree/topic-support-linux-aarch64-cross-build

.gclient file that I used is following.

solutions = [
  {
    "managed": False,
    "name": "src/flutter",
    "url": "https://github.com/HidenoriMatsubayashi/engine.git@9158ef84c326dc7260bfe39dd65189dd033d0b29",
    "custom_deps": {},
    "deps_file": "DEPS",
    "safesync_url": "",
  },
]

You need to modify flutter/DEPS file by yourself.

  'src': 'https://github.com/flutter/buildroot.git' + '@' + 'fe3b46e595e7ce1350e11aa0c90365976051f4a3',
↓
  'src': 'https://github.com/HidenoriMatsubayashi/buildroot.git' + '@' + '5658a5f9168a0025d25eb700b607abc4b1e247af',

Build Flutter Engine. The build is successful. But it is necessary to be careful about version combinations when actually run a Flutter app because of version check of DartVM.

$ ./flutter/tools/gn --target-os linux --linux-cpu arm64 --runtime-mode release
$ ninja -C out/linux_release_arm64

$ ./flutter/tools/gn --target-os linux --linux-cpu arm64 --runtime-mode debug
$ ninja -C out/linux_debug_arm64

This is great! When I get some free time I'll check this out.

@HidenoriMatsubayashi
Copy link
Member Author

HidenoriMatsubayashi commented Aug 5, 2020

@christopherfujino
(cc: @stuartmorgan )

I was able to run flutte sdk & flutter app on ARM64 Host (Jetson Nano). I will send a PR after this, so I would like to request a review.

Screenshot from 2020-08-05 12-52-42

@google-cla
Copy link

google-cla bot commented Feb 12, 2021

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

@HidenoriMatsubayashi
Copy link
Member Author

@jonahwilliams
Thanks for the review.

@christopherfujino
I updated the PR. Please have a look

Thanks so much for all of your work on this

Your welcome! Actually, I (We) really want to use Flutter in our embedded Linux systems or products. I'm rooting for you (Flutter) :)

@christopherfujino
Copy link
Member

I believe the Linux flutter_plugins failure will be fixed by rebasing upstream.

@HidenoriMatsubayashi
Copy link
Member Author

the Linux flutter_plugins failure will be fixed by rebasing upstream.

Should I do that?

Copy link
Member

@christopherfujino christopherfujino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@christopherfujino
Copy link
Member

christopherfujino commented Feb 12, 2021

the Linux flutter_plugins failure will be fixed by rebasing upstream.

Should I do that?

はい、おねがいします!

Actually, I'm mistaken, this is also happening upstream. The failure will have to be fixed upstream first.

@sinojelly
Copy link

Can we use this on android arm64?
I can install dart on android + termux,
but flutter doesn't support android host, and no prebuilt tools for android arm64.

@adriano-moreira
Copy link

Can we use this on android arm64?
I can install dart on android + termux,
but flutter doesn't support android host, and no prebuilt tools for android arm64.

on termux uname -m return aarch64, I believe so work,

I tested on my smartphone(POCO X3) the HidenoriMatsubayashi fork on an Ubuntu 20.04 over termux instaled with Andronix,
and work fine,

@psstoyanov
Copy link

psstoyanov commented Feb 12, 2021

Can we use this on android arm64?
I can install dart on android + termux,
but flutter doesn't support android host, and no prebuilt tools for android arm64.

Using Termux or similar approach may be possible (as long as the app provides aarch64 userland and you have enough RAM - both will be impacted by how Android operates).

Android build tools aren't available for aarch64 afaik. Pretty much the mention of Android Studio not supporting arm64 in the design document (note that IntelliJ IDEs are using Java and can run on arm devices however the android build tools to create the apps aren't available for that platform).

Android host doesn't seem likely but that has a lot more to do with the OS itself. Your best bet would be Termux or similar (results will be of questionable success).

This PR is addressing Linux aarch64 host support. If you get Linux aarch64 environment on Android - sure. Otherwise, the PR is not relevant to your use case

@christopherfujino
Copy link
Member

Can we use this on android arm64?
I can install dart on android + termux,
but flutter doesn't support android host, and no prebuilt tools for android arm64.

You are correct, Android host is not supported by this PR, which specifically targets desktop linux.

I'm not sure how Termux works, but I imagine even if you could get the CLI tool to start, there would be other blockers to actually being able to develop Flutter applications.

@christopherfujino
Copy link
Member

The plugins were fixed upstream. I merged tip of tree to this branch to re-trigger CI.

@google-cla
Copy link

google-cla bot commented Feb 12, 2021

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

@HidenoriMatsubayashi
Copy link
Member Author

@christopherfujino
All tests have passed now. Thank you so much.

@christopherfujino christopherfujino merged commit 4cc0ab2 into flutter:master Feb 12, 2021
@sinojelly
Copy link

Can we use this on android arm64?
I can install dart on android + termux,
but flutter doesn't support android host, and no prebuilt tools for android arm64.

You are correct, Android host is not supported by this PR, which specifically targets desktop linux.

I'm not sure how Termux works, but I imagine even if you could get the CLI tool to start, there would be other blockers to actually being able to develop Flutter applications.

This is how termux works, Termux is not FHS compliant, and it is compiled with android ndk.
https://wiki.termux.com/wiki/Differences_from_Linux

Many packages have termux version, including dart.
https://github.com/termux/termux-packages

@muratcezan
Copy link

muratcezan commented Mar 19, 2021

Hello,
I get this error, how can i fix that?
Thank you.

[144/258] python ../../build/rust/run.py /home/murat/dart-sdk/sdk/buildtools/linux-x64/rust/bin/cargo build --target-dir /home/murat/dart-sdk/sdk/out/ReleaseXARM64/obj/third_party/wasmer --manifest-path /home/murat/dart-sdk/sdk/third_party/wasmer/Cargo.toml --target aarch64-unknown-linux-gnu --release
FAILED: obj/third_party/wasmer/aarch64-unknown-linux-gnu/release/libwasmer.a
python ../../build/rust/run.py /home/murat/dart-sdk/sdk/buildtools/linux-x64/rust/bin/cargo build --target-dir /home/murat/dart-sdk/sdk/out/ReleaseXARM64/obj/third_party/wasmer --manifest-path /home/murat/dart-sdk/sdk/third_party/wasmer/Cargo.toml --target aarch64-unknown-linux-gnu --release

Compiling wasmer-c-api v1.0.2
error: linking with cc failed: exit code: 1
|

@stuartmorgan stuartmorgan mentioned this pull request Mar 25, 2021
cbracken pushed a commit to cbracken/flutter_recipes that referenced this pull request Apr 6, 2021
Here's a successful LED build
https://ci.chromium.org/raw/build/logs.chromium.org/flutter/led/fujino_google.com/8e8335c33506c69d98777c4d07f5eb68e447c7596d03c0c26ed8c40237f06824/+/annotations?server=chromium-swarm.appspot.com.
There is no builder set up for this recipe yet, so after this CL lands
I will update the builder config.

Bug: flutter/flutter#61221
Change-Id: Ice10ec84127d60ae379b825fd03196b63fe7482a
Reviewed-on: https://flutter-review.googlesource.com/c/recipes/+/9580
Commit-Queue: Christopher Fujino <fujino@google.com>
Reviewed-by: Keyong Han <keyonghan@google.com>
@lycstar
Copy link
Contributor

lycstar commented Jun 18, 2021

Can we use this on android arm64?
I can install dart on android + termux,
but flutter doesn't support android host, and no prebuilt tools for android arm64.

Flutter can run in termux, but you need to modify flutter_tools and replace dart-sdk.
And it can run flutter build web in termux,because the main part of Web SDK is dart.
Screenshot_2021-06-18-14-36-30-254_com termux
Screenshot_2021-06-18-14-43-39-250_com android chrome

@ydddx
Copy link

ydddx commented Apr 21, 2022

Does it now support cross-compiling linux arm64 on linux x64?

@danimesq

This comment was marked as off-topic.

@HidenoriMatsubayashi
Copy link
Member Author

@danimesq
Unfortunately, the answer is no.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a: desktop Running on desktop platform-linux Building on or for Linux specifically tool Affects the "flutter" command-line tool. See also t: labels.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet