Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow AOT executables to be cross-compiled #28617

Open
nex3 opened this issue Feb 2, 2017 · 30 comments
Open

Allow AOT executables to be cross-compiled #28617

nex3 opened this issue Feb 2, 2017 · 30 comments

Comments

@nex3
Copy link
Member

@nex3 nex3 commented Feb 2, 2017

Currently application snapshots (as described here) can only be compiled for the architecture of the machine doing the compilation. This makes releasing cross-platform apps using these snapshots much more difficult, requiring multiple machines (including at least one physical machine, since OS X isn't virtualizable) to compile for all platforms.

It seems that the compilation process could do the initial run on the current platform, but then preserve the profiling information and use it to cross-compile the result to other platforms.

nex3 added a commit to sass/dart-sass that referenced this issue Feb 2, 2018
Might as well give some subset of our users a speed-up while we wait
for dart-lang/sdk#28617.
nex3 added a commit to sass/dart-sass that referenced this issue Feb 2, 2018
Might as well give some subset of our users a speed-up while we wait
for dart-lang/sdk#28617.
@rmacnak-google
Copy link
Contributor

@rmacnak-google rmacnak-google commented Jun 29, 2018

App snapshots are tied to the target ABI of the VM, not the host OS and not the host ABI. If you create an app snapshot using an IA32 VM, you can run the same snapshot on Windows, Mac or Linux. If you create an app snapshot using an X64 VM on Mac or Linux, you run it on the other (they share the sysv ABI), but not Windows (which has its own ABI). If you create an app snapshot using a SIMARM VM on your desktop, you can run it with an ARM VM on, say, an Android device (this is how Flutter works).

We will not produce VMs that target multiple ABIs. The assumption of a single target ABI as a build-time decision is quite deep in the VM.

@nex3
Copy link
Member Author

@nex3 nex3 commented Jun 29, 2018

I want to push back strongly on this. Startup performance is very important for providing my users with a good experience, and the Dart VM's startup performance without using application snapshots compares poorly to just about any other language (including Ruby, whose Sass implementation we're in the process of deprecating because of performance issues).

Other languages are able to do this. Easy cross-compilation is a major reason that Go is so popular for writing command-line tools, and if Dart provided it with similar ease it would make the server-side Dart world very compelling. But without it, Dart users are forced to deal with sub-par performance when they could easily get performance and portability with another language.

@nex3
Copy link
Member Author

@nex3 nex3 commented Jul 9, 2018

Ping... efficient startup speeds across all platforms are a very important customer feature.

@nex3 nex3 reopened this Aug 3, 2018
@nex3
Copy link
Member Author

@nex3 nex3 commented Aug 3, 2018

I haven't received any response, so I'm re-opening this. It's a pressing customer issue.

@passsy
Copy link

@passsy passsy commented Aug 22, 2018

One workaround is https://github.com/filiph/dartbin which creates a go wrapper. Not Dart2 compatible though and it requires go as dependency.

@nex3 nex3 changed the title Allow application snapshots to be cross-compiled Allow AOT executables to be cross-compiled May 20, 2019
@nex3
Copy link
Member Author

@nex3 nex3 commented May 20, 2019

Updating to AOT executables, since that's not the best way to distribute CLI applications.

@nex3
Copy link
Member Author

@nex3 nex3 commented May 30, 2019

@mit-mit has told me offline that "getting to full cross-compilation is going to be a long journey", but "the journey has already started; we just don't have a timeline for when it will end".

@shinayser
Copy link

@shinayser shinayser commented Nov 5, 2019

Looking forward to see this feature! Loving to work with dart

@Schwusch
Copy link

@Schwusch Schwusch commented Nov 13, 2019

This feature is a show stopper for us in order to adopt Dart in AWS Lambda. Since the AWS CLI runs the compiled executable in a Docker container, offline development is only possible in a Linux environment. Manual deployment without CI/CD is also impossible in Mac and Windows.

@t0mmar
Copy link

@t0mmar t0mmar commented Nov 15, 2019

Also a cross-compile option for linux ARM architectures please!

@mit-mit
Copy link
Member

@mit-mit mit-mit commented Nov 18, 2019

@DrSensor
Copy link

@DrSensor DrSensor commented Dec 10, 2019

Just found this wiki: https://github.com/dart-lang/sdk/wiki/Building-Dart-SDK-for-ARM-processors#building

I wonder if dart2native and the produced executable/snapshot works on RaspberryPi 🤔?

@ifree92
Copy link

@ifree92 ifree92 commented May 18, 2020

Ah, that would be awesome to have a cross compilation !
I just was looking for this functionality, but found this issue.
If my vote for this topic can change the priorities for dart2native tools improvement - I would be really glad.

I'm trying to use Dart for several projects for Raspberry, but that's hard to easily win this battle without ability to get ARM binary using my host OS (MacOS).

What's about plan? Will this feature be included once?

@ThomasJaeger
Copy link

@ThomasJaeger ThomasJaeger commented Jun 27, 2020

Is there an update to this? Really interested in cross-compilation!

@jpnurmi
Copy link

@jpnurmi jpnurmi commented Sep 6, 2020

Would something like this be an interesting mid-term solution?

 $ dart2native foo.dart \
    --gen-snapshot out/ProductSIMARM/dart-sdk/bin/utils/gen_snapshot \
    --aot-runtime out/ProductXARM/dart-sdk/bin/dartaotruntime

jpnurmi@9625310

@mraleph
Copy link
Member

@mraleph mraleph commented Sep 6, 2020

Would something like this be an interesting mid-term solution?

That's pretty close to how it would be implemented in the long term too (if we had resources or if we had somebody willing to contribute external implementation). Few missing pieces to take it all the way:

  • We need to create X-configurations for producing gen_snapshot for all configurations of host-target OSes and architectures.
  • We need to figure out packaging and distribution for these binary artifacts. I don't think including them into SDK is scalable - we need to figure out "out of band" delivery mechanism (which might somewhat complicate Dart SDK release process).

C0MM4ND added a commit to ngchain/ngwallet-dart that referenced this issue Nov 14, 2020
@matsp
Copy link

@matsp matsp commented Apr 2, 2021

What needs to be done here? I am not an expert at all for this topic but really want to help and make this feature real. If someone is able to give me some lead I am willing to help.

@jairoareyes
Copy link

@jairoareyes jairoareyes commented Apr 23, 2021

Ah, that would be awesome to have a cross compilation !
I just was looking for this functionality, but found this issue.
If my vote for this topic can change the priorities for dart2native tools improvement - I would be really glad.

I'm trying to use Dart for several projects for Raspberry, but that's hard to easily win this battle without ability to get ARM binary using my host OS (MacOS).

What's about plan? Will this feature be included once?

Have you found any alternative to compile for Rpi devices?
I think the Dart dev team is very busy in these days.

@budius
Copy link

@budius budius commented Apr 24, 2021

The way I've done before:

@jairoareyes do the development normally on your Ubuntu/mac/win, download the dart SDK to one rPi and setup a lil script that rsync the project files to the pi and runs pub get and compile.

Not "the most" efficient, but it works nicely

@fzyzcjy
Copy link

@fzyzcjy fzyzcjy commented Jun 16, 2021

Any updates...? Or this will never be done?

@mraleph
Copy link
Member

@mraleph mraleph commented Jun 16, 2021

There are currently no immediate plans to address this. There is a bunch of improvements one could make to dart2native (e.g. fix how we link runtime with AOT snapshot to make binaries signable, fix packaging for dart2native artifacts, support cross compilation, etc), but we are too occupied with higher priority issues to work on them.

If somebody is willing to contribute, I will be happy to provide design and guidance.

@abasty

This comment was marked as off-topic.

@mraleph

This comment was marked as off-topic.

@abasty

This comment was marked as off-topic.

@mraleph

This comment was marked as off-topic.

@abasty

This comment was marked as off-topic.

@mraleph

This comment was marked as off-topic.

@maks
Copy link

@maks maks commented Sep 8, 2021

@mraleph per @jpnurmi method (https://medium.com/flutter-community/cross-compiling-dart-apps-f88e69824639) and patch to dart2native (or I guess just dart compile these days) is it now a matter of having the Dart SDK release process build & publish the "cross-platform" (x86 that produces for arm, arm that produces for x86) versions of gen_snapshot and dartaotruntime that would at least give end users all the building blocks to do cross-compilation without needing to download SDK src and patch/build those tools ourselves, without initially worrying about figuring out what the "out of band" delivery mechanism needs to be?

@mraleph
Copy link
Member

@mraleph mraleph commented Sep 8, 2021

@maks well, I don't think we want to include gen_snapshot and dart_precompiled_runtime pair for each configuration (which is actually a combination of CPU and OS - something that @jpnurmi did not tackle at all) into an SDK archive.

FWIW there is no complexity in figuring out-of-an-SDK delivery mechanism - we can just use a cloud storage bucket (in the same way Flutter does it). There will be of course some design around release process for such things, how CI testing would look like, etc.

@maks
Copy link

@maks maks commented Sep 8, 2021

@mraleph thanks for quick reply 👍🏻 and yes sorry that's what I was trying to say, sorry I didn't explain it very well.

What I thought was just that: as part of your the Dart release process, publishing somewhere like a cloud bucket all the combinations of gen_snapshot and dart_precompiled_runtime binaries and not including them in the SDK and then just let me, the user, download the specific ones I need. Of course it would be nice to have that automated by tooling the way Flutter does with downloading its engines, getting the correct matching versions etc, but as a first step I would be happy to just download them vs needing to build them locally out of the sdk src.

And for sure I guess there's a fair bit of work around doing that in the release process, but unfort its not really something that can be done except by the Dart team so I'll keep 🤞🏻 and watch this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet