New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Android / Arm #3376

Closed
carllerche opened this Issue Mar 14, 2015 · 20 comments

Comments

Projects
None yet
10 participants
@carllerche

carllerche commented Mar 14, 2015

I maintain a number of Rust projects. Rust supports Android / ARM processors as a compilation target. It would be nice to be able to include such an environment in CI runs.

For example, the following project provides bindings to OS APIs https://github.com/carllerche/nix-rust, and this one a non-blocking IO event loop: https://github.com/carllerche/mio

Both theoretically support being run on Android / ARM processors but running the tests as part of the CI run would be nice 😄

@meatballhat

This comment has been minimized.

Show comment
Hide comment
@meatballhat

meatballhat Mar 16, 2015

Member

Paging @tianon 😺

Member

meatballhat commented Mar 16, 2015

Paging @tianon 😺

@moul

This comment has been minimized.

Show comment
Hide comment
@moul

moul Mar 16, 2015

(subscribing)
I'm working on this and can provide ARM servers from labs.online.net

moul commented Mar 16, 2015

(subscribing)
I'm working on this and can provide ARM servers from labs.online.net

@meatballhat

This comment has been minimized.

Show comment
Hide comment
@meatballhat

meatballhat Mar 16, 2015

Member

@moul 👍 why thank you! In all likelihood we'll be implementing this as a backend over here: https://github.com/travis-ci/worker/tree/master/lib/backend. Any help you can offer with regard to how best to use the OnlineLabs API is greatly appreciated 😃.

/cc @henrikhodne

Member

meatballhat commented Mar 16, 2015

@moul 👍 why thank you! In all likelihood we'll be implementing this as a backend over here: https://github.com/travis-ci/worker/tree/master/lib/backend. Any help you can offer with regard to how best to use the OnlineLabs API is greatly appreciated 😃.

/cc @henrikhodne

@moul

This comment has been minimized.

Show comment
Hide comment
@moul

moul Mar 17, 2015

@meatballhat I'm building an image with travis-ci/worker in.

Repository: online-labs/image-service-travis

For now, it's only for testing, but we can imagine to use this base to build the official travis-ci image on Online Labs

note: it use Docker to build the image, but the final image is meant to be run on a real server without Docker

moul commented Mar 17, 2015

@meatballhat I'm building an image with travis-ci/worker in.

Repository: online-labs/image-service-travis

For now, it's only for testing, but we can imagine to use this base to build the official travis-ci image on Online Labs

note: it use Docker to build the image, but the final image is meant to be run on a real server without Docker

@meatballhat

This comment has been minimized.

Show comment
Hide comment
@meatballhat

meatballhat Mar 17, 2015

Member

@moul thanks so much for your work in this! I should clarify that there are at least two components needed.

One is travis-ci/worker, which ideally runs in our Amazon VPC and talks to an API (such as Online Labs). We already have some provisioning bits in place for this, thankfully, although it is lacking a backend for the Online Labs API.

The other is to have at least one image available for provisioning within a given cloud that includes the bare minimum tooling to be able to run a script generated by travis-build. As of today, this means bash and a few other shell basics. For all other clouds, we're currently using chef-solo in combination with either travis-images or packer to perform this provisioning, which is part of the reason why I tackled packer-builder-onlinelabs. We're hoping to standardize on packer, fwiw.

Member

meatballhat commented Mar 17, 2015

@moul thanks so much for your work in this! I should clarify that there are at least two components needed.

One is travis-ci/worker, which ideally runs in our Amazon VPC and talks to an API (such as Online Labs). We already have some provisioning bits in place for this, thankfully, although it is lacking a backend for the Online Labs API.

The other is to have at least one image available for provisioning within a given cloud that includes the bare minimum tooling to be able to run a script generated by travis-build. As of today, this means bash and a few other shell basics. For all other clouds, we're currently using chef-solo in combination with either travis-images or packer to perform this provisioning, which is part of the reason why I tackled packer-builder-onlinelabs. We're hoping to standardize on packer, fwiw.

@moul

This comment has been minimized.

Show comment
Hide comment
@moul

moul Mar 18, 2015

Ok, got it; Don't worry, I will keep this image for my tests :)
My goal is to be sure you have everything needed

I will try to generate some scripts using travis-build and get as much needed binaries as possible for the different languages (travis-build/lib/travis/build/script) in a single image

We will probably have some limitations on 32bits armhf and you will probably need to create a matrix of available tests per architecture

moul commented Mar 18, 2015

Ok, got it; Don't worry, I will keep this image for my tests :)
My goal is to be sure you have everything needed

I will try to generate some scripts using travis-build and get as much needed binaries as possible for the different languages (travis-build/lib/travis/build/script) in a single image

We will probably have some limitations on 32bits armhf and you will probably need to create a matrix of available tests per architecture

@meatballhat

This comment has been minimized.

Show comment
Hide comment
@meatballhat

meatballhat Mar 18, 2015

Member

Even if you're able to get the image capable of running a script from
language: generic, this is a great first step. There is a travis compile command available that may come in handy, although I'm told the
results are highly variable. We can also generate such a script via
internal Travis admin tools if necessary.
On Mar 18, 2015 5:19 AM, "Manfred Touron" notifications@github.com wrote:

Ok, got it; Don't worry, I will keep this image for my tests :)
My goal is to be sure you have everything needed

I will try to generate some scripts using travis-build
https://github.com/travis-ci/travis-build/ and get as much needed
binaries as possible for the different languages (
travis-build/lib/travis/build/script
https://github.com/travis-ci/travis-build/tree/master/lib/travis/build/script)
in a single image

We will probably have some limitations on 32bits armhf and you will
probably need to create a matrix of available tests per architecture


Reply to this email directly or view it on GitHub
#3376 (comment)
.

Member

meatballhat commented Mar 18, 2015

Even if you're able to get the image capable of running a script from
language: generic, this is a great first step. There is a travis compile command available that may come in handy, although I'm told the
results are highly variable. We can also generate such a script via
internal Travis admin tools if necessary.
On Mar 18, 2015 5:19 AM, "Manfred Touron" notifications@github.com wrote:

Ok, got it; Don't worry, I will keep this image for my tests :)
My goal is to be sure you have everything needed

I will try to generate some scripts using travis-build
https://github.com/travis-ci/travis-build/ and get as much needed
binaries as possible for the different languages (
travis-build/lib/travis/build/script
https://github.com/travis-ci/travis-build/tree/master/lib/travis/build/script)
in a single image

We will probably have some limitations on 32bits armhf and you will
probably need to create a matrix of available tests per architecture


Reply to this email directly or view it on GitHub
#3376 (comment)
.

@moul

This comment has been minimized.

Show comment
Hide comment
@moul

moul Mar 18, 2015

I'm interested to get those generated scripts if you can
Thanks

moul commented Mar 18, 2015

I'm interested to get those generated scripts if you can
Thanks

@meatballhat

This comment has been minimized.

Show comment
Hide comment
@meatballhat

meatballhat Mar 18, 2015

Member

@moul here's a stripped down example with echo for both the install and script steps: https://gist.github.com/meatballhat/dd3b723a25082aea0586

Member

meatballhat commented Mar 18, 2015

@moul here's a stripped down example with echo for both the install and script steps: https://gist.github.com/meatballhat/dd3b723a25082aea0586

@moul

This comment has been minimized.

Show comment
Hide comment
@moul

moul Mar 18, 2015

Perfect, I will give a try, is the /etc/profile custom and could block my tests ?

moul commented Mar 18, 2015

Perfect, I will give a try, is the /etc/profile custom and could block my tests ?

@meatballhat

This comment has been minimized.

Show comment
Hide comment
@meatballhat

meatballhat Mar 18, 2015

Member

@moul /etc/profile itself is pretty stock:

# /etc/profile: system-wide .profile file for the Bourne shell (sh(1))
# and Bourne compatible shells (bash(1), ksh(1), ash(1), ...).

if [ "$PS1" ]; then
  if [ "$BASH" ] && [ "$BASH" != "/bin/sh" ]; then
    # The file bash.bashrc already sets the default PS1.
    # PS1='\h:\w\$ '
    if [ -f /etc/bash.bashrc ]; then
      . /etc/bash.bashrc
    fi
  else
    if [ "`id -u`" -eq 0 ]; then
      PS1='# '
    else
      PS1='$ '
    fi
  fi
fi

# The default umask is now handled by pam_umask.
# See pam_umask(8) and /etc/login.defs.

if [ -d /etc/profile.d ]; then
  for i in /etc/profile.d/*.sh; do
    if [ -r $i ]; then
      . $i
    fi
  done
  unset i
fi

It's the contents of /etc/profile.d/*.sh that get interesting 😃

root@c5bf9b01f88e:/# ls -l /etc/profile.d
total 40
-rwxr-xr-x 1 root   root     42 Feb  5 14:46 clang.sh
-rw-r--r-- 1 root   root     44 Feb  5 14:51 java_home.sh
-rwxr-xr-x 1 travis travis  166 Feb  5 15:43 load_jdk_switcher.sh
-rwxr-xr-x 1 root   root     44 Feb  5 14:52 maven.sh
-rwxr-xr-x 1 travis travis   91 Feb  5 14:54 nvm.sh
-rw-r--r-- 1 root   root     36 Feb  5 15:08 phantomjs.sh
-rwxr-xr-x 1 travis travis   75 Feb  5 14:53 rvm.sh
-rwxr-xr-x 1 travis travis 1196 Feb  5 14:44 travis_environment.sh
-rw-r--r-- 1 root   root     26 Feb  5 15:40 travis.sh
-rwxr-xr-x 1 root   root    328 Feb  5 14:46 Z90-gimme.sh

All of these are written by cookbooks in the travis-cookbooks repo under the ci environment tree.

Member

meatballhat commented Mar 18, 2015

@moul /etc/profile itself is pretty stock:

# /etc/profile: system-wide .profile file for the Bourne shell (sh(1))
# and Bourne compatible shells (bash(1), ksh(1), ash(1), ...).

if [ "$PS1" ]; then
  if [ "$BASH" ] && [ "$BASH" != "/bin/sh" ]; then
    # The file bash.bashrc already sets the default PS1.
    # PS1='\h:\w\$ '
    if [ -f /etc/bash.bashrc ]; then
      . /etc/bash.bashrc
    fi
  else
    if [ "`id -u`" -eq 0 ]; then
      PS1='# '
    else
      PS1='$ '
    fi
  fi
fi

# The default umask is now handled by pam_umask.
# See pam_umask(8) and /etc/login.defs.

if [ -d /etc/profile.d ]; then
  for i in /etc/profile.d/*.sh; do
    if [ -r $i ]; then
      . $i
    fi
  done
  unset i
fi

It's the contents of /etc/profile.d/*.sh that get interesting 😃

root@c5bf9b01f88e:/# ls -l /etc/profile.d
total 40
-rwxr-xr-x 1 root   root     42 Feb  5 14:46 clang.sh
-rw-r--r-- 1 root   root     44 Feb  5 14:51 java_home.sh
-rwxr-xr-x 1 travis travis  166 Feb  5 15:43 load_jdk_switcher.sh
-rwxr-xr-x 1 root   root     44 Feb  5 14:52 maven.sh
-rwxr-xr-x 1 travis travis   91 Feb  5 14:54 nvm.sh
-rw-r--r-- 1 root   root     36 Feb  5 15:08 phantomjs.sh
-rwxr-xr-x 1 travis travis   75 Feb  5 14:53 rvm.sh
-rwxr-xr-x 1 travis travis 1196 Feb  5 14:44 travis_environment.sh
-rw-r--r-- 1 root   root     26 Feb  5 15:40 travis.sh
-rwxr-xr-x 1 root   root    328 Feb  5 14:46 Z90-gimme.sh

All of these are written by cookbooks in the travis-cookbooks repo under the ci environment tree.

@shouze

This comment has been minimized.

Show comment
Hide comment
@shouze

shouze Aug 11, 2015

@meatballhat well, when can we be able to make travis builds on arm machines ?! Thx @moul for this repo, can't wait!

shouze commented Aug 11, 2015

@meatballhat well, when can we be able to make travis builds on arm machines ?! Thx @moul for this repo, can't wait!

@bobvanderlinden

This comment has been minimized.

Show comment
Hide comment
@bobvanderlinden

bobvanderlinden Nov 22, 2015

For anyone interested, it's possible to build on emulated ARM through Travis using qemu-user-static and binfmt_misc. Here is an example of how we used it for our project. Basically, binfmt_misc is configured to use /usr/local/bin/qemu-arm-static whenever the kernel find a binary that is ARM-based. Then a ARM-based docker image is used, like armbuild/debian:wheezy (link). This docker image is a minimal version of ARM debian and (in addition) has added /usr/local/bin/qemu-arm-static. Everything that's being run inside of the container is being emulated.

Do note though that it is very slow. For our purposes it resulted in > 50m, so a timeout. For other purposes I can imagine it being sufficient.

Hardware-based ARM Travis workers would be ideal. Is there some way to host your own worker that travis-ci.org can use?

bobvanderlinden commented Nov 22, 2015

For anyone interested, it's possible to build on emulated ARM through Travis using qemu-user-static and binfmt_misc. Here is an example of how we used it for our project. Basically, binfmt_misc is configured to use /usr/local/bin/qemu-arm-static whenever the kernel find a binary that is ARM-based. Then a ARM-based docker image is used, like armbuild/debian:wheezy (link). This docker image is a minimal version of ARM debian and (in addition) has added /usr/local/bin/qemu-arm-static. Everything that's being run inside of the container is being emulated.

Do note though that it is very slow. For our purposes it resulted in > 50m, so a timeout. For other purposes I can imagine it being sufficient.

Hardware-based ARM Travis workers would be ideal. Is there some way to host your own worker that travis-ci.org can use?

@larsbrinkhoff

This comment has been minimized.

Show comment
Hide comment
@larsbrinkhoff

larsbrinkhoff Dec 30, 2015

👍 For ARM builds.

larsbrinkhoff commented Dec 30, 2015

👍 For ARM builds.

@moul

This comment has been minimized.

Show comment
Hide comment
@moul

moul Dec 30, 2015

Hello, I'm the author of the armbuild/debian:wheezy image quoted by @bobvanderlinden,

I moved my work from armbuild/* to multiarch/* (organization), which is basically the same but for any targets:

I also created a binfmt-support configurator as a Docker image (link)

So if your kernel supports the binfmt feature, you can on a fresh host start to use cross-platform Docker images with:

$ docker run --rm --privileged multiarch/qemu-user-static:register
$ docker run -it --rm multiarch/debian-debootstrap:armhf-jessie
root@88a6ead2fd63:/# uname -a
Linux 88a6ead2fd63 4.1.13-boot2docker #1 SMP Fri Nov 20 19:05:50 UTC 2015 armv7l GNU/Linux
root@88a6ead2fd63:/#

As it only uses Docker client, it works out of the box with local and remote Docker servers (tested on boot2docker and swarm-based remote Docker cluster)

moul commented Dec 30, 2015

Hello, I'm the author of the armbuild/debian:wheezy image quoted by @bobvanderlinden,

I moved my work from armbuild/* to multiarch/* (organization), which is basically the same but for any targets:

I also created a binfmt-support configurator as a Docker image (link)

So if your kernel supports the binfmt feature, you can on a fresh host start to use cross-platform Docker images with:

$ docker run --rm --privileged multiarch/qemu-user-static:register
$ docker run -it --rm multiarch/debian-debootstrap:armhf-jessie
root@88a6ead2fd63:/# uname -a
Linux 88a6ead2fd63 4.1.13-boot2docker #1 SMP Fri Nov 20 19:05:50 UTC 2015 armv7l GNU/Linux
root@88a6ead2fd63:/#

As it only uses Docker client, it works out of the box with local and remote Docker servers (tested on boot2docker and swarm-based remote Docker cluster)

@bcecchinato

This comment has been minimized.

Show comment
Hide comment
@bcecchinato

bcecchinato Jan 2, 2016

EDIT: using @moul docker run --rm --privileged multiarch/qemu-user-static:register for a docker build seemed to be enough. Thanks !

bcecchinato commented Jan 2, 2016

EDIT: using @moul docker run --rm --privileged multiarch/qemu-user-static:register for a docker build seemed to be enough. Thanks !

@meatballhat

This comment has been minimized.

Show comment
Hide comment
@meatballhat

meatballhat Nov 16, 2016

Member

I'm closing this out based on what appears to be a reasonable workaround. Please yell if you think otherwise! ❤️

Member

meatballhat commented Nov 16, 2016

I'm closing this out based on what appears to be a reasonable workaround. Please yell if you think otherwise! ❤️

@dlech

This comment has been minimized.

Show comment
Hide comment
@dlech

dlech Nov 16, 2016

Although qemu does work about 80-90% of the time in my experience, it would still be nice to be able to build on real ARM hardware for the times when qemu is not good enough.

dlech commented Nov 16, 2016

Although qemu does work about 80-90% of the time in my experience, it would still be nice to be able to build on real ARM hardware for the times when qemu is not good enough.

@giggio

This comment has been minimized.

Show comment
Hide comment
@giggio

giggio Dec 11, 2016

We can't use qemu to run Go based programs. See golang/go#13024 for more information.
I agree we should have support for ARM on Travis.

giggio commented Dec 11, 2016

We can't use qemu to run Go based programs. See golang/go#13024 for more information.
I agree we should have support for ARM on Travis.

@swarren

This comment has been minimized.

Show comment
Hide comment
@swarren

swarren Mar 14, 2018

FWIW, I wanted to use qemu-aarch64-static to build the Arduino IDE (and its many dependencies) for AArch64 but found that the emulation wasn't sufficient and had to switch to real HW. So, the workaround described here will work in many cases, sure, but it certainly doesn't work for all cases. True AArch64 (64-bit ARM) hardware would be quite useful.

swarren commented Mar 14, 2018

FWIW, I wanted to use qemu-aarch64-static to build the Arduino IDE (and its many dependencies) for AArch64 but found that the emulation wasn't sufficient and had to switch to real HW. So, the workaround described here will work in many cases, sure, but it certainly doesn't work for all cases. True AArch64 (64-bit ARM) hardware would be quite useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment