Cross-compile for ARM processors support #1332

Open
varosi opened this Issue Nov 10, 2015 · 27 comments

Comments

Projects
None yet
9 participants
@varosi

varosi commented Nov 10, 2015

Do someone thought about to include ARM based compilation support for Stack?
That way Haskell will become even more portable.

@borsboom

This comment has been minimized.

Show comment
Hide comment
@borsboom

borsboom Nov 12, 2015

Contributor

There's a GHC Linux ARM bindist, it shouldn't take much to support it. Aside from stack setup, it might even already work. Have you tried it?

Contributor

borsboom commented Nov 12, 2015

There's a GHC Linux ARM bindist, it shouldn't take much to support it. Aside from stack setup, it might even already work. Have you tried it?

@varosi

This comment has been minimized.

Show comment
Hide comment
@varosi

varosi Nov 12, 2015

I was tried before to build cross-compiler, i.e. building GHC on Windows to build ARM builds. But without much success. That is way it would be great if there is easier way for using Haskell on ARM. This will open the door for using Haskell in Embedded systems more easily.

varosi commented Nov 12, 2015

I was tried before to build cross-compiler, i.e. building GHC on Windows to build ARM builds. But without much success. That is way it would be great if there is easier way for using Haskell on ARM. This will open the door for using Haskell in Embedded systems more easily.

@borsboom borsboom changed the title from ARM processors support to Cross-compile for ARM processors support Nov 13, 2015

@borsboom borsboom modified the milestones: Future, P3: Optional Nov 13, 2015

@borsboom borsboom removed the type: discuss label Nov 13, 2015

@borsboom

This comment has been minimized.

Show comment
Hide comment
@borsboom

borsboom Nov 13, 2015

Contributor

We may someday support cross-compiling, but that day isn't here yet. I've put this in the "Future" milestone to keep track of the idea.

Contributor

borsboom commented Nov 13, 2015

We may someday support cross-compiling, but that day isn't here yet. I've put this in the "Future" milestone to keep track of the idea.

@varosi

This comment has been minimized.

Show comment
Hide comment
@varosi

varosi Nov 13, 2015

Thank you!
What about normal compiling on ARM based architecture, like Raspberry Pi?

varosi commented Nov 13, 2015

Thank you!
What about normal compiling on ARM based architecture, like Raspberry Pi?

@borsboom

This comment has been minimized.

Show comment
Hide comment
@borsboom

borsboom Nov 18, 2015

Contributor

Have you tried the GHC Linux ARM bindist I mentioned
earlier? In theory, that should work with Stack (you'll just have to
install it manually, since stack setup doesn't know about it yet). If
that works, we can look at adding stack setup support (and if not, we can
see about getting it to work).

Contributor

borsboom commented Nov 18, 2015

Have you tried the GHC Linux ARM bindist I mentioned
earlier? In theory, that should work with Stack (you'll just have to
install it manually, since stack setup doesn't know about it yet). If
that works, we can look at adding stack setup support (and if not, we can
see about getting it to work).

@asakeron

This comment has been minimized.

Show comment
Hide comment
@asakeron

asakeron Nov 27, 2015

I decided to try the approach suggested by @borsboom and I couldn't get stack to build through cabal-install on an ARM machine because one of its dependency, cryptonite-0.9, fails to compile due to the SSE2 instruction set not being supported by the processor. I believe there are no ARM processors that implement this instruction set.

I decided to try the approach suggested by @borsboom and I couldn't get stack to build through cabal-install on an ARM machine because one of its dependency, cryptonite-0.9, fails to compile due to the SSE2 instruction set not being supported by the processor. I believe there are no ARM processors that implement this instruction set.

@borsboom

This comment has been minimized.

Show comment
Hide comment
@borsboom

borsboom Nov 27, 2015

Contributor

That is unfortunate. I'd suggest filing an issue upstream to see if
there's a way around that. It looks like the blake2 library may only be
used for a hash algorithm which isn't necessary for TLS, so cryptonite
having a flag to build without it on ARM seems like it would be useful.

Contributor

borsboom commented Nov 27, 2015

That is unfortunate. I'd suggest filing an issue upstream to see if
there's a way around that. It looks like the blake2 library may only be
used for a hash algorithm which isn't necessary for TLS, so cryptonite
having a flag to build without it on ARM seems like it would be useful.

@mgsloan

This comment has been minimized.

Show comment
Hide comment
@mgsloan

mgsloan Nov 27, 2015

Collaborator

Technically, flags aren't supposed to change the API. However, this isn't necessarily what's actually done in practice, and stack does make it reasonable to add and use a manual flag for this purpose.

Collaborator

mgsloan commented Nov 27, 2015

Technically, flags aren't supposed to change the API. However, this isn't necessarily what's actually done in practice, and stack does make it reasonable to add and use a manual flag for this purpose.

@varosi

This comment has been minimized.

Show comment
Hide comment
@varosi

varosi Dec 8, 2015

@borsboom, I succeeded to build from sources easily GHC 7.10.2, so it is buildable on ARM v7. There is also cabal-install 1.20, too.

Here you could see some ready packages for ARM:
https://packages.debian.org/en/jessie/haskell/

I think if it is possible to build stack on top of that?
And the second feature request is to support Stackage on ARM, too.

varosi commented Dec 8, 2015

@borsboom, I succeeded to build from sources easily GHC 7.10.2, so it is buildable on ARM v7. There is also cabal-install 1.20, too.

Here you could see some ready packages for ARM:
https://packages.debian.org/en/jessie/haskell/

I think if it is possible to build stack on top of that?
And the second feature request is to support Stackage on ARM, too.

@andreyk0

This comment has been minimized.

Show comment
Hide comment
@andreyk0

andreyk0 Dec 12, 2015

Haven't tried to figure out cross-compiler but Docker / Static QEMU combo seems to work for a few arm projects.
Latest debian comes with working GHC, 7.8.x is the version I ended up with.
Installed on SD card on the device, took file system off of SD card and loaded into docker container.
Installed static QEMU with binformat support, there's a few articles about it, e.g. https://github.com/dweinstein/dockerfile-qemu-arm-chroot .
This gives a working ARM compiler and cabal, from there stack can be compiled and used. Takes a while and occasionally crashes and needs to be restarted but usable.
Interesting directories can be 'mounted' as docker volumes to access current project and persist files across different runs.

E.g. here's my 'platform' launcher script, hope this helps:
#!/bin/sh
exec docker run
-it --rm -u 1000:1000
-v /home:/home
-v /home/andrey/.armhf/local:/home/andrey/.local
-v /home/andrey/.armhf/cabal:/home/andrey/.cabal
-v /home/andrey/.armhf/stack/precompiled:/home/andrey/.stack/precompiled
-v /tmp:/tmp
-v /opt/qemu-2.4.0/bin/qemu-arm:/usr/bin/qemu-arm-static
-w pwd
-e QEMU_SET_ENV=QEMU_CPU=cortex-a9
-e HOME=$HOME
-e PATH=$PATH
debian/haskell-platform-armhf:latest "$@"

Haven't tried to figure out cross-compiler but Docker / Static QEMU combo seems to work for a few arm projects.
Latest debian comes with working GHC, 7.8.x is the version I ended up with.
Installed on SD card on the device, took file system off of SD card and loaded into docker container.
Installed static QEMU with binformat support, there's a few articles about it, e.g. https://github.com/dweinstein/dockerfile-qemu-arm-chroot .
This gives a working ARM compiler and cabal, from there stack can be compiled and used. Takes a while and occasionally crashes and needs to be restarted but usable.
Interesting directories can be 'mounted' as docker volumes to access current project and persist files across different runs.

E.g. here's my 'platform' launcher script, hope this helps:
#!/bin/sh
exec docker run
-it --rm -u 1000:1000
-v /home:/home
-v /home/andrey/.armhf/local:/home/andrey/.local
-v /home/andrey/.armhf/cabal:/home/andrey/.cabal
-v /home/andrey/.armhf/stack/precompiled:/home/andrey/.stack/precompiled
-v /tmp:/tmp
-v /opt/qemu-2.4.0/bin/qemu-arm:/usr/bin/qemu-arm-static
-w pwd
-e QEMU_SET_ENV=QEMU_CPU=cortex-a9
-e HOME=$HOME
-e PATH=$PATH
debian/haskell-platform-armhf:latest "$@"

@Chobbes

This comment has been minimized.

Show comment
Hide comment
@Chobbes

Chobbes Dec 18, 2015

I have the same problem with cryptonite when attempting to install stack on ARM. This is really sad because stack would make development on ARM so much more tolerable as compiling is so slow that cabal-sandboxes for each project can really suck a lot of time out of you.

I have recently gotten GHC working on various ARM things, and have even gotten a cross compiler to (sort of) work with template Haskell. This has been documented in a blog post here:

https://haskellembedded.github.io/posts/2015-12-15-arm.html

if anybody is having trouble with GHC on ARMv7 then feel free to contact me (@Chobbez on twitter, Chobbes on freenode - come join #haskell-embedded maybe!)

There are some problems with cross-compilation involving template Haskell and FFI stuff. I have gotten template Haskell to work in a cross compiler by using binfmt_misc to transparently execute any ARM template Haskell with qemu-arm.

I'm currently looking into the FFI issues that I'm having with the cross-compiler, if anybody has any suggestions they would be greatly appreciated.

Additionally I am willing to put in work to get Stack working in some fashion for ARM development, because this is something that I desire greatly!

@andreyk0 how have you managed to install Stack on ARM with the issue that @asakeron mentioned? Is it just an x86 / x86_64 stack that calls an ARM GHC?

Chobbes commented Dec 18, 2015

I have the same problem with cryptonite when attempting to install stack on ARM. This is really sad because stack would make development on ARM so much more tolerable as compiling is so slow that cabal-sandboxes for each project can really suck a lot of time out of you.

I have recently gotten GHC working on various ARM things, and have even gotten a cross compiler to (sort of) work with template Haskell. This has been documented in a blog post here:

https://haskellembedded.github.io/posts/2015-12-15-arm.html

if anybody is having trouble with GHC on ARMv7 then feel free to contact me (@Chobbez on twitter, Chobbes on freenode - come join #haskell-embedded maybe!)

There are some problems with cross-compilation involving template Haskell and FFI stuff. I have gotten template Haskell to work in a cross compiler by using binfmt_misc to transparently execute any ARM template Haskell with qemu-arm.

I'm currently looking into the FFI issues that I'm having with the cross-compiler, if anybody has any suggestions they would be greatly appreciated.

Additionally I am willing to put in work to get Stack working in some fashion for ARM development, because this is something that I desire greatly!

@andreyk0 how have you managed to install Stack on ARM with the issue that @asakeron mentioned? Is it just an x86 / x86_64 stack that calls an ARM GHC?

@andreyk0

This comment has been minimized.

Show comment
Hide comment
@andreyk0

andreyk0 Dec 19, 2015

@Chobbes
# file stack
stack: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=2f5d590056ac8e25c6ef69fbe75abe4fe8e2c42a, stripped

I never actually ran it on the ARM boards, only via static QEMU(2.4.1)/Docker(1.9.1).
I've arrived at the current version of stack incrementally. I think the first version I've successfully built was 0.1.5.0, after that stack upgraded itself to 0.1.10.0.
In the ~/.stack dir I find arm-linux-ghc-7.10.3/cryptonite-0.6, so, whatever cabal-install version used that and can build some version of stack can be used to bootstrap.

Here's my stack-0.1.10.0-Arm.gz binary, uploaded as a GH gist, if that helps.

Build process crashes / hangs occasionally but I'm sure QEMU bugs are responsible for some of that, it's a bit better with 2.4.1 than with the earlier versions I've tried.

Generated binaries seem Ok/stable on debian stretch, cortex-a8 A10 Olinuxino Lime board.
Less luck with debian jessie on A20 Lime, 1 out of 3 projects crashes (segfault), haven't figured out yet, maybe it doesn't like binaries built with stretch container.

@Chobbes
# file stack
stack: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.32, BuildID[sha1]=2f5d590056ac8e25c6ef69fbe75abe4fe8e2c42a, stripped

I never actually ran it on the ARM boards, only via static QEMU(2.4.1)/Docker(1.9.1).
I've arrived at the current version of stack incrementally. I think the first version I've successfully built was 0.1.5.0, after that stack upgraded itself to 0.1.10.0.
In the ~/.stack dir I find arm-linux-ghc-7.10.3/cryptonite-0.6, so, whatever cabal-install version used that and can build some version of stack can be used to bootstrap.

Here's my stack-0.1.10.0-Arm.gz binary, uploaded as a GH gist, if that helps.

Build process crashes / hangs occasionally but I'm sure QEMU bugs are responsible for some of that, it's a bit better with 2.4.1 than with the earlier versions I've tried.

Generated binaries seem Ok/stable on debian stretch, cortex-a8 A10 Olinuxino Lime board.
Less luck with debian jessie on A20 Lime, 1 out of 3 projects crashes (segfault), haven't figured out yet, maybe it doesn't like binaries built with stretch container.

@varosi

This comment has been minimized.

Show comment
Hide comment
@varosi

varosi Dec 19, 2015

Do you use LLVM 3.5.2+ ?

2015-12-19 2:59 GMT+02:00 Andrey Kartashov notifications@github.com:

@Chobbes https://github.com/Chobbes

file stack

stack: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically
linked (uses shared libs), for GNU/Linux 2.6.32,
BuildID[sha1]=2f5d590056ac8e25c6ef69fbe75abe4fe8e2c42a, stripped

I never actually ran it on the ARM boards, only via static
QEMU(2.4.1)/Docker(1.9.1).
I've arrived at the current version of stack incrementally. I think the
first version I've successfully built was 0.1.5.0, after that stack
upgraded itself to 0.1.10.0.
In the ~/.stack dir I find arm-linux-ghc-7.10.3/cryptonite-0.6, so,
whatever cabal-install version used that and can build some version of
stack can be used to bootstrap.

Here's my stack-0.1.10.0-Arm.gz
https://gist.github.com/andreyk0/9f5a15270c5e9c711384/raw/9eafff82b5218d1876d0b21f54044bface8db903/stack-0.1.10.0-Arm.gz
binary, uploaded as a GH gist, if that helps.

Build process crashes / hangs occasionally but I'm sure QEMU bugs are
responsible for some of that, it's a bit better with 2.4.1 than with the
earlier versions I've tried.

Generated binaries seem Ok/stable on debian stretch, cortex-a8 A10
Olinuxino Lime https://www.olimex.com/wiki/A10-OLinuXino-LIME board.
Less luck with debian jessie on A20 Lime
https://www.olimex.com/wiki/A20-OLinuXino-LIME, 1 out of 3 projects
crashes (segfault), haven't figured out yet, maybe it doesn't like binaries
built with stretch container.


Reply to this email directly or view it on GitHub
#1332 (comment)
.

varosi commented Dec 19, 2015

Do you use LLVM 3.5.2+ ?

2015-12-19 2:59 GMT+02:00 Andrey Kartashov notifications@github.com:

@Chobbes https://github.com/Chobbes

file stack

stack: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically
linked (uses shared libs), for GNU/Linux 2.6.32,
BuildID[sha1]=2f5d590056ac8e25c6ef69fbe75abe4fe8e2c42a, stripped

I never actually ran it on the ARM boards, only via static
QEMU(2.4.1)/Docker(1.9.1).
I've arrived at the current version of stack incrementally. I think the
first version I've successfully built was 0.1.5.0, after that stack
upgraded itself to 0.1.10.0.
In the ~/.stack dir I find arm-linux-ghc-7.10.3/cryptonite-0.6, so,
whatever cabal-install version used that and can build some version of
stack can be used to bootstrap.

Here's my stack-0.1.10.0-Arm.gz
https://gist.github.com/andreyk0/9f5a15270c5e9c711384/raw/9eafff82b5218d1876d0b21f54044bface8db903/stack-0.1.10.0-Arm.gz
binary, uploaded as a GH gist, if that helps.

Build process crashes / hangs occasionally but I'm sure QEMU bugs are
responsible for some of that, it's a bit better with 2.4.1 than with the
earlier versions I've tried.

Generated binaries seem Ok/stable on debian stretch, cortex-a8 A10
Olinuxino Lime https://www.olimex.com/wiki/A10-OLinuXino-LIME board.
Less luck with debian jessie on A20 Lime
https://www.olimex.com/wiki/A20-OLinuXino-LIME, 1 out of 3 projects
crashes (segfault), haven't figured out yet, maybe it doesn't like binaries
built with stretch container.


Reply to this email directly or view it on GitHub
#1332 (comment)
.

@andreyk0

This comment has been minimized.

Show comment
Hide comment
@andreyk0

andreyk0 Dec 19, 2015

@varosi yeah, LLVM 3.5.2, available from package manager in debian stretch.
For jessie you can grab a binary from LLVM downloads page, you'll need to create /usr/bin/opt-3.5, /usr/bin/llc-3.5 and /usr/bin/lli-3.5 symlinks, then it works.
My GHC is the ghc-7.10.3 ARM build from GHC downloads page, ghc-7.10.2 also worked.

@varosi yeah, LLVM 3.5.2, available from package manager in debian stretch.
For jessie you can grab a binary from LLVM downloads page, you'll need to create /usr/bin/opt-3.5, /usr/bin/llc-3.5 and /usr/bin/lli-3.5 symlinks, then it works.
My GHC is the ghc-7.10.3 ARM build from GHC downloads page, ghc-7.10.2 also worked.

@ChristopherKing42

This comment has been minimized.

Show comment
Hide comment
@blitzcode

This comment has been minimized.

Show comment
Hide comment
@blitzcode

blitzcode May 6, 2016

For what it's worth, I've written a guide on how I got a stack based project build on an Raspberry Pi, hopefully it's useful for somebody stumbling on this thread:

https://github.com/blitzcode/hue-dashboard#raspberry-pi

For what it's worth, I've written a guide on how I got a stack based project build on an Raspberry Pi, hopefully it's useful for somebody stumbling on this thread:

https://github.com/blitzcode/hue-dashboard#raspberry-pi

@varosi

This comment has been minimized.

Show comment
Hide comment
@varosi

varosi Jun 8, 2016

Great-great! Will there be ARM distros of Stack officially?

varosi commented Jun 8, 2016

Great-great! Will there be ARM distros of Stack officially?

@mgsloan

This comment has been minimized.

Show comment
Hide comment
@mgsloan

mgsloan Jun 8, 2016

Collaborator

We don't intend to have official stack support for archs that don't have official ghc binary distributions.

Collaborator

mgsloan commented Jun 8, 2016

We don't intend to have official stack support for archs that don't have official ghc binary distributions.

@andreyk0

This comment has been minimized.

Show comment
Hide comment
@andreyk0

andreyk0 Jun 8, 2016

@mgsloan is this not considered official? https://www.haskell.org/ghc/download_ghc_7_10_3#linux_armv7
Seems to be available for a while and works pretty well.

andreyk0 commented Jun 8, 2016

@mgsloan is this not considered official? https://www.haskell.org/ghc/download_ghc_7_10_3#linux_armv7
Seems to be available for a while and works pretty well.

@mgsloan

This comment has been minimized.

Show comment
Hide comment
@mgsloan

mgsloan Jun 8, 2016

Collaborator

Oh, yeah, you're right! Ok, yeah it'd be reasonable to add support in stack, then, if all the details get worked out.

Collaborator

mgsloan commented Jun 8, 2016

Oh, yeah, you're right! Ok, yeah it'd be reasonable to add support in stack, then, if all the details get worked out.

@borsboom

This comment has been minimized.

Show comment
Hide comment
@borsboom

borsboom Jun 9, 2016

Contributor

See #2103 for the issue tracking adding ARM as an official platform for Stack, which we very much hope to do. This is distinct from cross-compiling for ARM, which is more complex and will need general support in Stack for cross-compiling.

The main obstacle is a solution to the LLVM problem (like Stack also managing installation much like it manages GHC's) to have a decent user experience. Unfortunately, the current LLVM bindists don't seem to work on Scaleway's ARM VPS so I haven't made any headway on this.

Contributor

borsboom commented Jun 9, 2016

See #2103 for the issue tracking adding ARM as an official platform for Stack, which we very much hope to do. This is distinct from cross-compiling for ARM, which is more complex and will need general support in Stack for cross-compiling.

The main obstacle is a solution to the LLVM problem (like Stack also managing installation much like it manages GHC's) to have a decent user experience. Unfortunately, the current LLVM bindists don't seem to work on Scaleway's ARM VPS so I haven't made any headway on this.

@varosi

This comment has been minimized.

Show comment
Hide comment
@varosi

varosi Jun 9, 2016

@blitzcode, now it doesn't compile on ARMv7, because of: haskell-crypto/cryptonite#90

cryptonite is a dependency of Stack

varosi commented Jun 9, 2016

@blitzcode, now it doesn't compile on ARMv7, because of: haskell-crypto/cryptonite#90

cryptonite is a dependency of Stack

@blitzcode

This comment has been minimized.

Show comment
Hide comment
@blitzcode

blitzcode Jun 9, 2016

That's an unfortunate regression, hope it'll be fixed soon. Might need to change stack to pin to an older version for now. Wish I could build stack with stack for more reproducible builds ;-)

That's an unfortunate regression, hope it'll be fixed soon. Might need to change stack to pin to an older version for now. Wish I could build stack with stack for more reproducible builds ;-)

@varosi

This comment has been minimized.

Show comment
Hide comment
@varosi

varosi Jun 9, 2016

Yes, but it compiles with cryptonite-0.15 so it may be constrained.

varosi commented Jun 9, 2016

Yes, but it compiles with cryptonite-0.15 so it may be constrained.

@andreyk0

This comment has been minimized.

Show comment
Hide comment

Replied in #2103

@alexanderkjeldaas

This comment has been minimized.

Show comment
Hide comment
@alexanderkjeldaas

alexanderkjeldaas Apr 2, 2018

Contributor

Is it possible to use the cross-compilers at http://hackage.mobilehaskell.org/ and the overlay somehow with stack?

Contributor

alexanderkjeldaas commented Apr 2, 2018

Is it possible to use the cross-compilers at http://hackage.mobilehaskell.org/ and the overlay somehow with stack?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment