-
-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can Binary Releases be Done for aarch64 and armv7h #5450
Comments
stormdragon2976 <notifications@github.com> writes:
Aarch64 and Armv7h machines are becoming more and more common place now days. There are Windows, Mac, and Linux binaries, but the Linux binaries do not cover these architectures. Please add binary releases for them.
The latest ghc bindist I've found for arm architecture
is ghc 7.10; debian has 7.8.x; and recent versions of
pandoc require ghc >= 8.
I'm sure there's some way to do this. Hopefully
someone who needs this can put in the time to figuring
out how, and report back here. We could then consider
making it part of the release process.
|
AFAIK, both @vmchale and @2i3r managed to cross-compile pandoc. Maybe they have any insights on how we could support these platforms? See also hslua/hslua#73. |
I have my |
On my tries direct compilation of |
@2i3r do you think it would be hard to set this up on CircleCI (or similar), which we currently use for the pandoc Linux builds? |
I don't have any experience with |
@2i3r, how much time such build would normally take? One concern to use CI to build is the time limit. In the past people has tried using QEMU in CI but it becomes too slow to be manageable. Otherwise, especially when a docker can be set up, it should be easy to do it in CircleCI. |
I'm also interested in this activity. Drone.io have builders for AArch64 and ArmV7. If that is of interest, I could begin to work on this with a colleague. |
@rhenwood-arm - since pandoc is open source, nothing stops you from building it and making the binaries available. I think that would be appreciated! Whether we want to provide officially endorsed ARM binaries here (on the GitHub releases page) is another question. For now, I'd be happy to have a third-party source for them. |
My experience on directly compiling on ARM: For armv7l, I'm trying to provide it as an unofficial effort here: https://github.com/ickc/pandoc-arm/releases. Probably will fall in pandoc-extras later. (By the way, it took Raspberry Pi 4 about 2.8 GiB of RAM and 3 hours to compile.) For aarch64, I have trouble getting stack/cabal compiled. If I'm not mistaken it is not supported. If someone can give me the exact command of cross compiling pandoc targeting the 2 platforms, I can try to build them and test it on those arch. In the long run, if it doesn't take too much time to compile with CI, and if there's a way to run tests on the target arch (current tests are run using cabal/stack in the compiling stage), then may be it can be provided as an official release. By the way, am I right that for the current released binaries, the Windows' come from CI, but he macOS/Linux builds come from @jgm directly? I think this might be true in the past but not sure any more. |
Correct, but this may soon change. I've been working on getting releases built for all three architectures with GitHub Actions CI. See the rc/test branch. |
@stormdragon2976 if you want binaries, I have some here: https://github.com/vmchale/ghc-cross/releases/tag/Binaries I also have linux -> linux cross-compilers for anyone who wants to try on their own: https://github.com/vmchale/ghc-cross/releases/tag/8.6.5 (both armv7 and aarch64) |
Howdy Peter,
Thanks for this, it is very awesome. Could you also provide armv7h binaries? I don't currently have an X86 to try the cross compiler.
One small problem that happened when I tried to convert a file:
Could not find data file /home/vanessa/.cabal/share/aarch64-linux-ghc-8.6.5/pandoc-2.7.3/data/abbreviations
Thanks,
Storm
…On Sat, Nov 30, 2019 at 08:48:57AM -0800, Peter hyman wrote:
@stormdragon2976 if you want binaries, I have some here: https://github.com/vmchale/ghc-cross/releases/tag/Binaries
I also have linux -> linux cross-compilers for anyone who wants to try on their own: https://github.com/vmchale/ghc-cross/releases/tag/8.6.5 (both armv7 and aarch64)
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#5450 (comment)
--
⛈🐲
Accessible low cost computers for everyone! Get your slice of the Pi: https://asliceofthepi.com
My youtube channel: https://www.youtube.com/channel/UCdaTl5vl404OaRxAJPvb8Ew
get my public PGP key: gpg --keyserver wwwkeys.pgp.net --recv-key 43DDC193
Follow me on GNU Social: https://social.wolfe.casa/storm
"Have you ever been alone at night, thought you heard footsteps behind, and turned around and no one's there? And as you quicken up your pace you'll find it hard to look again because you're sure that someone's there."
Iron Maiden - Fear of the Dark
|
Set the |
Howdy Peter,
I just downloaded the aarch64 binary and tried it on the N2. That's when I got the error.
Thanks,
Storm
…On Sat, Nov 30, 2019 at 10:27:57AM -0800, Peter hyman wrote:
> One small problem that happened when I tried to convert a file:
>
> Could not find data file /home/vanessa/.cabal/share/aarch64-linux-ghc-8.6.5/pandoc-2.7.3/data/abbreviations
Set the `embed_data_files` flag when you compile; this will ensure that the data files are included in the binary.
--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#5450 (comment)
--
⛈🐲
Accessible low cost computers for everyone! Get your slice of the Pi: https://asliceofthepi.com
My youtube channel: https://www.youtube.com/channel/UCdaTl5vl404OaRxAJPvb8Ew
get my public PGP key: gpg --keyserver wwwkeys.pgp.net --recv-key 43DDC193
Follow me on GNU Social: https://social.wolfe.casa/storm
"I don't wanna face my fears! I'm afraid of 'em!"
SpongeBob SquarePants
|
I work on Chromebrew and we'd like to have ARMv7 binaries as well since Google chose to ship all ARMv8 Chromebooks with 32-bit user ABIs due to only having 4GB of memory. |
Any progress for this issue? I try to build pandoc-2.9.2 for raspberry pi 4b, and got error
|
I have exactly the same issue on both aarch64 and armv7l on Raspberry Pi 4 4GB model with pandoc 2.9+. 4GB was just enough to compile pandoc briefly after Raspberry Pi 4 launched but then pandoc's need soon exceeds that. If you can't wait there's a build in https://github.com/ickc/pandoc-arm/releases, with the last version compiled successfully on RPi4, I think 2.8. I should emphasize that you can always add more swap. Long time ago I think I successfully compiled pandoc with 1GB RAM (forgot if it was a RPi original or another low mem. laptop.) Just add a ton of swap and it will work, albeit very slowly and your swap device's life shortened if it is flash (e.g. microSD.) I was about to experiment with zram if I have time and see if it helps. (As other has noted, the proper way to do it is to cross-compile and do it in CI, and distribute it officially. But until that happens...) |
@ickc the version is 2.7.3. It has problem on epub toc. So I am looking for 2.9+ |
Then build your own, add a ton of swap and wait for a few days. |
I successfully compiled 2.9.2 . see https://github.com/arm4rpi/pandoc-arm/releases |
How? Just ton of swap? Are you going to provide that continuously in the future? If so there's no need for 2 of us doing the same thing. Also, if you decided to continually provided that, you may want to think about security implication. e.g. as outlined in my repo I found that it is reproducible build so providing checksum should be suffices (may be this is the reason we still need to have 2 independent builds just for others to double check.) As said above if people working on cross compiling has a solution then that would be preferred as doing it in the cloud in the open means less trust on us is needed. But before that it may be something you and I can do in the interim. |
@ickc I am trying to build using github actions. But got some errors, see https://github.com/arm4rpi/pandoc-arm/runs/483167534
do you know how to solve it? |
Sorry no experience on GitHub Action. What was the earlier errors? Do you see something fail to compile? Could you bring me up to speed about the GitHub Action? e.g. do they provide armv7l and aarch64 or are you cross-compiling? In the past experience when dealing with CI, is the "debug loop" can be troublesome. One way people often try to solve situation like this is to use the docker image of the CI instance itself (e.g. Travis CI) and debug it interactively. |
Seem no armv7l and aarch64 provided, I am using for debug. run
docker image |
Now I am trying to build for cabal resolving dependencies problemLimited by execution time(6 hours), it is not possible to build successfull. So I want to pre build dependencies( see https://github.com/arm4rpi/pandoc-deps), but the problem is
armv7l always out of memoryI had add huge swap.
alway out of memory when
|
I think the only solution for the out-of-memory error is to further break up Not sure about the dependency problems. The differences in hashes are probably due to different flags being used. Maybe try using more specific constraint |
Next steps on my part:
Then, the digests for all Docker images and multi-arch manifest are publicly available in the GitLab CI/CD Pipeline build logs. |
Sorry: I am pretty ignorant about docker. |
@benz0li I've built multi-arch Docker images before. Unless the setup has changed, they run on an x86_64 server and build the other architectures using a static QEMU layer to emulate them. The upside is that you can build a multi-arch image for anything QEMU can emulate - arm64, PowerPC, IBM Z series, etc. So they'll work for anything where your base image - Ubuntu or Debian - has the binary toolchain to build I vaguely recall trying it (for just arm64) on my workstation last fall when I got my new one. And I got it to run on Travis CI but it couldn't complete a few steps within their 50 minute time limit. |
@jgm My fault! The manifest for tag No, you don't have to tell which arch to use. If the manifest consists of multiple os/arch images, Docker pulls the image for the arch Docker is running on. |
I ran out of memory on an 8 GB instance -- not even on one of the memory-hungry modules. I'm a good way through a build now on a 16 GB t4g.xlarge AWS instance. |
@jgm I've never tried it on 8 GB - just 4, which fails, and 16, which works. My recollection is that the crashes are in the back end, and that I tried both the gcc and LLVM back ends. I don't remember if I ever got the native GHC back end to work or not, or which version of GHC I used - everything works in 16 GB with the GHC 8.0 that ships in Ubuntu 18.04 so I stopped experimenting. |
I can't explain the enormous memory needs while building pandoc on ARM. Tried on a t4g.xlarge (16 GB RAM) myself... and failed. Just started a new build with a 16 GB swapfile in addition. I used some heavy machinery, t4g.2xlarge (8 vCPUS, 32 GB RAM + 16 GB swapfile), while building pandoc-2.11.4-1-arm64.deb as GHC Docker Images were built in parallel. |
Same experience here on t4g.xlarge. Trying now with t4g.2xlarge! It may mean that creating the release package on arm costs $1-2. We can afford that. But it's strange, the difference between ARM and non-ARM. |
Interesting - my AGX Xavier has 16 GB of RAM, Ubuntu18.04 LTS for All I have to do to build Pandoc is upgrade My recollection from watching these things go by with One other possibility for Ubuntu: there is a PPA on Launchpad with Haskell backports including Pandoc: https://launchpad.net/~savoury1/+archive/ubuntu/haskell-build/. It does not appear to be set up to build for |
I've been able to build with the 2xl instance. I'm adding a script to automate this process, and I'll try to include an arm64 binary in the next release. (We can then get statistics to see how much use it gets.) |
I have been able to build it with a t4g.xlarge AWS EC2 instance:
Build time: 5 hours. |
The issue with the GitLab CI/CD Pipeline is resolved, too. I am going to rebuild all images and multi-arch manifests for I will continue to maintain these images and keep them publicly available. @jgm Let me know if you need a version in-between 8.10.1 and 8.10.4. |
Hmm ... maybe I should move my Ubuntu build chain to GitLab and do my R and RStudio builds there on top of your Pandoc build. That would check off a huge box for me; I could go with R from the Ubuntu source repos instead of from upstream. I only need the Jetson base / NVIDIA tools for the last step in my processing. |
@jgm pandoc-2.12-linux-arm64.tar.gz seems to be broken: $ tar -xfz pandoc-2.12-linux-arm64.tar.gz
gzip: stdin: unexpected end of file
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now |
Oh, weird! I've manually unpacked the deb and created a tarball from it, replacing the one on releases. I think I can guess what happened. My build script waits until the .tar.gz appears in artifacts, and then downloads it. Perhaps it downloaded it while it was incomplete? IN any case the build script needs some improvement. |
Upacking works fine now. But I'm not sure, if this is the intended structure and content for the arm64 release:
Compared to the amd64 release: $ tree pandoc-2.12-amd64
pandoc-2.12-amd64
├── bin
│ └── pandoc
└── share
└── man
└── man1
└── pandoc.1.gz
4 directories, 2 files |
No, that's not right. Let me fix it manually; I don't want to wait another 3 hours for a build to complete! |
@benz0li Good news. |
My experiments didn't gain much from multiple cores. A bunch of small-ish libraries get built at the beginning, but once they're done, it's single-threaded for the large ones. |
@jgm More good news. A Docker settings used: ℹ️ Memory increased from 4 GB to 16 GB; Swap increased from 1 GB to 4 GB; CPUs left at 4 👉 the Mac mini M1 has 4 performance cores. Bearing in mind that Docker Desktop for Apple M1 runs in a virtual machine, which uses the new Virtualization Framework, the performance is outstanding. |
Nice, so this produces a native M1 build? (See #6960) |
No. Docker Desktop for Apple M1 is effectively
|
@benz0li Is the Apple operating system shim as light-weight as the Windows version of Docker Desktop, which uses WSL 2? |
Aarch64 and Armv7h machines are becoming more and more common place now days. There are Windows, Mac, and Linux binaries, but the Linux binaries do not cover these architectures. Please add binary releases for them.
The text was updated successfully, but these errors were encountered: