Support arm64 and powerpc64el

[elijahr]

With this changeset choosenim can now be a universal method for installing nim on all platforms that Travis supports (that nim also supports). Build-times will vary since official nim binaries only exist for some platforms/channels.

My initial PR #213 simply tried to run the travis config on other architectures, but it turned out more was needed.

I worked with @genotrance to add arm64 and powerpc64el support in nimarchive, via nimarchive 0.5.3, so I've bumped the requirement here.

I forked genotrance's Travis script to support Travis's arm64 and ppc64le (aka powerpc64el) archs. I also updated the example .travis.yml there to show a test matrix with these architectures. @genotrance has integrated my travis.sh changes upstream. The script is now maintained @ https://github.com/genotrance/nim-travis

Nim has nightly binaries for linux_arm64 and macosx_x64, so choosenim devel will now install those.

choosenim update self will build choosenim when binaries are not available for the architecture. This makes adding new architectures in the future easier, since the can update self test will now pass even when choosenim binaries don't yet exist.

Next steps:

• Once this is merged and released, choosenim binaries will be available for these architectures, so the new travis.sh script can be amended to use choosenim on all platforms.
• If/when official nim stable binaries for other architectures are made available (linux_arm64 and macosx_x64 seem likely since there are nightly builds, and maybe someday macosx_arm64), the changeset to choosenim to support them should be minimal. Probably just a few lines in downloadImpl. And the Travis workarounds here for non-amd64 archs can likely be removed.

[elijahr]

I removed this because it is isn't necessary. On macs, uname -m is x86_64, so the x86_64 -> amd64 transformation will occur on L152.

[elijahr]

I think the build failures might have to do with caching, or lack thereof. Compilation could be taking a long time. I have a green build with this exact changeset here: https://travis-ci.org/github/elijahr/choosenim/builds/721557412.

I'm trying a build with travis_wait here: https://travis-ci.org/github/elijahr/choosenim/builds/721565811. If that succeeds I'll pull the change into this PR.

[elijahr]

The build is green and this is ready for review. I had to implement some workarounds so Travis wouldn't timeout for the new architectures. I tried to make these workarounds as unobtrusive and clean as possible, and included some detailed comments in travis.yml.

Notably, the use of a persistent ccache helps significantly with building nim from source. In the best case for linux_amd64, I was seeing the full test suite pass in <5m (down from ~15-25m on master). linux_arm64 builds still take a while, but no longer timeout. Windows/macOS didn't seem to benefit much from using ccache. From what I understand, ccache is extraneous when building nim code, since nim already caches object files, but for building nim itself on Linux, ccache is a boon.

I've updated the PR description to reflect the current state of this changeset.

[genotrance]

Change url maybe.

[elijahr]

Good idea, changed in e81e2e1

[genotrance]

Why config.nims vs. choosenim.nims?

[elijahr]

Actually, I can revert this. In an earlier iteration of the code I was defining some stuff here for use in both choosenim and tester, but that's not needed anymore. Reverted in 8fd54dd

[elijahr]

So with my last batch of changes, the ppc64le build failed due to timing out after 10m of no output. Since the builds are still failing intermittently and I've already sunk a bunch of time into getting Travis to play nice, I'm just not comfortable leaning on Travis for this. I'm going to explore some other options - perhaps github workflows. From what I understand nim nightlies are built with that now. Any suggestions?

[elijahr]

Hello, just an update on this: I have been working on a fork of @genotrance's nim-travis project that also includes a Github Actions config for building Nim projects across a matrix of architectures (link).

Closing this pull request; I plan to submit a (third, and hopefully final!) PR in the near future which would move choosenim CI from Travis to Github Actions.

[dom96]

Hey @elijahr, any news on this? :)

[elijahr]

@dom96 Hello! I've been working here and there on it as I have time, it's still my primary focus outside of work. I expect that I'll have another pull request within the next 2-3 weeks.