Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installer for Aarch64 (and ppc64le) #871

Closed
hmaarrfk opened this issue Sep 21, 2019 · 30 comments
Closed

Installer for Aarch64 (and ppc64le) #871

hmaarrfk opened this issue Sep 21, 2019 · 30 comments

Comments

@hmaarrfk
Copy link
Contributor

Now that the aarch migration is well underway, it seems like it would be to create a Miniconda installer for aarch64

Not too sure why this one is not working
Archiconda/build-tools#23

@jjhelmus
Copy link
Contributor

I'd be in favor of this. The question is where to host the installer? Perhaps a GitHub release would be reasonable with a link to the installer from the conda-forge.org.

@mariusvniekerk
Copy link
Member

Github release seems prudent.

@isuruf
Copy link
Member

isuruf commented Oct 9, 2019

@hmaarrfk, let us know what we can do to help like creating a github repo, activating a CI, etc.

@jakirkham
Copy link
Member

Thoughts @msarahan?

@hmaarrfk
Copy link
Contributor Author

hmaarrfk commented Oct 12, 2019

Jewlerconda3/construct.yaml

constructor --platform=linux-aarch64  Jewlerconda3/  
# what to call this???
name: Jewlerconda3
version: 0.1.0

channels:
  - https://conda.anaconda.org/conda-forge
  - https://conda.anaconda.org/anaconda
  # There still seems to be a dependency needed from c4aarch64
  - https://conda.anaconda.org/c4aarch64

write_condarc: True
#license_file: installer_license.txt

specs:
  - python 3.7.3
  - conda
  - pip

@mariusvniekerk
Copy link
Member

forge-conda?

@hadim
Copy link
Member

hadim commented Nov 12, 2019

An aarch64 flavor of conda provided by Anaconda or conda-forge would indeed be super nice. Any news or ETA on this? What does remain to be done? Anything I can do to help.

In the meantime, I use Archiconda which does the job well.

@hmaarrfk
Copy link
Contributor Author

somebody needs to test it on real metal hardware.

I got RPi4, but still haven't had a chance to connect it to a monitor.

@hadim
Copy link
Member

hadim commented Nov 12, 2019

I just tested the last Archiconda3 installer on an Rpi4 with Ubuntu 19.10 arm64 and it works perfectly.

I couldn't installed either update conda because of an issue with the university network here (DNS won't resolve) but I'll report once it works.

@hadim
Copy link
Member

hadim commented Nov 13, 2019

Waiting for the DNS issue to be solved I tried to build an installer with the packages I need:

name: Archiconda
version: 0.3.0

channels:
  - https://conda.anaconda.org/conda-forge
  - https://conda.anaconda.org/anaconda
  # There still seems to be a dependency needed from c4aarch64
  - https://conda.anaconda.org/c4aarch64

write_condarc: True

specs:
  - python
  - conda
  - pip
  - numpy
  - scipy
  - pandas
  - sqlalchemy
  - pyserial
  - slacker
constructor --platform=linux-aarch64 .

I built it on my host machine (Ubuntu x86_64) but when trying the installation on RPi 4 I have the following error:

$ bash Archiconda-0.3.0-Linux-aarch64.sh 

Welcome to Archiconda 0.3.0

Archiconda will now be installed into this location:
/home/ubuntu/archiconda

  - Press ENTER to confirm the location
  - Press CTRL-C to abort the installation
  - Or specify a different location below

[/home/ubuntu/archiconda] >>> 
PREFIX=/home/ubuntu/archiconda
Unpacking payload ...
Archiconda-0.3.0-Linux-aarch64.sh: line 259: /home/ubuntu/archiconda/conda.exe: cannot execute binary file: Exec format error
ERROR: could not extract tar starting at offset 000000000000011467+9231072+2

I guess it's because the installer is x86_64 only compatible. Any idea of how I could build an installer that works on arm64? Docker (but then how do you initially install constructor without conda)?

Also, I noticed a pretty important package is not available yet for arm64: matplotlib. I have open an issue about it: conda-forge/matplotlib-feedstock#221

@isuruf
Copy link
Member

isuruf commented Nov 13, 2019

Docker (but then how do you initially install constructor without conda)?

Use condaforge/linux-anvil-aarch64 docker image which can be run on x86_64 after running https://github.com/conda-forge/conda-smithy/blob/84652a0239b1ffa0660bed8374f809b2dca390f1/conda_smithy/templates/azure-pipelines-linux.yml.tmpl#L25

@isuruf
Copy link
Member

isuruf commented Nov 13, 2019

Btw, we can get rid of c4aarch64 if we built https://github.com/conda-forge/ctng-compilers-feedstock

@hmaarrfk
Copy link
Contributor Author

I just setup a repo (for my own company) that would use Travis to upload binaries to GitHub on tags.

If we have a name setup for the installer, I'm glad to setup a CI to get everything going at this stage.

We can eventually build up automated tests to test it invarious docker instances for different flavors of Linux.

@hadim
Copy link
Member

hadim commented Nov 13, 2019

So I have been able to build my own installer and everything works smoothly (except for matplotlib that needs to installed manually).

An official (or at least semi-official) installer would be nice indeed. Happy to help/test/debug if needed.

@hmaarrfk
Copy link
Contributor Author

The Archiconda one comes pretty close. I would only rename it and move it locations. Is anything (other than matplotlib) missing?

@hadim
Copy link
Member

hadim commented Nov 13, 2019

matplotlib should be available soon: conda-forge/matplotlib-feedstock#215

But you only need to build a minimum installer. People can install whatever they want after the installation.

Thanks anyway for doing that!

@hmaarrfk
Copy link
Contributor Author

Right, so archiconda is that minimal installer with ths idea that people will update and install whatever they want.

We could call it Bananaconda

@hmaarrfk
Copy link
Contributor Author

But honestly, I like archi conda. So if you create a repo for me, I'll just run a chron or something on it that would create weekly releases with the archi conda constructor above

@hadim
Copy link
Member

hadim commented Nov 13, 2019

Archiconda looks fine to me but I don't really have a strong feeling about it.

@hadim
Copy link
Member

hadim commented Nov 13, 2019

Some feedback with the Rpi 4 Ubuntu 19.10 (arm64).

I installed archiconda (the old version) and was able to update all the packages conda update --all.

Then I installed matplotlib, numpy, scipy and pandas. Everything runs smoothly and works without a glitch. Honestly, I wasn't excepting those packages to work out-of-the-box like that.

Great job guys as always.

@isuruf
Copy link
Member

isuruf commented Nov 13, 2019

How about miniforge, so that the name is generic? It was @jakirkham's idea a couple of years ago.

@hadim
Copy link
Member

hadim commented Nov 13, 2019

I like it. Would miniforge be only for arm64 and ppc64le? If it's an installer released by conda-forge, why not providing also other flavors such as win64, linux_x86_64 and so forth?

@hmaarrfk
Copy link
Contributor Author

Miniforge is good. Muscle memory with mini tab Carries over

@hadim
Copy link
Member

hadim commented Nov 13, 2019

As for what packages to include, I would stick to what's in Miniconda. Here is a list on a Linux system from a freshly installed Miniconda:

asn1crypto==1.2.0
certifi==2019.9.11
cffi==1.13.0
chardet==3.0.4
conda==4.7.12
conda-package-handling==1.6.0
cryptography==2.8
idna==2.8
pycosat==0.6.3
pycparser==2.19
pyOpenSSL==19.0.0
PySocks==1.7.1
requests==2.22.0
ruamel-yaml==0.15.46
six==1.12.0
tqdm==4.36.1
urllib3==1.24.2

(I couldn't find the construct.yaml file for miniconda)

@hmaarrfk
Copy link
Contributor Author

You were able to install other packages with the archiconda installer. I think that is all that will be in there.

Pip, python, conda

@isuruf
Copy link
Member

isuruf commented Nov 14, 2019

@hmaarrfk, https://github.com/conda-forge/miniforge

@isuruf
Copy link
Member

isuruf commented Nov 14, 2019

@hmaarrfk, I've given you permissions, so that you can enable CI, make releases etc.

@CJ-Wright
Copy link
Member

We should make a team for that.

@isuruf
Copy link
Member

isuruf commented Nov 14, 2019

#922

@jakirkham
Copy link
Member

Good to close?

@scopatz scopatz closed this as completed Dec 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

8 participants