Make Rancher ARM compatible #4703

Open
deniseschannon opened this Issue May 5, 2016 · 38 comments

Projects

None yet
@deniseschannon
Member
deniseschannon commented May 5, 2016 edited

Main components

Additional Features Added

@imikushin imikushin changed the title from Make Rancher on ARM compatible to Make Rancher ARM compatible May 5, 2016
@faddat
faddat commented May 6, 2016 edited

That's one awe-inspiring checklist :).

How will you be going about porting each component? Is it just a matter of getting an arm build for each to work? That, I can help with.

@nunofgs
nunofgs commented May 13, 2016

Can't wait for this ๐Ÿ˜„

@teleki
teleki commented May 15, 2016

Just assembled a 4 unit ODROID C2 cluster and can't wait to get Rancher going on it.

@imikushin
Member

@teleki you can! Just build rancher/server and rancher/agent for ARM from #4704, rancher/agent-instance from rancher/agent-instance#18

@teleki
teleki commented May 28, 2016

@imikushin Thank you for the advice. I tried building rancher/server, but I am getting errors. Here is the output of running as docker ./build-image.arm.sh in the rancher/server folder:

++ dirname ./build-image.arm.sh
+ cd .
+ . ../.docker-env.arm
./build-image.arm.sh: line 6: ../.docker-env.arm: No such file or directory
+ echo 'Need to source .docker-env.arm to access Docker on ARM'
Need to source .docker-env.arm to access Docker on ARM
+ '[' '!' -e target/arm/.done ']'
+ mkdir -p target/arm
++ cat /dev/urandom
++ env LC_CTYPE=C tr -cd a-f0-9
++ head -c 16
+ S6_BUILDER=0484087bb24a011b
+ trap 'docker rm -fv 0484087bb24a011b 2>/dev/null || :' EXIT
+ docker run --name=0484087bb24a011b rancher/s6-builder:v2.2.4.3_arm /opt/build.sh
Unable to find image 'rancher/s6-builder:v2.2.4.3_arm' locally
Pulling repository docker.io/rancher/s6-builder
docker: Tag v2.2.4.3_arm not found in repository docker.io/rancher/s6-builder.
See 'docker run --help'.
+ docker rm -fv 0484087bb24a011b
+ :

Here is the output of git status:

On branch pr/4704
Your branch is up-to-date with 'origin/pr/4704'.
nothing to commit, working directory clean

docker version

Client:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.6.1
 Git commit:   20f81dd
 Built:        Wed, 20 Apr 2016 14:19:16 -0700
 OS/Arch:      linux/arm64

Server:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.6.1
 Git commit:   20f81dd
 Built:        Wed, 20 Apr 2016 14:19:16 -0700
 OS/Arch:      linux/arm64

Here is the output of docker images:

REPOSITORY                TAG                 IMAGE ID            CREATED             SIZE
<none>                    <none>              a4ed6a02234b        About an hour ago   122 MB
ubuntu                    16.04               2fa927b5cdd3        27 hours ago        122 MB
armv7/armhf-ubuntu_core   latest              2c357551076b        7 months ago        109.5 MB

The Linux version message is:

Welcome to Ubuntu 16.04 LTS (GNU/Linux 3.14.29-56 aarch64)

This is my first attempt to venture into this type of stuff, so I am rather green. I wonder if I am missing some additional configuration settings... Your help is appreciated.

@teleki
teleki commented May 30, 2016

@imikushin: update: Successfully built rancher/agent and rancher/agent-instance from the respective PRs. However, rancher/server still has the same error as mentioned above.

@alphawolf1988
alphawolf1988 commented May 31, 2016 edited

@imikushin Succesfully built rancher/agent and rancher/server.

image

@teleki You require to:

  1. git clone https://github.com/imikushin/s6-builder
  2. ./build.arm.sh

Then you should have the required s6-builder docker image :) and everything should build just fine.
I can run the rancher/server on my Raspberry Pi 3, but it takes hours to start. One cpu is up to 100% all the time (java process), while starting.

@imikushin Could it be, that the process still uses OpenJDK instead of OracleJDK?

@teleki
teleki commented Jun 1, 2016

@imikushin Thank you for the help; it is much appreciated. I have successfully built s6-builder.

However, now I am getting a different error as I am trying to build rancher/server. The error says that there is no "/usr/bin/java" which is probably true, since based on the logs, it seems that Java is in "/usr/lib/jvm". Here are the relevant snippets of the output:

....
Connecting to download.oracle.com (download.oracle.com)|204.237.184.213|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 78321279 (75M) [application/x-gzip]
Saving to: 'jdk-8u91-linux-arm64-vfp-hflt.tar.gz'
...
2016-06-01 02:48:01 (1.51 MB/s) - 'jdk-8u91-linux-arm64-vfp-hflt.tar.gz' saved [78321279/78321279]

Download done.
Removing outdated cached downloads...
update-alternatives: using /usr/lib/jvm/java-8-oracle/jre/bin/java to provide /usr/bin/java (java) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/jre/bin/jjs to provide /usr/bin/jjs (jjs) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/jre/bin/keytool to provide /usr/bin/keytool (keytool) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/jre/bin/orbd to provide /usr/bin/orbd (orbd) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/jre/bin/pack200 to provide /usr/bin/pack200 (pack200) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/jre/bin/policytool to provide /usr/bin/policytool (policytool) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/jre/bin/rmid to provide /usr/bin/rmid (rmid) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/jre/bin/rmiregistry to provide /usr/bin/rmiregistry (rmiregistry) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/jre/bin/servertool to provide /usr/bin/servertool (servertool) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/jre/bin/tnameserv to provide /usr/bin/tnameserv (tnameserv) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/jre/bin/unpack200 to provide /usr/bin/unpack200 (unpack200) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/jre/lib/jexec to provide /usr/bin/jexec (jexec) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/appletviewer to provide /usr/bin/appletviewer (appletviewer) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/extcheck to provide /usr/bin/extcheck (extcheck) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/idlj to provide /usr/bin/idlj (idlj) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/jar to provide /usr/bin/jar (jar) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/jarsigner to provide /usr/bin/jarsigner (jarsigner) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/javac to provide /usr/bin/javac (javac) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/javadoc to provide /usr/bin/javadoc (javadoc) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/javah to provide /usr/bin/javah (javah) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/javap to provide /usr/bin/javap (javap) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/jcmd to provide /usr/bin/jcmd (jcmd) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/jconsole to provide /usr/bin/jconsole (jconsole) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/jdb to provide /usr/bin/jdb (jdb) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/jdeps to provide /usr/bin/jdeps (jdeps) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/jhat to provide /usr/bin/jhat (jhat) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/jinfo to provide /usr/bin/jinfo (jinfo) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/jmap to provide /usr/bin/jmap (jmap) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/jps to provide /usr/bin/jps (jps) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/jrunscript to provide /usr/bin/jrunscript (jrunscript) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/jsadebugd to provide /usr/bin/jsadebugd (jsadebugd) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/jstack to provide /usr/bin/jstack (jstack) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/jstat to provide /usr/bin/jstat (jstat) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/jstatd to provide /usr/bin/jstatd (jstatd) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/native2ascii to provide /usr/bin/native2ascii (native2ascii) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/rmic to provide /usr/bin/rmic (rmic) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/schemagen to provide /usr/bin/schemagen (schemagen) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/serialver to provide /usr/bin/serialver (serialver) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/wsgen to provide /usr/bin/wsgen (wsgen) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/wsimport to provide /usr/bin/wsimport (wsimport) in auto mode
update-alternatives: using /usr/lib/jvm/java-8-oracle/bin/xjc to provide /usr/bin/xjc (xjc) in auto mode
Oracle JDK 8 installed

...
 ---> 7dd5b4837334
Removing intermediate container d1461dd50c46
Step 33 : RUN cd / && mkdir -p $CATTLE_HOME && /usr/share/cattle/cattle.sh extract
 ---> Running in 437df28b035c
/usr/share/cattle/cattle.sh: line 197: /usr/bin/java: No such file or directory
The command '/bin/sh -c cd / && mkdir -p $CATTLE_HOME && /usr/share/cattle/cattle.sh extract' returned a non-zero code: 1
+ docker rm -fv bd1ba640d63ce3d8
bd1ba640d63ce3d8

Apparently somehow java is not in the container. However, if I look at the log is seems to show that Oracle Java was installed.

Any ideas where I'm going wrong?

@alphawolf1988

@teleki Could you show us your current used server Dockerfile.arm and are you using the build-image.arm.sh for building?

@teleki
teleki commented Jun 1, 2016

@alphawolf1988 The Dockerfile.arm is the one I got from PR 4704 referenced a few messages above by @imikushin. I only have added one file, ".docker-env.arm", because it was not in the original code, so I got that from here: https://github.com/rancher/os-images/blob/master/.docker-env.arm.sample but changed the IP to my machine's IP.

Here is the my rancher/server/Dockerfile.arm:

FROM armhf/ubuntu:16.04

ENV TERM xterm
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends openjdk-8-jre-headless \
                    curl \
                    mysql-server \
                    xz-utils \
                    keychain \
                    unzip \
                    openssh-client \
                    iptables \
                    iproute2 \
                    git \
                    redis-server \
                    zookeeper \
                    spiped
RUN mkdir -p /var/run/mysqld && chmod -R 777 /var/run/mysqld

RUN apt-get install -y --no-install-recommends software-properties-common
RUN add-apt-repository -y ppa:webupd8team/java && apt-get update
RUN echo 'yes' | DEBIAN_FRONTEND=Teletype apt-get install -y --no-install-recommends oracle-java8-installer

ADD https://github.com/imikushin/giddyup/releases/download/v0.10.1_arm/giddyup /usr/bin/
ADD https://github.com/imikushin/cluster-manager/releases/download/v0.1.4_arm/cluster-manager /usr/bin/
RUN chmod +x /usr/bin/giddyup /usr/bin/cluster-manager
COPY bin/* /usr/bin/

ENV CATTLE_HOME /var/lib/cattle
ENV DEFAULT_CATTLE_API_UI_INDEX //releases.rancher.com/ui/1.0.5
ENV CATTLE_DB_CATTLE_DATABASE mysql
ENV CATTLE_USE_LOCAL_ARTIFACTS true
ENV no_proxy localhost,127.0.0.1,localaddress,.localdomain.com
ADD artifacts /usr/share/cattle

ADD service /service
ENV S6_SERVICE_DIR /service

COPY target/arm/*static.tar.gz /s6-statics/

EXPOSE 8080
ENV CATTLE_HOST_API_PROXY_MODE embedded
ENV CATTLE_RANCHER_SERVER_IMAGE v1.0.1_arm
ENV CATTLE_RANCHER_COMPOSE_VERSION v0.7.4
ENV DEFAULT_CATTLE_RANCHER_COMPOSE_LINUX_URL   https://releases.rancher.com/compose/${CATTLE_RANCHER_COMPOSE_VERSION}/rancher-compose-linux-amd64-${CATTLE_RANCHER_COMPOSE_VERSION}.tar.gz
ENV DEFAULT_CATTLE_RANCHER_COMPOSE_DARWIN_URL  https://releases.rancher.com/compose/${CATTLE_RANCHER_COMPOSE_VERSION}/rancher-compose-darwin-amd64-${CATTLE_RANCHER_COMPOSE_VERSION}.tar.gz
ENV DEFAULT_CATTLE_RANCHER_COMPOSE_WINDOWS_URL https://releases.rancher.com/compose/${CATTLE_RANCHER_COMPOSE_VERSION}/rancher-compose-windows-386-${CATTLE_RANCHER_COMPOSE_VERSION}.zip
ENV DEFAULT_CATTLE_CATALOG_URL="library=https://github.com/rancher/rancher-catalog.git,community=https://github.com/rancher/community-catalog.git"

EXPOSE 3306
ENV CATTLE_CATTLE_VERSION v0.159.7-arm-4
ADD https://github.com/imikushin/cattle/releases/download/${CATTLE_CATTLE_VERSION}/cattle.jar /usr/share/cattle/

RUN cd /usr && for i in $(ls /s6-statics/*static.tar.gz);do tar -zxvf $i;done && rm -rf /s6-statics/*static.tar.gz
RUN cd / && mkdir -p $CATTLE_HOME && /usr/share/cattle/cattle.sh extract
RUN curl -sLf https:${DEFAULT_CATTLE_API_UI_INDEX}.tar.gz | tar xvzf - -C /usr/share/cattle/war --strip-components=1
RUN /usr/share/cattle/install_cattle_binaries && \
    cd $CATTLE_HOME && export IFS="," && \
    for i in $DEFAULT_CATTLE_CATALOG_URL; do rancher-catalog-service -validate -catalogUrl=$i;done

VOLUME /var/lib/mysql /var/log/mysql /var/lib/cattle

ENV DEFAULT_CATTLE_MACHINE_EXECUTE true
ENV DEFAULT_CATTLE_COMPOSE_EXECUTOR_EXECUTE true
ENV DEFAULT_CATTLE_CATALOG_EXECUTE true

CMD ["/usr/bin/s6-svscan", "/service"]

Thank you!

@alphawolf1988
alphawolf1988 commented Jun 1, 2016 edited

@teleki please try:

FROM armhf/ubuntu:16.04

ENV TERM xterm

RUN apt-get update && apt-get install -y --no-install-recommends software-properties-common
RUN add-apt-repository -y ppa:webupd8team/java && apt-get update
RUN echo 'yes' | DEBIAN_FRONTEND=Teletype apt-get install -y --no-install-recommends oracle-java8-installer

RUN java -version

RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
                    curl \
                    mysql-server \
                    xz-utils \
                    keychain \
                    unzip \
                    openssh-client \
                    iptables \
                    iproute2 \
                    git \
                    redis-server \
                    zookeeper \
                    spiped
RUN mkdir -p /var/run/mysqld && chmod -R 777 /var/run/mysqld

ADD https://github.com/imikushin/giddyup/releases/download/v0.10.1_arm/giddyup /usr/bin/
ADD https://github.com/imikushin/cluster-manager/releases/download/v0.1.4_arm/cluster-manager /usr/bin/
RUN chmod +x /usr/bin/giddyup /usr/bin/cluster-manager
COPY bin/* /usr/bin/

ENV CATTLE_HOME /var/lib/cattle
ENV DEFAULT_CATTLE_API_UI_INDEX //releases.rancher.com/ui/1.0.5
ENV CATTLE_DB_CATTLE_DATABASE mysql
ENV CATTLE_USE_LOCAL_ARTIFACTS true
ENV no_proxy localhost,127.0.0.1,localaddress,.localdomain.com
ADD artifacts /usr/share/cattle

ADD service /service
ENV S6_SERVICE_DIR /service

COPY target/arm/*static.tar.gz /s6-statics/

EXPOSE 8080
ENV CATTLE_HOST_API_PROXY_MODE embedded
ENV CATTLE_RANCHER_SERVER_IMAGE v1.0.1_arm
ENV CATTLE_RANCHER_COMPOSE_VERSION v0.7.4
ENV DEFAULT_CATTLE_RANCHER_COMPOSE_LINUX_URL   https://releases.rancher.com/compose/${CATTLE_RANCHER_COMPOSE_VERSION}/rancher-compose-linux-amd64-${CATTLE_RANCHER_COMPOSE_VERSION}.tar.gz
ENV DEFAULT_CATTLE_RANCHER_COMPOSE_DARWIN_URL  https://releases.rancher.com/compose/${CATTLE_RANCHER_COMPOSE_VERSION}/rancher-compose-darwin-amd64-${CATTLE_RANCHER_COMPOSE_VERSION}.tar.gz
ENV DEFAULT_CATTLE_RANCHER_COMPOSE_WINDOWS_URL https://releases.rancher.com/compose/${CATTLE_RANCHER_COMPOSE_VERSION}/rancher-compose-windows-386-${CATTLE_RANCHER_COMPOSE_VERSION}.zip
ENV DEFAULT_CATTLE_CATALOG_URL="library=https://github.com/rancher/rancher-catalog.git,community=https://github.com/rancher/community-catalog.git"

EXPOSE 3306
ENV CATTLE_CATTLE_VERSION v0.159.7-arm-4
ADD https://github.com/imikushin/cattle/releases/download/${CATTLE_CATTLE_VERSION}/cattle.jar /usr/share/cattle/

RUN cd /usr && for i in $(ls /s6-statics/*static.tar.gz);do tar -zxvf $i;done && rm -rf /s6-statics/*static.tar.gz
RUN cd / && mkdir -p $CATTLE_HOME && /usr/share/cattle/cattle.sh extract
RUN curl -sLf https:${DEFAULT_CATTLE_API_UI_INDEX}.tar.gz | tar xvzf - -C /usr/share/cattle/war --strip-components=1
RUN /usr/share/cattle/install_cattle_binaries && \
    cd $CATTLE_HOME && export IFS="," && \
    for i in $DEFAULT_CATTLE_CATALOG_URL; do rancher-catalog-service -validate -catalogUrl=$i;done

VOLUME /var/lib/mysql /var/log/mysql /var/lib/cattle

ENV DEFAULT_CATTLE_MACHINE_EXECUTE true
ENV DEFAULT_CATTLE_COMPOSE_EXECUTOR_EXECUTE true
ENV DEFAULT_CATTLE_CATALOG_EXECUTE true

CMD ["/usr/bin/s6-svscan", "/service"]
@alphawolf1988

Successfully registered agent :)
rancher_rpi2

@teleki
teleki commented Jun 2, 2016

@alphawolf1988 Thank you for the help. Still no love. Here is what I get when I run the build-image.arm.sh:

...
Oracle JDK 8 installed
 ---> 2bbfa3297100
Removing intermediate container 4784336319ff
Step 6 : RUN java -version
 ---> Running in b2a6a91defee
/bin/sh: 1: java: not found
The command '/bin/sh -c java -version' returned a non-zero code: 127

As I was Googling around, some messages were pointing me to think that the container may have a mixture of 32-bit and 64-bit libraries somehow. However, I am unable to assess if this is true or not.

I logged into the container:

docker run -ti 2bbfa3297100 /bin/bash
root@7c150d68f7f9:/# java -version
bash: /usr/bin/java: No such file or directory

Then I navigated to the Java directory. Here is what I got there:

root@7c150d68f7f9:/# cd /usr/lib/jvm/java-8-oracle/bin/
root@7c150d68f7f9:/usr/lib/jvm/java-8-oracle/bin# ls -la java*
-rwxr-xr-x 1 root root 7791 Jun  2 02:27 java
-rwxr-xr-x 1 root root 1809 Jun  2 02:27 java-rmi.cgi
-rwxr-xr-x 1 root root 8006 Jun  2 02:27 javac
-rwxr-xr-x 1 root root 8014 Jun  2 02:27 javadoc
-rwxr-xr-x 1 root root 8006 Jun  2 02:27 javah
-rwxr-xr-x 1 root root 8006 Jun  2 02:27 javap
root@7c150d68f7f9:/usr/lib/jvm/java-8-oracle/bin# ./java
bash: ./java: No such file or directory

Here is the puzzle: I see the file, it is there and when I run "file" on it, this what I get:

root@7c150d68f7f9:/usr/lib/jvm/java-8-oracle/bin# file ./java
./java: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=c46954c0ae919fe0f71b35f6716d88a811eaa92e, not stripped

Any more ideas?

@alphawolf1988

@teleki Are you using a raspberry pi 3 or 2 and what OS distribution (and version) is used ?

@teleki
teleki commented Jun 2, 2016 edited

@alphawolf1988 I am attempting to build a 4 unit ODROID C2 cluster. The specs are:

  • Amlogic S905 ARMยฎ Cortexยฎ-A53 (ARMv8) **2GHz quad core CPU**
    
  • **2GB DDR3 SDRAM**
    
  • **32 GB eMMC 5.0 HS400 Flash Storage**
    
  • ARM Maliโ„ข-450 MP3 GPU (OpenGL ES 2.0/1.1 for Linux)
    
  • HDMI 2.0 4K/60Hz display
    
  • H.265 4K/60FPS and H.264 4K/30FPS capable VPU
    
  • Gigabit Ethernet
    
  • Ubuntu 16.04 LTS based on Kernel 3.14 LTS (GNU/Linux 3.14.65-66 aarch64)
    
@teleki
teleki commented Jun 11, 2016

@alphawolf1988 @imikushin unfortunately, still no love. I started from scratch again. It stopped with the following error:

...
Step 35 : RUN cd / && mkdir -p $CATTLE_HOME && /usr/share/cattle/cattle.sh extract
 ---> Running in 03035cafc513
/usr/share/cattle/cattle.sh: line 197: /usr/bin/java: No such file or directory
The command '/bin/sh -c cd / && mkdir -p $CATTLE_HOME && /usr/share/cattle/cattle.sh extract' returned a non-zero code: 1

Attached the entire output. build-image.arm.out.txt

Also attached the Dockerfile.arm file. Dockerfile.arm.txt

It seems that the problem is that java won't run, possibly due to a mixture of 64 bit and 32 bit code.
Any more ideas? Thanks in advance!

@lavvy
lavvy commented Jun 13, 2016

@teleki did you manage to get it run on the C2

@lavvy
lavvy commented Jun 13, 2016

Or trying other OS like Debian

@teleki
teleki commented Jun 13, 2016

@lavvy not yet. I am stuck for now. I have only tried it with the default OS that came on C2.

@faddat
faddat commented Jun 15, 2016

This comment comes from a sadly underinformed place, but overall I've got
to observe that it seems that Java is the enemy of this quest.

Jacob Gadikian
E-mail: faddat@gmail.com
SKYPE: faddat
Phone/SMS: +84 167 789 6421

On Tue, Jun 14, 2016 at 9:36 PM, teleki notifications@github.com wrote:

@lavvy https://github.com/lavvy not yet. I am stuck for now. I have
only tried it with the default OS that came on C2.

โ€”
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#4703 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AGz6iSZvdFvcmF7oRrJSK3OZGI4UmanSks5qLrx6gaJpZM4IYe7D
.

@teleki
teleki commented Jun 16, 2016

@faddat I'm afraid you are right. I installed Oracle Java directly onto the C2 and that worked fine. For some reason it doesn't work the way it is installed in the container. So far I have not had a chance to figure it out why.

@imikushin imikushin referenced this issue in rancher/host-api Jun 21, 2016
Closed

Multi-arch build #77

@withinboredom

For anyone who wants to give this a go... I have prebuilt images and instructions on my blog ... even with the incredible work that's been done so far, there's still a fair bit of manual work to bring up a node -- I didn't build rancher/server, just the things to run rancher/agent on arm while rancher/server is on amd64.

@faddat
faddat commented Jun 25, 2016

Have you attempted the Ur-Power heterogeneous cluster yet? Clearly on the
server/agent level you have, but how might we make it work with some x86
agents and some ARM agents?

I'm thinking labels and careful rules, maybe? (obviously anything using
ARM at this stage is a ways from production ready, but... I think
specialized compute is one of few lifelines Moore's Law has remaining.

:)

-Jake

Jacob Gadikian
E-mail: faddat@gmail.com
SKYPE: faddat
Phone/SMS: +84 167 789 6421

On Sun, Jun 26, 2016 at 3:06 AM, Rob Landers notifications@github.com
wrote:

For anyone who wants to give this a go... I have prebuilt images and
instructions on my blog
https://www.withinboredom.info/post/rancher-on-pi/ ... even with the
incredible work that's been done so far, there's still a fair bit of manual
work to bring up a node -- I didn't build rancher/server, just the things
to run rancher/agent on arm while rancher/server is on amd64.

โ€”
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#4703 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AGz6iSoNilX1Vc81w1J3cY15mu79629wks5qPYo4gaJpZM4IYe7D
.

@withinboredom
withinboredom commented Jun 26, 2016 edited

@faddat Eventually (one day) rancher might read the metadata on the image and see what architecture it's built for and only deploy to the architecture its built for ;) Docker is kinda already working on that where pulling ubuntu:16.04 on x86 will pull the appropriate image, and will pull the equivelant on an arm architecture.

Until then, complex rules will be the norm in heterogeneous clusters.

Actually, it would be super spiffy for rancher to implement the manifest thing currently, it would probably save a headache of changing a bunch of code. Specialized utils could exist in a container instead of downloaded from the rancher server....

@lavvy
lavvy commented Jul 18, 2016

+1

@lavvy
lavvy commented Jul 25, 2016

what is the progress of this issue, is there any other place we can be tracking it from, or is it in a little halt.
what about the agent , can't we be able to add an arm node at least, before the server comes.

@jgato
jgato commented Aug 24, 2016

following @withinboredom blog I am trying to run add a Raspberry Pi 2 as a host, but it is failing to me

INFO: ENV: RANCHER_AGENT_IMAGE=rancher/agent:v1.0.2
Template parsing error: template: :1:8: executing "" at <.Volumes>: map has no entry for key "Volumes"
dirname: missing operand
Try 'dirname --help' for more information.
dirname: missing operand
Try 'dirname --help' for more information.
dirname: missing operand
Try 'dirname --help' for more information.
docker: Error response from daemon: Invalid volume spec ":": volumeinvalid: Invalid volume specification: ':'.
See 'docker run --help'. 

Which distribution do you suggest me to use in the raspberry pi?

@zerodivide1

Hi, I'm having the same issue that @jgato is having. I am using Hypriot as the host OS:

pirate@black-pearl:~$ uname -a; docker version
Linux black-pearl 4.4.15-hypriotos-v7+ #1 SMP PREEMPT Mon Jul 25 08:46:52 UTC 2016 armv7l GNU/Linux
Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:31:15 2016
 OS/Arch:      linux/arm

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:31:15 2016
 OS/Arch:      linux/arm
@zerodivide1

Doing a little digging and I found the line causing the error message @jgato and I were seeing:

https://github.com/rancher/rancher/blob/207b67ef6d5ff91db0e6dbde33a82a1c0288eb44/agent/run.sh#L193

When I run the command, I see the same error, but when I run the following, it seems to return fine:

pirate@black-pearl:~$ echo $(dirname $(dirname $(dirname $(docker inspect -f '{{index .Config.Volumes "/var/lib/cattle"}}' rancher-agent-state))))
.
@jgato
jgato commented Sep 12, 2016

@zerodivide1 how could I try it? Do I need to build my own agent images changing this line?

@jerryz1982

@jgato the problem with the arm image is that the docker client in it is a new version which has a different output format than the old version which run.sh is compatible with.
there are other issues even with the work around by @zerodivide1
https://www.adelton.com/docs/docker/docker-inspect-volumes-mounts#zero-value

The pulling request to build arm agent images is still in progress.

@will-chan will-chan modified the milestone: Unscheduled Oct 8, 2016
@natami
natami commented Nov 8, 2016

Any progress in this task?

@tritao
tritao commented Dec 1, 2016

Same question as @natami. Any status to when we can expect to get working Rancher ARM agent binaries?

@fredVT
fredVT commented Jan 26, 2017

@teleki : did you finally succeed to install rancher (server & agent) on your odroid-c2 cluster ? I would like to do the same thing, so I'm interested...

@truedat101

@mflor35 and I are also trying this with a PicoCluster running PINE64 boards. We didn't discover this issue until after we started. I'll post my results here.

@teleki
teleki commented Jan 28, 2017

@fredVT Sorry. I haven't had a chance to get back to this since last year, so no progress on it.

@LarsVader666

Yes, Rancher ARM agent is highly appreciated. Lots of RPi 3 idling ;-)

@kingprimex

I subscribe on this issue as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment