ARM Support for halcyon-kubernetes #41

aric49 · 2017-01-24T14:37:00Z

Enabling ARM and ARM64 Support for halcyon-kubernetes deployment feature enhancement requested by @ravishivt

To enable ARM Support:

in group_vars/all.yml, set arm_enabled: true
set kube_sdn: flannel

@ravishivt -- Let me know if this is the functionality you are looking for.

This change is

v1k0d3n

I'd like to see this performed in a more ansible-native way, querying the system for platform specifics, and deploying as such. I think this will result in less forethought from the operator and it will just....work (for n=x platform).

v1k0d3n · 2017-01-24T18:52:00Z

kube-deploy/group_vars/all.yml

@@ -18,6 +18,8 @@ bootstrap_os: ubuntu
 public_iface: enp0s3
 nat_iface: enp0s3
 proxy_enable: false
+#Only enable this if you are deploying on ARM or ARM64 with Fannel SDN provider
+arm_enabled: false


@aric49 one approach I was thinking was to read off of the system for the type of architecture, rather than including additional flags for each of them. For instance, Ansible can be used to return system traits...use that rather than piling on our global_vars inputs.

@v1k0d3n I was thinking along similar lines originally. However, I don't have an arm environment I can use to validate that the playbooks are going to run. I thought that adding it to group_vars would be a good initial approach so that the users can turn it on or off as needed. I agree that pulling it dynamically is a better approach, I just don't have a way to test it at the moment. Any ideas?

Looking at the setup module output on my arm64 boards, there are two ansible variables we could use. Alternatively, we could pull it directly from the board using uname -m.

"ansible_machine": "aarch64", "ansible_architecture": "aarch64",

To map these architecture strings to the names used in actual docker image, take a look at the case statement in https://github.com/kubernetes/kubernetes/blob/85d328f8da12f0f0e5b7e6dd2356e56cd584fc3e/hack/local-up-cluster.sh#L234

We could exclude s390x and x86 because google doesn't even release images for those. https://console.cloud.google.com/kubernetes/images/list?location=GLOBAL&project=google-containers CTRL+F for a common k8s image like "kube-apiserver" and note the architectures kubernetes supports.

As for needing test instances, I can try to put up a couple instances on the internet. Alternatively, scaleway has pretty cheap arm64 VPS instances at $2.99/month.

i would strongly suggest using facts based on the CPU for arm64. we can always take a stab at it, and let @ravishivt try it. we can also see if @luxas would be willing to try our project, since he is a great Pi resource on the Kubernetes team. @aric49 if all else fails...I will bring the infamous Pi Cluster to the office, and you can use that to test with. :)

aric49 · 2017-01-24T20:51:25Z

Re-opening PR for re-factoring

aric49 · 2017-01-24T21:05:54Z

I refactored this PR so it will dynamically check to see if the system is ARM-based by parsing the output of uname -a, as per @ravishivt

ravishivt · 2017-01-24T21:33:50Z

kube-deploy/roles/kube-addons/tasks/addons-dashboard.yml

@@ -14,7 +14,18 @@
 # limitations under the License.
 #

+#For ARM support only
+- name: Checking for ARM support
+  shell: uname -a | grep -i "arm"


arm64 boards actually return back aarch64 and arm boards return arm.*. We'll need to do a conditional logic similar to https://github.com/kubernetes/kubernetes/blob/85d328f8da12f0f0e5b7e6dd2356e56cd584fc3e/hack/local-up-cluster.sh#L234. Also, uname -m gives the machine info so you won't have to grep the output.

From Raspberry Pi (32-bit ARM, arm)

$ uname -a Linux rpi2 4.1.19-v7+ #858 SMP Tue Mar 15 15:56:00 GMT 2016 armv7l armv7l armv7l GNU/Linux $ uname -m armv7l

From Odroid C2 (64-bit ARM, arm64)

$ uname -a Linux ravi-arm51 3.14.79-103 #1 SMP PREEMPT Fri Jan 13 07:06:12 BRST 2017 aarch64 aarch64 aarch64 GNU/Linux $ uname -m aarch64

ravishivt · 2017-01-24T21:34:30Z

kube-deploy/roles/kube-addons/tasks/addons-dashboard.yml

+  ignore_errors: true
+
+- name: deploying the kubernetes dashboard - ARM support only
+  shell: curl -sSL https://rawgit.com/kubernetes/dashboard/master/src/deploy/kubernetes-dashboard.yaml | sed "s/amd64/arm64/g" | kubectl create -f -


The replacement of arm or arm64 should be dependent on architecture. See other comment.

So if I am understanding your comments properly, if the uname -m output returns: aarch64, or arm*, then we need to replace the amd64 strings in the kubernetes manifest with arm64, correct?

I am asking because I was thinking through an approach in which I grab the value of "ansible_architecture" (same output as uname -m) and using that value in your sed clause.. However, based on your examples, I don't think that is a valid approach. I am trying to think through this from the perspective of supporting not only ARM and x86 but others as well. Do you know if any other architectures are supported in Flannel, or is it just x86/64 and ARM? If not, I will only need to test if "x86" is in the ansible_architecture variable and if it isn't, then use your sed clause to replace with arm.

Let me know if this sounds like a good approach to move forward with.

Regarding your first question, that logic is not quite correct. There are two common ARM architectures that we should support, arm and arm64. They are fairly popular, think of Raspberry Pi's and their competitors with more modern architectures (Odroid C2, Pine64). So we need to conditionally check if the device is arm64 or arm and then do a sed replacement based on that. The tricky part is that uname -m doesn't return arm or arm64 directly. It returns either arm<> or aarch64. The case statement from my earlier link gives an example of how to translate uname -m to the right image name. Order of checks do matter.

I am trying to think through this from the perspective of supporting not only ARM and x86 but others as well. Do you know if any other architectures are supported in Flannel, or is it just x86/64 and ARM?

My goal is the same regarding support for other architectures, not just arm boards. Flannel currently has released images for amd64, arm, arm64, and ppc64le. I came up with that list from kubernetes container registry.

…p for bad syntax

aric49 · 2017-01-25T15:45:20Z

Thank you for the input @ravishivt! My latest commit addresses all of the possible architecture scenarios for Flannel and dashboard deployments. Please review and see if this is a good solution.

v1k0d3n

much better @aric49. this covers other scenarios that could come up re: architecture.

@ravishivt can you please test this know and let us know how this works for you? be aware that you may run into docker issue #42, but i have asked for changes on this PR.

ravishivt · 2017-01-26T01:59:37Z

I will give it a go. I glanced through the code and it looks good. Great work @aric49!

Aric Renzo added 2 commits January 23, 2017 17:26

ARM/ARM64 support to Halcyon deployment

4791662

Reverting back to defaults

6f76335

aric49 requested review from alanmeadows and v1k0d3n January 24, 2017 14:37

v1k0d3n requested changes Jan 24, 2017

View reviewed changes

aric49 closed this Jan 24, 2017

aric49 reopened this Jan 24, 2017

Aric Renzo added 4 commits January 24, 2017 15:53

refactoring for dynamic ARM detection

d278c74

refactoring for dynamic ARM detection

99a9d3d

Merge branch 'master' of github.com:aric49/halcyon-kubernetes

6f49beb

Changing back to weave

4fe9aa5

ravishivt suggested changes Jan 24, 2017

View reviewed changes

Adding dynamic support for alternative CPU architectures, also cleanu…

29ea837

…p for bad syntax

v1k0d3n approved these changes Jan 25, 2017

View reviewed changes

v1k0d3n merged commit dd0a6a1 into att-comdev:master Jan 25, 2017

v1k0d3n mentioned this pull request Feb 2, 2017

Multi-platform support #40

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ARM Support for halcyon-kubernetes #41

ARM Support for halcyon-kubernetes #41

aric49 commented Jan 24, 2017 •

edited by v1k0d3n

Loading

v1k0d3n left a comment

v1k0d3n Jan 24, 2017

aric49 Jan 24, 2017

ravishivt Jan 24, 2017

v1k0d3n Jan 24, 2017 •

edited

Loading

aric49 commented Jan 24, 2017

aric49 commented Jan 24, 2017

ravishivt Jan 24, 2017 •

edited

Loading

ravishivt Jan 24, 2017

aric49 Jan 24, 2017

ravishivt Jan 25, 2017

aric49 commented Jan 25, 2017

v1k0d3n left a comment •

edited

Loading

ravishivt commented Jan 26, 2017

ARM Support for halcyon-kubernetes #41

ARM Support for halcyon-kubernetes #41

Conversation

aric49 commented Jan 24, 2017 • edited by v1k0d3n Loading

v1k0d3n left a comment

Choose a reason for hiding this comment

v1k0d3n Jan 24, 2017

Choose a reason for hiding this comment

aric49 Jan 24, 2017

Choose a reason for hiding this comment

ravishivt Jan 24, 2017

Choose a reason for hiding this comment

v1k0d3n Jan 24, 2017 • edited Loading

Choose a reason for hiding this comment

aric49 commented Jan 24, 2017

aric49 commented Jan 24, 2017

ravishivt Jan 24, 2017 • edited Loading

Choose a reason for hiding this comment

ravishivt Jan 24, 2017

Choose a reason for hiding this comment

aric49 Jan 24, 2017

Choose a reason for hiding this comment

ravishivt Jan 25, 2017

Choose a reason for hiding this comment

aric49 commented Jan 25, 2017

v1k0d3n left a comment • edited Loading

Choose a reason for hiding this comment

ravishivt commented Jan 26, 2017

aric49 commented Jan 24, 2017 •

edited by v1k0d3n

Loading

v1k0d3n Jan 24, 2017 •

edited

Loading

ravishivt Jan 24, 2017 •

edited

Loading

v1k0d3n left a comment •

edited

Loading