Skip to content

calico

Jump to bottom Edit New page
Lei Zhang edited this page Apr 18, 2017 · 17 revisions

Building Calicoctl

The instructions provided below specify the steps to build Calico for Docker version v1.1.1 on Linux on the IBM z Systems for following distributions:

  • RHEL (7.1, 7.2, 7.3)
  • SLES (12, 12 SP1, 12 SP2)
  • Ubuntu (16.04, 16.10)

General Notes:

  • When following the steps below please use a standard permission user unless otherwise specified.

  • A directory /<source_root>/ will be referred to in these instructions, this is a temporary writable directory anywhere you'd like to place it.

Prerequisites

  • Go -- Instructions for building Go can be found here.
  • Docker -- Instructions for install Docker can be found here
  • etcd -- Instructions for building etcd can be found here

Building Calicoctl

1. Install the system dependencies

  • For RHEL (7.1, 7.2, 7.3)
   sudo yum install curl git wget tar gcc glibc-static.s390x make
  • For SLES (12, 12 SP1, 12 SP2)
   sudo zypper install curl git wget tar gcc glibc-static.s390x make
  • For Ubuntu (16.04, 16.10)
   sudo apt-get update
   sudo apt-get install git curl tar gcc glibc-static.s390x make

2. Build dependent 3rd part packages

2.1 Build runit 
   cd /<source_root>/
   wget http://smarden.org/runit/runit-2.1.2.tar.gz
   tar xvf runit-2.1.2.tar.gz
   cd admin/runit-2.1.2/src
   make load
   cd ../

Update src/load to add -static option to make link statically

   #!/bin/sh
   # WARNING: This file was auto-generated. Do not edit!
   main="$1"; shift
   exec gcc -s -o "$main" "$main".o -static ${1+"$@"}

Then build and pack executable files

   package/compile
   tar cvf runit.tar command/
   gzip runit.tar

This runit.tar.gz is used in building calico/node docker images

2.2 Build glibc 
   cd /<source_root>/
   git clone https://github.com/sgerrand/docker-glibc-builder
   cd docker-glibc-builder/

Update Dockerfile to use s390x/ubuntu base image:

@@ -1,4 +1,4 @@
-FROM ubuntu-debootstrap:14.04
+FROM s390x/ubuntu
 MAINTAINER Sasha Gerrand <github+docker-glibc-builder@sgerrand.com>

Build an docker image and then use it to build glibc

   docker build -t glibc-builder .
   docker run --rm -e STDOUT=1 glibc-builder 2.24 /usr/glibc-compat > glibc-bin-2.24.tar.gz

glibc package is used in building docker images.

3. Build calicoctl and other components of calico

3.1 Build go-build

This builds a docker image calico/go-build that is used to build other components by containerized build

   cd /<source_root>/
   mkdir -p calico/src/github.com/projectcalico
   cd calico/src/github.com/projectcalico
   git clone https://github.com/projectcalico/go-build
   cd go-build
   git checkout v0.5

Modify Dockerfile for s390x

@@ -1,4 +1,4 @@
-FROM golang:1.8.1-alpine
+FROM tmh1999/alpine-s390x
 MAINTAINER Tom Denham <tom@projectcalico.org>

 # Install su-exec for use in the entrypoint.sh (so processes run as the right user)
@@ -9,22 +9,25 @@ MAINTAINER Tom Denham <tom@projectcalico.org>
 # Install wget for fetching glibc
 # Install make for building things
 # Install util-linux for column command (used for output formatting).
+RUN apk update
 RUN apk add --no-cache su-exec curl bash git mercurial make wget util-linux

+RUN apk add go
 # Install glibc
 RUN wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://raw.githubusercontent.com/sgerrand/alpine-pkg-glibc/master/sgerrand.rsa.pub
-RUN wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.23-r3/glibc-2.23-r3.apk
-RUN apk add glibc-2.23-r3.apk
+COPY pkg /pkg/
+RUN  cd / && \
+     tar xvf pkg/glibc-bin-2.24.tar.gz && \
+     mv /pkg/ld.so.conf /usr/glibc-compat/etc/   &&  \
+     /usr/glibc-compat/sbin/ldconfig /lib /usr/glibc/usr/lib && \
+     rm -rf /pkg

 # Disable cgo so that binaries we build will be fully static.
 ENV CGO_ENABLED=0
+ENV GOROOT=/usr/lib/go
+ENV GOPATH=/go
+ENV PATH=$PATH:$GOROOT/bin:$GOPATH/bin

-# Apply patches to Go runtime and recompile.
-# See https://github.com/golang/go/issues/5838 for defails of vfork patch.
-COPY patches/use-clone-vfork-b7edfba429d982e3e065d637334bcc63ad49f8f9.patch \
-     /tmp/use-clone-vfork-b7edfba429d982e3e065d637334bcc63ad49f8f9.patch
-RUN cd /usr/local/go && \
-    patch -p 1 < /tmp/use-clone-vfork-b7edfba429d982e3e065d637334bcc63ad49f8f9.patch
 RUN go install -v -a syscall

   mkdir pkg
   cp /<source_root>/docker-glibc-builder/glibc-bin-2.24.tar.gz pkg/

Create a file pkg/ld.so.conf with follow content:

     /usr/local/lib
     /usr/glibc-compat/lib
     /usr/lib
     /lib

Then start to build calico/go-build image

   make
3.2 Build calicoctl

This component builds interactive command interface calicoctl and docker image calico/node

   cd /<source_root>/calico/src/github.com/projectcalico
   git clone https://github.com/projectcalico/calicoctl
   cd calicoctl
   git checkout v1.1.1

Modify Makefile.calicoctl for s390x:

@@ -6,7 +6,7 @@
 ###############################################################################
 # Determine which OS / ARCH.
 OS := $(shell uname -s | tr A-Z a-z)
-ARCH := amd64
+ARCH := s390x
 ###############################################################################
 # Subcomponent versions:
 GO_BUILD_VER:=latest
@@ -75,11 +75,11 @@ binary: $(CALICOCTL_FILES) vendor
        GOOS=$(OS) GOARCH=$(ARCH) CGO_ENABLED=0 go build -v $$INSTALL_FLAG -o dist/calicoctl-$(OS)-$(ARCH) $(LDFLAGS) "./calicoctl/calicoctl.go"

 dist/calicoctl: $(CALICOCTL_FILES) vendor
-       $(MAKE) dist/calicoctl-linux-amd64
-       mv dist/calicoctl-linux-amd64 dist/calicoctl
+       $(MAKE) dist/calicoctl-linux-s390x
+       mv dist/calicoctl-linux-s390x dist/calicoctl

-dist/calicoctl-linux-amd64: $(CALICOCTL_FILES) vendor
-       $(MAKE) OS=linux ARCH=amd64 binary-containerized
+dist/calicoctl-linux-s390x: $(CALICOCTL_FILES) vendor
+       $(MAKE) OS=linux ARCH=s390x binary-containerized

Then make dist/calicoctl to build, the executable file is in dist/calicoctl Because calico/node needs the executable files built from other components, let's build other components first, then come back.

   make  dist/calicoctl
   mkdir -p /<source_root>/calico/src/github.com/projectcalico/calicoctl/calico_node/filesystem/bin/    ## store executable files built from other components
3.3 Build bird 
   cd /<source_root>/calico/src/github.com/projectcalico
   git clone https://github.com/projectcalico/bird
   cd bird
   git checkout v0.3.0

Modify Dockerfile for s390x

@@ -1,4 +1,4 @@
-FROM alpine:latest
+FROM tmh1999/alpine-s390x
 MAINTAINER Tom Denham <tom@projectcalico.org>

Then run build.sh to build 3 executable files (dist/) that are required by building calico/node image.

   ./build.sh
   cp dist/* /<source_root>/calico/src/github.com/projectcalico/calicoctl/calico_node/filesystem/bin/
3.4 Build confd 
   cd /<source_root>
   git clone https://github.com/kelseyhightower/confd.git calico/src/github.com/kelseyhightower/confd
   cd calico/src/github.com/kelseyhightower/confd
   export GOROOT=<go_instll_dir>
   export PATH=$GOROOT/bin:$PATH
   go build -ldflags "-v -linkmode=external '-extldflags=-v -static'"
   cp confd /<source_root>/calico/src/github.com/projectcalico/calicoctl/calico_node/filesystem/bin/

This built executable file confd required by calico/node.

3.4 Build libnetwork-plugin 
   cd /<source_root>/calico/src/github.com/projectcalico
   git clone https://github.com/projectcalico/libnetwork-plugin
   cd libnetwork-plugin
   git checkout out v1.1.0

Modify Dockerfile to change FROM alpine to FROM tmh1999/alpine-s390x

   make calico/libnetwork-plugin
   cp dist/libnetwork-plugin /<source_root>/calico/src/github.com/projectcalico/calicoctl/calico_node/filesystem/bin/

This builds executable file dist/libnetwork-plugin required by calico/node and docker image calico/libnetwork-plugin

3.5 Build felix 
   cd /<source_root>/calico/src/github.com/projectcalico
   git clone https://github.com/projectcalico/felix
   cd felix
   git checkout 2.1.1

Modify Makefile to use calico/go-build:latest

@@ -80,7 +80,7 @@ help:
 all: deb rpm calico/felix
 test: ut

-GO_BUILD_CONTAINER?=calico/go-build:v0.4
+GO_BUILD_CONTAINER?=calico/go-build

To build felix it needs felixbackend.pb.go that is generated by a docker image calico/protoc. Let's first built this images.

   cd /<source_root>/calico/src/github.com/projectcalico
   git clone https://github.com/tigera/docker-protobuf
   cd  docker-protobuf

Modify Dockerfile as follows:

@@ -1,4 +1,4 @@
-FROM golang:1.6
+FROM s390x/ubuntu

 MAINTAINER Shaun Crampton <shaun@tigera.io>

@@ -8,6 +8,10 @@ WORKDIR /src
 # We want v3.0.0 but the tag for v3.0.0 has broken download links.  Pin to
 # master for now :-(
 ENV PROTOBUF_TAG master
+RUN apt-get -y  update \
+     && apt-get -y  install golang git curl
+ENV GOROOT=/usr/lib/go
+ENV GOPATH=/usr/local

Then build docker images calico/protoc

   docker build -t calico/protoc .

Let's come back to felix and build

   cd /<source_root>/calico/src/github.com/projectcalico/felix
   make bin/calico-felix
   cp bin/calico-felix /<source_root>/calico/src/github.com/projectcalico/calicoctl/calico_node/filesystem/bin/

Built calico-felix is in bin/

3.6 Build calico-bgp-daemon 
   cd /<source_root>/calico/src/github.com/projectcalico
   git clone https://github.com/projectcalico/calico-bgp-daemon
   cd calico-bgp-daemon
   git checkout v0.2.0

Modify Dockerfile to change to FROM tmh1999/alpine-s390x

@@ -1,4 +1,4 @@
-FROM alpine:3.4
+FROM tmh1999/alpine-s390x

   make build-containerized
   cp dist/* /<source_root>/calico/src/github.com/projectcalico/calicoctl/calico_node/filesystem/bin/

Two executable files calico-bgp-daemon and gobgp are in dist/

3.7 Build image calico/node

Now let's come beck to calicoctl and build image calico/node

   cd  /<source_root>/calico/src/github.com/projectcalico/calicoctl
   cp -r /<source_root>/calico/src/github.com/projectcalico/go-build/pkg  calico_node/
   cp /<source_root>/admin/runit-2.1.2/runit.tar.gz calico_node/pkg/

Modify Makefile.calico-node:

@@ -123,7 +123,7 @@ $(NODE_CONTAINER_BIN_DIR)/allocate-ipip-addr: dist/allocate-ipip-addr

 ## Build startup.go
 startup:
-       GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -v -i -o dist/startup $(LDFLAGS) "./calico_node/startup/startup.go"
+       GOOS=linux GOARCH=s390x CGO_ENABLED=0 go build -v -i -o dist/startup $(LDFLAGS) "./calico_node/startup/startup.go"

 dist/startup: $(STARTUP_FILES) vendor
        mkdir -p dist
@@ -139,7 +139,7 @@ dist/startup: $(STARTUP_FILES) vendor

 ## Build allocate_ipip_addr.go
 allocate-ipip-addr:
-       GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -v -i -o dist/allocate-ipip-addr $(LDFLAGS) "./calico_node/allocateipip/allocate_ipip_addr.go"
+       GOOS=linux GOARCH=s390x CGO_ENABLED=0 go build -v -i -o dist/allocate-ipip-addr $(LDFLAGS) "./calico_node/allocateipip/allocate_ipip_addr.go"

 dist/allocate-ipip-addr: $(ALLOCATE_IPIP_FILES) vendor
        mkdir -p dist
@@ -198,8 +198,8 @@ certs/.certificates.created:
        touch certs/.certificates.created

 busybox.tar:
-       docker pull busybox:latest
-       docker save --output busybox.tar busybox:latest
+       docker pull s390x/busybox:latest
+       docker save --output busybox.tar s390x/busybox:latest

and modify docker file calico_node/Dockerfile

@@ -11,25 +11,24 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-FROM alpine:3.4
+FROM tmh1999/alpine-s390x
 MAINTAINER Tom Denham <tom@projectcalico.org>

 # Set the minimum Docker API version required for libnetwork.
 ENV DOCKER_API_VERSION 1.21

 # Download and install glibc for use by non-static binaries that require it.
+COPY pkg /pkg/
+RUN apk update
 RUN apk --no-cache add wget ca-certificates libgcc && \
     wget -q -O /etc/apk/keys/sgerrand.rsa.pub https://raw.githubusercontent.com/sgerrand/alpine-pkg-glibc/master/sgerrand.rsa.pub && \
-    wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.23-r3/glibc-2.23-r3.apk && \
-    wget https://github.com/sgerrand/alpine-pkg-glibc/releases/download/2.23-r3/glibc-bin-2.23-r3.apk && \
-    apk add glibc-2.23-r3.apk glibc-bin-2.23-r3.apk && \
-    /usr/glibc-compat/sbin/ldconfig /lib /usr/glibc/usr/lib && \
-    apk del wget && \
-    rm -f glibc-2.23-r3.apk glibc-bin-2.23-r3.apk
+    tar xvf pkg/glibc-bin-2.24.tar.gz && \
+    mv pkg/ld.so.conf /usr/glibc-compat/etc/   &&  \
+    /usr/glibc-compat/sbin/ldconfig /lib /usr/glibc/usr/lib

 # Install runit from the community repository, as its not yet available in global
-RUN apk add --no-cache --repository "http://alpine.gliderlabs.com/alpine/edge/community" runit
-
+RUN  tar xvf pkg/runit.tar.gz  && \
+     cp command/* /usr/local/bin/
 # Install remaining runtime deps required for felix from the global repository
 RUN apk add --no-cache ip6tables ipset iputils iproute2 conntrack-tools

Start to build calico/node:

shell
make calico/node
docker tag calico/node quay.io/calico/node

=======lei

4. Install Calico CNI plugins

mkdir -p /opt/cni/bin

Build CNI plugins calico and calico-ipam

cd /<source_root>/calico/src/github.com/projectcalico/
git clone https://github.com/projectcalico/cni-plugin.git
cd cni-plugin
git checkout v1.6.2
make dist/calico dist/calico-ipam
cp dist/* /opt/cni/bin

Build the standard CNI loopback plugin

cd /<source_root>/calico/src/github.com/projectcalico/
git clone https://github.com/containernetworking/cni.git
cd cni
git checkout v0.5.2
./build.sh
cp bin/loopback /opt/cni/bin

5. Install the Calico network policy controller

Build calico/kube-policy-controller image

cd /<source_root>/calico/src/github.com/projectcalico/
git clone https://github.com/projectcalico/k8s-policy.git
cd k8s-policy
git checkout v0.5.4
make docker-image
docker tag calico/kube-policy-controller quay.io/calico/kube-policy-controller:v0.5.4

6. Using Calico with Kubernetes

Run a etcd service locally

export ETCD_UNSUPPORTED_ARCH=s390x
cd /<source_root>/src/github.com/coreos/etcd
./bin/etcd &

The Kubernetes master and each Kubernetes node require the calico/node container. Each node must also be recorded in the Calico datastore. The calico/node container can be run directly through docker, or it can be done using the calicoctl utility.

mkdir -p /<demo>/
cd /<demo>/
cp /<source_root>/calico/src/github.com/projectcalico/calicoctl/dist/calicoctl .
ETCD_ENDPOINTS=http://<ETCD_IP>:<ETCD_PORT> ./calicoctl node run

Replace <ETCD_IP>:<ETCD_PORT> with your etcd configuration. In our case it is 127.0.0.1:2379.

Prepare a Calico CNI configuration file

mkdir -p /etc/cni/net.d
cat >/etc/cni/net.d/10-calico.conf <<EOF
{
    "name": "calico-k8s-network",
    "type": "calico",
    "etcd_endpoints": "http://<ETCD_IP>:<ETCD_PORT>",
    "log_level": "info",
    "ipam": {
        "type": "calico-ipam"
    },
    "policy": {
        "type": "k8s"
    },
    "kubernetes": {
        "kubeconfig": "</PATH/TO/KUBECONFIG>"
    }
}
EOF

The calico/kube-policy-controller implements the Kubernetes NetworkPolicy API by watching the Kubernetes API for Pod, Namespace, and NetworkPolicy events and configuring Calico in response. It runs as a single pod managed by a Deployment. To install the policy controller, download the policy controller manifest:

wget http://docs.projectcalico.org/v2.1/getting-started/kubernetes/installation/policy-controller.yaml

Modify <ETCD_ENDPOINTS> to point to your etcd cluster. Then install it using kubectl:

kubectl create -f policy-controller.yaml

4. Test calicoctl (optional)

5. Run calico with Docker

After building calictoctl and calico/node, it is ready to run calico with docker, please refer Document

References:

https://github.com/projectcalico

Add a custom footer
Add a custom sidebar

Clone this wiki locally