Permalink
Switch branches/tags
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
373 lines (276 sloc) 11.9 KB

Deploying OpenShift Build System

OSBS consists of several components, namely:

  • operating system - RHEL 7, Centos 7 or Fedora (latest greatest) should work out of the box
  • Docker
  • OpenShift 3 - Enterprise and Origin both work
  • atomic-reactor
  • (optional) authenticating proxy based on Apache httpd
  • (optional) docker registry where built images are going to be pushed

The recommended way to install OSBS is to use our ansible playbook. Please note that the playbook may not be in sync with this guide - feel free to report any issues.

Docker

Use the docker shipped with your operating system. Consult OpenShift documentation - there might be restrictions on what versions of Docker you can use.

$ dnf install docker

Docker storage

Before using docker it is necessary to set up its storage. Refer to this guide and the manual page of docker-storage-setup for more information. On RPM-based distros it's most likely that you want to use the direct-lvm method.

OpenShift

Both OpenShift Origin or OpenShift Enterprise can be used to host OSBS.

Installing OpenShift Origin

See the upstream documentation for installation instructions.

Origin is available in Fedora, you can install it by running

$ dnf install origin-master origin-node

Other option is to install OpenShift from its source.

Installing OpenShift Enterprise

See OpenShift Enterprise documentation for installation instructions.

# correct repositories have to be enabled
$ yum install atomic-openshift-master atomic-openshift-node atomic-openshift-sdn-ovs

Configuration

OpenShift uses various files for master and node that are generated when RPMs are installed:

  • SSL certificates
  • policies
  • master and node configs
  • /etc/origin/master/master-config.yaml
  • /etc/origin/node-$hostname/node-config.yaml
  • init scripts configuration (e.g. log level)
  • /etc/sysconfig/origin-master
  • /etc/sysconfig/origin-node

Data will be stored in /var/lib/origin. Inspect the configs and change them accordingly. In case of more drastic changes you may want to re-generate the configuration using different parameters with openshift start master --write-config and oadm create-node-config commands.

Management

Starting OpenShift:

$ systemctl start origin-master && systemctl start origin-node

Wiping all runtime data:

$ systemctl stop origin-master origin-node
$ rm -rf /var/lib/origin/*
$ systemctl start origin-master origin-node

Talking to OpenShift via command line

All communication with OpenShift is performed via executable oc (the OpenShift Client). The binary needs to authenticate, otherwise all the requests will be denied. Authentication is handled via configuration file for oc. You need to set an environment variable to point oc to the admin config:

$ export KUBECONFIG=/etc/origin/master/admin.kubeconfig

OpenShift client is available in Fedora through the origin-clients package.

$ dnf install origin-clients

Useful Commands

  • oc get builds — list builds
  • oc get pods — list pods
  • oc describe policyBindings :default — show authorization setup
  • oc describe build <build> — get info about build
  • oc build-logs <build> — get build logs (or docker logs <container>), -f to follow

For more information see OpenShift's documentation. Good starting point is also this guide.

Namespaces and projects

OpenShift and Kubernetes use namespaces for isolation. OpenShift extends namespace concept to project.

In order to build images in OpenShift, you don't need to create new project/namespace. There is namespace called default and it's present after installation. If your OpenShift instance is being used by other people, make sure that everyone is using their own project/namespace. Do not ever share your namespace with others.

Authentication and authorization

In case you would like to turn the authentication off (which is not recommended, but should be fine for testing), run following in your namespace:

$ oadm policy add-role-to-group edit system:unauthenticated system:authenticated

In production, you most likely want to set up some form of authentication and then configure suitable authorization policy.

Certificates based authentication setup

In order to be able to submit builds, your user needs to have at least edit role on the chosen namespace.

The easiest way to access OpenShift is to create a dedicated user and set just enough rights for the user:

$ oadm create-api-client-config \
  --certificate-authority='/etc/origin/master/ca.crt' \
  --client-dir='/etc/origin/master/user-builder' \
  --signer-cert='/etc/origin/master/ca.crt' \
  --signer-key='/etc/origin/master/ca.key' \
  --signer-serial='/etc/origin/master/ca.serial.txt' \
  --user='system:builder'

This will create kubeconfig and certificates at /etc/origin/master/user-builder. All we need now is to set proper permissions for the user:

$ oadm policy add-role-to-user -n default edit system:builder

htpasswd based authentication setup

This is very well documented in OpenShift's documentation.

  1. create htpasswd file:
htpasswd -c /etc/origin/htpasswd builder
  1. update master config at /etc/origin/master/master-config.yaml:
oauthConfig:
  ...
  identityProviders:
  - name: htpasswd_provider
    challenge: true
    login: true
    provider:
      apiVersion: v1
      kind: HTPasswdPasswordIdentityProvider
      file: /etc/origin/htpasswd
  1. set correct permissions for the builder user:
$ oadm policy add-role-to-user -n default edit htpasswd_provider:builder
  1. put username and password to config of osbs-client:
[the-instance]
openshift_uri = https://localhost:8443/
username = builder
password = ...

kerberos based authentication setup

OpenShift itself cannot use Kerberos authentication, however it can delegate the authentication to reverse proxy. See the corresponding section below for details.

Apache-based authenticating proxy

You can setup OpenShift with a proxy in front of it. This proxy may have an authentication, e.g. kerberos or basic auth. The proxy should then forward username to OpenShift via X-Remote-User http header.

Communication between proxy and OpenShift needs to be secure. Proxy needs to use specific SSL client certificate signed by CA which is known (and preconfigured) in OpenShift. We can use self-signed certificate for this because it won't be exposed to the outside world.

For more information, see the upstream guide.

Here's how to do it:

$ cd /var/lib/origin
$ openssl req -new -nodes -x509 -days 3650 -extensions v3_ca -keyout proxy_auth.key -out proxy_auth.crt
$ openssl rsa -in proxy_auth.key -out proxy_auth.key
$ cp /etc/origin/master/ca.crt /etc/pki/tls/certs/openshift_ca.crt
$ cat proxy_auth.{crt,key} > /etc/pki/tls/private/openshift_certkey.crt

OpenShift master configuration snippet (it uses RequestHeaderIdentityProvider):

  oauthConfig:
    identityProviders:
    - name: my_request_header_provider
       challenge: false
       login: false
       provider:
         apiVersion: v1
         kind: RequestHeaderIdentityProvider
         clientCA: /var/lib/origin/proxy_auth.crt
         headers:
         - X-Remote-User

Note that the certificate we generated can be used as a CA here because it is self-signed and is thus its own CA.

Kerberos authentication example

<VirtualHost *:9443>
    SSLProxyEngine On
    SSLProxyCACertificateFile /etc/pki/tls/certs/openshift_ca.crt
    SSLProxyMachineCertificateFile /etc/pki/tls/private/openshift_certkey.crt

    <Location "/">
        ProxyPass https://127.0.0.1:8443/ connectiontimeout=30 timeout=300
        ProxyPassReverse https://127.0.0.1:8443/
    </Location>

    # don't auth /oauth/token/request and /oauth/token/display
    <ProxyMatch /oauth/token/.*>
        Require all granted
    </ProxyMatch>

    # /oauth/authorize and /oauth/approve should be protected by Apache.
    <ProxyMatch /oauth/a.*>
        AuthType Kerberos
        AuthName "OSBS Kerberos Authentication"
        KrbMethodNegotiate on
        KrbMethodK5Passwd off
        KrbServiceName Any
        KrbAuthRealms REALM.COM
        Krb5Keytab /path/to/keytab
        KrbSaveCredentials off
        Require valid-user
        RequestHeader set X-Remote-User %{REMOTE_USER}s
        RequestHeader unset Authorization
        RequestHeader unset WWW-Authenticate
    </ProxyMatch>

    # All other requests should use Bearer tokens. These can only be verified by
    # OpenShift so we need to let these requests pass through.
    <ProxyMatch ^/oauth/>
        SetEnvIfNoCase Authorization Bearer passthrough
        Require env passthrough
    </ProxyMatch>
</VirtualHost>

With this configuration run the following to allow all authenticated users to start builds:

$ oadm policy add-role-to-group edit system:authenticated

atomic-reactor

In order to build images, you need to have a build image. It is the image used by OpenShift to perform builds. The image has installed component called atomic-reactor, which performs the build itself.

Please see the project's documentation for more complete information. There's also ansible role that can be used to pull or build the image.

Unless configured otherwise in osbs-client, build image is expected to be tagged as buildroot:latest on the build host. If you are using multinode setup you should set up integrated docker registry and refer to the build image by it's ImageStream tag.

Pulling build image

The latest development version of the build image is available at docker hub:

$ docker pull slavek/atomic-reactor

Building build image

You can also build the image yourself using Dockerfile from root of this repository.

Optional packages

  • osbs-client — if you would like to submit results back to OpenShift (requires atomic-reactor-metadata)
  • atomic-reactor-kojiatomic-reactor plugin for getting packages from koji targets
  • fedpkg — atomic-reactor can fetch artifacts from lookaside cache of dist-git

Time to build it:

$ docker build --no-cache=true --tag=buildroot .

Docker registry

The built images need to be pushed somewhere. You can use legacy docker registry:

$ dnf install docker-registry
$ systemctl enable docker-registry
$ systemctl start docker-registry

Or you can use current v2 docker registry:

$ dnf install docker-distribution
$ systemctl enable docker-distribution
$ systemctl start docker-distribution