Alpine-Buildbot

Container for Alpine Linux + Python3 + Buildbot

---

This image serves as the base container for running a Buildbot instance ( master/localworker or standalone worker roles ) to build applications, run tasks, and other automations.

Based on Alpine Linux from my alpine-python3 image with the s6 init system overlayed in it.

The image is tagged respectively for the following architectures/roles,

• Buildmaster - to run the master node

  • armhf
  • x86_64 (retagged as the latest )

• Buildworker - to run a standalone worker on other devices.

  • armhf
  • x86_64 (retagged as the latest )

armhf builds have embedded binfmt_misc support and contain the qemu-user-static binary that allows for running it also inside an x64 environment that has it.

---

Get the Image

---

Pull the master image for your architecture it's already available from Docker Hub.

# make pull
docker pull woahbase/alpine-buildmaster:x86_64

or just pull the worker,

# make pull ROLE=worker
docker pull woahbase/alpine-buildworker:x86_64

---

Configuration Defaults

---

• This images already has a user alpine configured to drop privileges to the passed PUID/PGID which is ideal if its used to run in non-root mode. That way you only need to specify the values at runtime and pass the -u alpine if need be. (run id in your terminal to see your own PUID/PGID values.)

• The env variable ROLE determines if you are running a master or worker. This also determines what image you'll be running when used with the makefile.

• The WORKERNAME variable determines the name of the worker, or the default worker in case of master, defaults to hostname.

• For the master node, mount the configurations at the BUILDBOT_HOME directory inside the container, by default it is /home/alpine/buildbot-config. This is optional for workers, however make sure you have enough cpu power and memory for the workers to do the heavy lifting if required.

• To create a default master check the setup target in the makefile. This will generate a master with default configurations, which also includes a remote worker, so for setting up the worker credentials you need to provide the password in the env variable PASSWORD.

• The above can also setup a worker depending on the ROLE variable, for workers, you need to pass the MASTERADDRESS and PASSWORD.

• Before the master or worker is started, there is option to install project dependencies as APK or PIP packages if required. The env variables REQUIRED_APK and REQUIRED_PIP installs the list of packages from alpine repository or pip respectively.

• If you need to perform some tasks before the services are started, consider dropping a cont-init script inside /etc/cont-init.d/, the name should begin with atleast 20-* as 10-setup is already taken.

---

Setup / Run

---

If you want to run images for other architectures, you will need to have binfmt support configured for your machine. multiarch, has made it easy for us containing that into a docker container.

# make regbinfmt
docker run --rm --privileged multiarch/qemu-user-static:register --reset

Without the above, you can still run the image that is made for your architecture, e.g for an x86_64 machine..

Run setup to generate the default configurations if you don't already have one.

# make setup ROLE=master PASSWORD=insecurebydefault
docker run --rm -it \
  --name docker_buildmaster --hostname buildmaster \
  -c 256 -m 256m \
  -e BUILDBOT_HOME=/home/alpine/buildbot-config \
  -e PASSWORD=insecurebydefault \
  -e PGID=1000 -e PUID=1000 \
  -e ROLE=master \
  -e WORKERNAME=buildbot \
  -p 9989:9989 -p 8010:8010 -p 9990:9990 \
  -v config:/home/alpine/buildbot-config \
  woahbase/alpine-buildmaster:x86_64

for the worker,

# make setup ROLE=worker PASSWORD=insecurebydefault
docker run --rm -it \
  --name docker_buildworker --hostname buildworker \
  -e BUILDBOT_HOME=/home/alpine/buildbot-config \
  -e MASTERADDRESS=localhost \
  -e PASSWORD=insecurebydefault \
  -e PGID=1000 -e PUID=1000 \
  -e ROLE=worker \
  -e WORKERNAME=buildbot \
  -v config:/home/alpine/buildbot-config \
  woahbase/alpine-buildworker:x86_64

Running make starts the service.

# make ROLE=master REQUIRED_PIP="txrequests" REQUIRED_APK="curl git"
docker run --rm -it \
  --name docker_buildmaster --hostname buildmaster \
  -c 256 -m 256m \
  -e BUILDBOT_HOME=/home/alpine/buildbot-config \
  -e PGID=1000 -e PUID=1000 \
  -e REQUIRED_APK="curl git" \
  -e REQUIRED_PIP="txrequests" \
  -e ROLE=master \
  -e WORKERNAME=buildbot \
  -v config:/home/alpine/buildbot-config  \
  woahbase/alpine-buildmaster:x86_64

or,

# make ROLE=worker REQUIRED_PIP="" REQUIRED_APK="curl git make"
docker run --rm -it \
  --name docker_buildworker --hostname buildworker \
  -e BUILDBOT_HOME=/home/alpine/buildbot-config \
  -e PGID=1000 -e PUID=1000 \
  -e REQUIRED_APK="curl git make" \
  -e REQUIRED_PIP="" \
  -e ROLE=worker \
  -e WORKERNAME=buildbot \
  -v config:/home/alpine/buildbot-config \
  woahbase/alpine-buildworker:x86_64

Stop the container with a timeout, (defaults to 2 seconds)

# make stop ROLE=master
docker stop -t 2 docker_buildmaster

# make stop ROLE=worker
docker stop -t 2 docker_buildworker

Removes the container, (always better to stop it first and -f only when needed most)

# make rm ROLE=master
docker rm -f docker_buildmaster

# make rm ROLE=worker
docker rm -f docker_buildworker

Restart the container with

# make restart ROLE=master
docker restart docker_buildmaster

# make restart ROLE=worker
docker restart docker_buildworker

---

Shell access

---

Get a shell inside a already running container,

# make shell ROLE=master
docker exec -it docker_buildmaster /bin/bash

# make shell ROLE=worker
docker exec -it docker_buildworker /bin/bash

set user or login as root,

# make rshell ROLE=master
docker exec -u root -it docker_buildmaster /bin/bash

# make rshell ROLE=worker
docker exec -u root -it docker_buildworker /bin/bash

To check logs of a running container in real time

# make logs ROLE=master
docker exec -it docker_buildmaster \
  tail -f /home/alpine/buildbot-config/$(WORKERNAME)-master/twistd.log

# make logs ROLE=worker
docker exec -it docker_buildworker \
  tail -f /home/alpine/buildbot-config/$(WORKERNAME)-worker/twistd.log

---

Development

---

If you have the repository access, you can clone and build the image yourself for your own system, and can push after.

---

Setup

---

Before you clone the repo, you must have Git, GNU make, and Docker setup on the machine.

git clone https://github.com/woahbase/alpine-buildbot
cd alpine-buildbot

You can always skip installing make but you will have to type the whole docker commands then instead of using the sweet make targets.

---

Build

---

You need to have binfmt_misc configured in your system to be able to build images for other architectures.

Otherwise to locally build the image for your system. [ARCH defaults to x86_64, need to be explicit when building for other architectures.]

# make ARCH=x86_64 ROLE=master build
# sets up binfmt if not x86_64
docker build --rm --force-rm --compress \
  --no-cache=true --pull \
  -f /home/arch/software/woahbase/alpine-buildbot/Dockerfile_x86_64 \
  --build-arg ARCH=x86_64 \
  --build-arg DOCKERSRC=alpine-python3 \
  --build-arg PGID=1000 \
  --build-arg PUID=1000 \
  --build-arg ROLE=master \
  --build-arg USERNAME=woahbase \
  -t woahbase/alpine-buildmaster:x86_64 \
  .

and for the worker,

# make ARCH=x86_64 ROLE=worker build
# sets up binfmt if not x86_64
docker build --rm --force-rm --compress \
  --no-cache=true --pull \
  -f /home/arch/software/woahbase/alpine-buildbot/Dockerfile_x86_64 \
  --build-arg ARCH=x86_64 \
  --build-arg DOCKERSRC=alpine-python3 \
  --build-arg PGID=1000 \
  --build-arg PUID=1000 \
  --build-arg ROLE=worker \
  --build-arg USERNAME=woahbase \
  -t woahbase/alpine-buildworker:x86_64 \
  .

To check if its working..

# make ARCH=x86_64 ROLE=master test
docker run --rm -it \
  --name docker_buildmaster --hostname buildmaster \
  -e WORKERNAME=buildbot \
  -e REQUIRED_PIP="PyMySQL txrequests" \
  -e REQUIRED_APK="curl git" \
  --entrypoint buildbot \
  woahbase/alpine-buildmaster:x86_64 \
  '--version'

# make ARCH=x86_64 ROLE=worker test
docker run --rm -it \
  --name docker_buildworker --hostname buildworker \
  -e WORKERNAME=buildbot \
  --entrypoint buildbot-worker \
  woahbase/alpine-buildworker:x86_64 \
  '--version'

And finally, if you have push access,

# make ARCH=x86_64 ROLE=master push
docker push woahbase/alpine-buildmaster:x86_64

# make ARCH=x86_64 ROLE=worker push
docker push woahbase/alpine-buildworker:x86_64

---

Maintenance

---

Sources at Github. Built at Travis-CI.org (armhf / x64 builds). Images at Docker hub. Metadata at Microbadger.

Maintained by WOAHBase.