Skip to content

Commit

Permalink
Jobs based on custom runners: build environment docs and playbook
Browse files Browse the repository at this point in the history
To run basic jobs on custom runners, the environment needs to be
properly set up.  The most common requirement is having the right
packages installed.

The playbook introduced here covers the QEMU's project s390x and
aarch64 machines.  At the time this is being proposed, those machines
have already had this playbook applied to them.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210630012619.115262-3-crosa@redhat.com>
Message-Id: <20210709143005.1554-3-alex.bennee@linaro.org>
  • Loading branch information
clebergnu authored and stsquad committed Jul 12, 2021
1 parent 8942605 commit d40cd72
Show file tree
Hide file tree
Showing 4 changed files with 159 additions and 0 deletions.
40 changes: 40 additions & 0 deletions docs/devel/ci.rst
Expand Up @@ -30,3 +30,43 @@ The GitLab CI jobs definition for the custom runners are located under::
Custom runners entail custom machines. To see a list of the machines
currently deployed in the QEMU GitLab CI and their maintainers, please
refer to the QEMU `wiki <https://wiki.qemu.org/AdminContacts>`__.

Machine Setup Howto
-------------------

For all Linux based systems, the setup can be mostly automated by the
execution of two Ansible playbooks. Create an ``inventory`` file
under ``scripts/ci/setup``, such as this::

fully.qualified.domain
other.machine.hostname

You may need to set some variables in the inventory file itself. One
very common need is to tell Ansible to use a Python 3 interpreter on
those hosts. This would look like::

fully.qualified.domain ansible_python_interpreter=/usr/bin/python3
other.machine.hostname ansible_python_interpreter=/usr/bin/python3

Build environment
~~~~~~~~~~~~~~~~~

The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will
set up machines with the environment needed to perform builds and run
QEMU tests. This playbook consists on the installation of various
required packages (and a general package update while at it). It
currently covers a number of different Linux distributions, but it can
be expanded to cover other systems.

The minimum required version of Ansible successfully tested in this
playbook is 2.8.0 (a version check is embedded within the playbook
itself). To run the playbook, execute::

cd scripts/ci/setup
ansible-playbook -i inventory build-environment.yml

Please note that most of the tasks in the playbook require superuser
privileges, such as those from the ``root`` account or those obtained
by ``sudo``. If necessary, please refer to ``ansible-playbook``
options such as ``--become``, ``--become-method``, ``--become-user``
and ``--ask-become-pass``.
2 changes: 2 additions & 0 deletions scripts/ci/setup/.gitignore
@@ -0,0 +1,2 @@
inventory

116 changes: 116 additions & 0 deletions scripts/ci/setup/build-environment.yml
@@ -0,0 +1,116 @@
# Copyright (c) 2021 Red Hat, Inc.
#
# Author:
# Cleber Rosa <crosa@redhat.com>
#
# This work is licensed under the terms of the GNU GPL, version 2 or
# later. See the COPYING file in the top-level directory.
#
# This is an ansible playbook file. Run it to set up systems with the
# environment needed to build QEMU.
---
- name: Installation of basic packages to build QEMU
hosts: all
tasks:
- name: Check for suitable ansible version
delegate_to: localhost
assert:
that:
- '((ansible_version.major == 2) and (ansible_version.minor >= 8)) or (ansible_version.major >= 3)'
msg: "Unsuitable ansible version, please use version 2.8.0 or later"

- name: Update apt cache / upgrade packages via apt
apt:
update_cache: yes
upgrade: yes
when:
- ansible_facts['distribution'] == 'Ubuntu'

- name: Install basic packages to build QEMU on Ubuntu 18.04/20.04
package:
name:
# Originally from tests/docker/dockerfiles/ubuntu1804.docker
- ccache
- gcc
- gettext
- git
- glusterfs-common
- libaio-dev
- libattr1-dev
- libbrlapi-dev
- libbz2-dev
- libcacard-dev
- libcap-ng-dev
- libcurl4-gnutls-dev
- libdrm-dev
- libepoxy-dev
- libfdt-dev
- libgbm-dev
- libgtk-3-dev
- libibverbs-dev
- libiscsi-dev
- libjemalloc-dev
- libjpeg-turbo8-dev
- liblzo2-dev
- libncurses5-dev
- libncursesw5-dev
- libnfs-dev
- libnss3-dev
- libnuma-dev
- libpixman-1-dev
- librados-dev
- librbd-dev
- librdmacm-dev
- libsasl2-dev
- libsdl2-dev
- libseccomp-dev
- libsnappy-dev
- libspice-protocol-dev
- libssh-dev
- libusb-1.0-0-dev
- libusbredirhost-dev
- libvdeplug-dev
- libvte-2.91-dev
- libzstd-dev
- make
- python3-yaml
- python3-sphinx
- python3-sphinx-rtd-theme
- ninja-build
- sparse
- xfslibs-dev
state: present
when:
- ansible_facts['distribution'] == 'Ubuntu'

- name: Install packages to build QEMU on Ubuntu 18.04/20.04 on non-s390x
package:
name:
- libspice-server-dev
- libxen-dev
state: present
when:
- ansible_facts['distribution'] == 'Ubuntu'
- ansible_facts['architecture'] != 's390x'

- name: Install basic packages to build QEMU on Ubuntu 18.04
package:
name:
# Originally from tests/docker/dockerfiles/ubuntu1804.docker
- clang
when:
- ansible_facts['distribution'] == 'Ubuntu'
- ansible_facts['distribution_version'] == '18.04'

- name: Install basic packages to build QEMU on Ubuntu 20.04
package:
name:
# Originally from tests/docker/dockerfiles/ubuntu2004.docker
- clang-10
- genisoimage
- liblttng-ust-dev
- libslirp-dev
- netcat-openbsd
when:
- ansible_facts['distribution'] == 'Ubuntu'
- ansible_facts['distribution_version'] == '20.04'
1 change: 1 addition & 0 deletions scripts/ci/setup/inventory.template
@@ -0,0 +1 @@
localhost

0 comments on commit d40cd72

Please sign in to comment.