Skip to content

Latest commit

 

History

History
96 lines (54 loc) · 5.93 KB

BUILD.md

File metadata and controls

96 lines (54 loc) · 5.93 KB
Raw

AppScope Build System

We use simple Makefiles to build the binaries and run the tests. GitHub Workflows/Actions handle CI. This document explans aspects of both.

Building

To get started building the code on a Linux machine you'll need a handful of dependencies first.

  • At a minimum, you'll need git, make, docker, docker-compose, and jq.
  • If you are going to build the code locally without our container-based build system, you will also need gcc, autoconf, automake, libtool, cmake, and lcov.

With the required tools in place, git clone the repository, cd into the working directory, and use the following commands.

make build

This will run docker build first to build our "appscope-builder" container image. It tries to pull the existing image from the project's image repository to speed things up but this step can take a while the first time.

Once the builder image is ready, it runs make all test in the container. The local working directory is mounted into the container so the result should end up the in the same place as if you'd run that command locally.

You can override the command setting CMD like below.

make build CMD="make coreall"

By default, this will build for the local CPU architecture; uname -m. We support building for other architectures via QEMU and binfmt by setting ARCH like below. Note we only support x86_64 and aarch64 currently

make build ARCH=aarch64

The resulting binaries are listed below. $ARCH will be x86_64 or aarch64.

  • lib/linux/$ARCH/libscope.so
  • bin/linux/$ARCH/scope

make run

This uses the same builder container image as make build but drops into a shell instead of running a command in the container. We often use this to keep the builder container up and run the local commands make all, make test, etc. without having to restart.

make all

This starts a build of the core (the library and loaders) and CLI. Use make coreall if you'd like to skip building the CLI. Use make scope for the opposite.

These commands run the build in the local environment using locally installed dependencies. This is only supported on Ubuntu 18.04 and will generated a warning if it detects another local operating system. You are encoraged to use make build instead.

make test

This runs the unit-tests for the core and CLI. As above, use make coretest or make clitest to only run one test suite or the other.

make clean

This removes built binaries and intermediate files generated by the build and test steps. Use make coreclean or make cliclean to only clean the corresponding parts of the project.

Testing

The make test target runs unit tests for the core and CLI. CLI tests are typical Go tests mixed in with the code in *_test.go files. Tests for the core are a mix of C programs using cmocka and shell scripts all in the test/ folder.

We maintain a suite of container images used to perform integration tests. They're kept intest/integration. See the README there for details.

CI Workflows

We use GitHub Actions to build, test, package, and distribute AppScope automatically. They're defined in .github/workflows/.

Caching

Running the build, test, and package stages in the CI workflows now takes logn enough that we've had to spend some effort to cache the results of some steps.

Contribs

Some of the contrib libraries build into a separate folder and don't make any changes in the source folder; cmocka, funchook, openssl, and pcre2. In CI, we are caching their build folders using a key generated from a hash of the source files. The resulting build performance improvement is significant.

Go Modules

The Makefile for the CLI sents a few environment variables that cause Go to store downloaded modules, binary programs, and other cacheable content in cli/.go*/ subdirectories. This was done initially so the cached content was accessible in our build containers and not lost when the container exits. It was also done to avoid breaking the user's $HOME/.go/ setup when running cross-architecture builds.

In CI, we are caching the contents of these subdirectories to speed up the build and unit-test stages.

Test Containers

The container images for the integration tests are pulled from the project's package repository at GitHub. They're named appscope-test-* and tagged with either x86_64 or aarch64 depending on the architecture they're run on. The image is rebuilt using the current Dockerfile and other content in case anything has changed on the branch being built. The resulting image is pushed back up to GitHub only for push actions on the default branch.

Notes

Various aspects of the build system that didn't fit else where are listed below.

  • We use the top-level Makefile to build the library and loaders (i.e. the C code) running there in the top-level directory. We don't recursively cd down into src/ or test/ as some other build system do.

  • The top-level Makefile detects the local OS and includes os/linux/Makefile or os/macOS/Makefile. The logic aborts if another OS is detected. It also warns that the MacOS build and Linux builds on anything other than Ubuntu 18.04 are unsupported.

    Note: We only really support building on Linux. The MacOS build support has essentially been abandoned.

  • The os/linux/Makefile includes architecture-specific os/linux/x86_64.mk or os/linux/aarch64.mk files.

  • The top-level Makefile has a cli% pattern rule and relays to corresponding targets in cli/Makefile. It cd's into/cli to run those targets. We also have a scope target as a shortcut to build the CLI from the top-level directory.