baur manages builds and artifacts in mono repositories
Clone or download
fho docs: update readme
Remove the "go get" instruction from the README, only suggest to install baur
from a release archive. When baur is installed via "go get" the intern version
variable can not be set to the current commit.
When somebody submits later a bugreport it won't be possible to figure out from
which commit his baur version was build which makes debugging much harder.

Also add subsection in Quickstart to make it easier readable and change the
wording slightly.
Latest commit f95242d Dec 14, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci ci: set GOFLAGS=-mod=vendor env variable Nov 19, 2018
build build: store build results in the postgresql storage Jun 19, 2018
cfg cfg: remove redundant if error checks Dec 6, 2018
cmd/baur move main() for dep binary to cmd/baur, cobra cmds to command/ Apr 23, 2018
command cmd: deduplicate the passed appnames and paths Dec 14, 2018
digest group local packages in import statement Oct 15, 2018
exec exec: improve log output Nov 20, 2018
format format: simplify writeHeaders() Oct 30, 2018
fs resolve: fail with error if GOROOT directory does not exist Nov 19, 2018
git resolve: improve error message if GitFiles path did not match anything Nov 27, 2018
log log: colorize the ERROR: prefix in log messages Oct 31, 2018
prettyprint add a prettyprint package Jun 19, 2018
resolve adapt formatting to gofmt -s suggestions Dec 6, 2018
storage adapt formatting to gofmt -s suggestions Dec 6, 2018
term fix minor golint complaints Apr 26, 2018
testutils resolver: rework gosource resolver Nov 14, 2018
upload upload: add local file copy uploader Nov 12, 2018
vendor go: update vendor libraries Dec 14, 2018
version version: support to parse partial version string Nov 19, 2018
.editorconfig add .editorconfig file Nov 12, 2018
.gitignore move main() for dep binary to cmd/baur, cobra cmds to command/ Apr 23, 2018
.gometalinter.json gometalinter: enable gofmt -s linter Dec 6, 2018
LICENSE inputs: implement go source file discovery for inputs Aug 8, 2018
Makefile make: create .sha256 checksum files for releases Dec 4, 2018
README.md docs: update readme Dec 14, 2018
app.go app: allocate map with required capacity in pathsToUniqFiles() Dec 14, 2018
baur.go fix minor golint complaints Apr 26, 2018
build.go build: rename outstanding status to pending Nov 27, 2018
buildoutput.go upload: consolidate upload packages Nov 12, 2018
buildoutputbackends.go upload: consolidate upload packages Nov 12, 2018
dockerartifact.go upload: consolidate upload packages Nov 12, 2018
dockerimg.go inputs: rename URL() to URI() Oct 26, 2018
file.go inputs: refactor build input resolver Nov 4, 2018
fileartifact.go upload: add local file copy uploader Nov 12, 2018
go.mod go: update vendor libraries Dec 14, 2018
go.sum go: update vendor libraries Dec 14, 2018
repository.go resolve: fail with error if GOROOT directory does not exist Nov 19, 2018
ver set version to 0.13 Dec 4, 2018

README.md

baur CircleCI Go Report Card

baur png

baur is a build management tool for Git based monolithic repositories.

Developers specify configuration in a TOML file:

  • what the inputs for the build process are
  • which command must be run to build the application
  • which outputs are generated by the build
  • where the output artifacts should be uploaded to

baur detects which applications need to be built by calculating a digest of the application's build inputs and comparing them with digests from previous builds. If a build with the same build input digest was done previously, a build is not necessary. If the build input digest differs from the stored ones, baur runs a user-specified command to build the application.

asciicast

Content

Example Repository

You can find an example monorepository that uses baur at: https://github.com/simplesurance/baur-example. Please follow the quickstart guide for the example repository.

Quickstart

This chapter describes a quick way to setup baur for experimenting with it without using the Example Repository.

For setting it up in a production environment see the chapter Production Setup.

Installation

Install baur by downloading a release archive from the Release Page and extracting baur into your $PATH

Setup

baur uses a PostgreSQL database to store information about builds, the quickest way to setup a PostgreSQL for local testing is with docker:

docker run -p 5432:5432 -e POSTGRES_DB=baur postgres:latest

Afterwards your are ready to create your baur repository configuration.

In the root directory of your Git repository run:

baur init repo

Next, follow the printed steps to create the database and application config files.

First Steps

Some commands to start with are:

command action
baur ls apps List applications in the repository with their build status
baur build Build all applications with pending builds, upload their artifacts and records the result
baur ls all List recorded builds
baur show currency-service Show information about an application called "currency-service"
baur ls inputs --digests shop-api Show inputs with their digests of an application called "shop-api"

To get more information about a command pass the --help parameter to baur.

Key Features:

  • Detecting Changed Applications The inputs of applications are specified in the .app.toml config file for each application. baur calculates a SHA384 digest for all inputs and stores the digest in the database when an application was built and its artifacts uploaded (baur build). The digest is used to detect if a previous build for the same input files exists. If a build exist, the application does not need to be rebuilt, otherwise a build is done. This allows specific applications to be run through a CI pipeline that changed in a given commit. This approach also prevents applications from unnecessarily being rebuilt if commits are reverted in the Git repository.

  • Artifact Upload to S3 and Docker Registries baur supports uploading built File artifacts to S3 buckets and produced docker images to docker registries.

  • Managing Applications baur can be used as management tool in monorepositories to list applications and find their locations.

  • CI Optimized: baur is aimed to be run in CI environments and allows to print relevant output in CSV format to be easily parsed by scripts.

  • Build Statistics: The data that baur stores in its PostgreSQL database enables the graphing of statistics about builds such as which application changes most, which produces the biggest build artifacts, which build runs the longest.

Why?

Monorepositories come with new challenges in CI environments. When a Git repository contains only a single application, every commit can trigger the whole CI workflow of builds, checks, tests and deployments. This is not effective in Monorepositories when a repository can contain hundreds of different applications. Running the whole CI flow for all applications on every commit takes a lot of time and wastes resources. Therefore the CI environment has to detect which applications changed to only run those through the CI flow.

When all build inputs per applications are isolated in directories and CI artifacts are always produced for the reference branches, the git-history can be used to determine which files changed. Simple Shell-Scripts or the mbt build tool can be used for it.

When applications in the monorepository share libraries (Protobuf or other files), standard solutions are not sufficient anymore. Full-fledged build tools like Bazel and pants exist to address those issues in Monorepositories but they come with complex configurations and complex usage. Developers have to get used to defining the build steps in those tools instead of relying on their more favorite build tools.

baur solves these problems by concentrating on tracking build inputs and build outputs while enabling to use the build tool of your choice.

Documentation

Documentation is available in the wiki.

Graphana Dashboard

Graphana baur Dashboard

The dashboard is available at: https://grafana.com/dashboards/8835

Project Status

baur is used in production CI setups since the first version. It's not considered as API-Stable yet, interface breaking changes can happen in any release.

We are happy to receive Pull Requests, Feature Requests and Bug Reports to further improve baur.