Skip to content
Branch: master
Find file History
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
docs Move Remote Builder to correct point in cloud-builders-community repo Jan 8, 2018
examples
Dockerfile Move Remote Builder to correct point in cloud-builders-community repo Jan 8, 2018
README.md `gcloud container builds` --> `gcloud builds`. Jul 24, 2018
cloudbuild.yaml Adds 'tags: [cloud-builders-community]' to all builders (#261) Jul 10, 2019
run-builder.sh Fix copy back Jun 25, 2019

README.md

Container Builder Remote Build Step

Introduction

Architecture Diagram

Some continuous integration workloads require special builder types. You may require things like:

  1. High CPU/Memory machines
  2. Custom image
  3. GPUs attached
  4. Fast or large disks
  5. Machines in a particular network
  6. Pre-emptibility

In these cases you can leverage Container Builder to trigger your builds and manage their workflow but run the actual build steps on an instance with exactly the configuration you need.

How?

When using the remote-builder image, the following will happen:

  1. A temporary SSH key will be created in your Container Builder workspace
  2. A instance will be launched with your configured flags
  3. The workpace will be copied to the remote instance
  4. Your command will be run inside that instance's workspace
  5. The workspace will be copied back to your Container Builder workspace

Usage

In order to use this step, first build the builder:

gcloud builds submit --config=cloudbuild.yaml .

Then, create an appropriate IAM role with permissions to create and destroy Compute Engine instances in this project:

export PROJECT=$(gcloud info --format='value(config.project)')
export PROJECT_NUMBER=$(gcloud projects describe $PROJECT --format 'value(projectNumber)')
export CB_SA_EMAIL=$PROJECT_NUMBER@cloudbuild.gserviceaccount.com
gcloud services enable cloudbuild.googleapis.com
gcloud services enable compute.googleapis.com
gcloud projects add-iam-policy-binding $PROJECT --member=serviceAccount:$CB_SA_EMAIL --role='roles/iam.serviceAccountUser' --role='roles/compute.instanceAdmin.v1' --role='roles/iam.serviceAccountActor'

Then, configure your build step as follows:

steps:
- name: gcr.io/$PROJECT_ID/remote-builder
  env:
    - COMMAND=ls -la

This will launch an instance with the default parameters and then run the command ls -la inside the instance's workspace.

Configuration

The following options are configurable via environment variables passed to the build step in the env parameter:

Options Description Default
COMMAND Command to run inside the remote workspace None, must be set
USERNAME Username to use when logging into the instance via SSH admin
REMOTE_WORKSPACE Location on remote host to use as workspace /home/${USERNAME}/workspace/
INSTANCE_NAME Name of the instance that is launched builder-$UUID
ZONE Compute zone to launch the instance in us-central1-f
INSTANCE_ARGS Parameters to the instance creation command. For a full list run gcloud compute instances create --help --preemptible

To give it a try, see the examples directory.

This is not an official Google product.

Trade-offs

  1. Paying for builder + VM
  2. Spin up time of VM increases build time
You can’t perform that action at this time.