Skip to content
Browse files

Add support for testing against a specific Xcode version (#609)

* Add support for testing against a specific Xcode version

* Protect against cache poisoning by including the Xcode version and path in the host_platform_remote_properties_override
  • Loading branch information...
philwo committed Apr 12, 2019
1 parent 3291154 commit 380f1e63ec7c327dc9d326574b17f3fdb5dc3a9e
Showing with 79 additions and 3 deletions.
  1. +30 −2 buildkite/
  2. +49 −1 buildkite/
@@ -214,7 +214,7 @@ tasks:
- "..."

In this case we can omit the `platforms` field since there is a 1:1 mapping between tasks and platforms. Consequently, the format looks almost identical to the old one:
In this case we can omit the `platform` field since there is a 1:1 mapping between tasks and platforms. Consequently, the format looks almost identical to the old one:

@@ -235,7 +235,7 @@ The CI uses [Bazelisk]( to support older vers
bazel: 0.20.0
- "..."
@@ -251,6 +251,34 @@ platforms:
In this example the jobs on Windows and MacOS would use 0.20.0, whereas the job on Ubuntu would run 0.18.0.
Please see the [Bazelisk documentation]( for a list of all supported version values.

### macOS: Using a specific version of Xcode

We upgrade the CI machines to the latest version of Xcode shortly after it is released and this
version will then be used as the default Xcode version. If required, you can specify a fixed Xcode
version to test against in your pipeline config.

The general policy is to *not* specify a fixed Xcode version number, so that we can update the
default version more easily and don't have to update every single CI configuration file out there.

However, if you know that you need to test against multiple versions of Xcode or that newer versions
frequently break you, you can use this feature.

# Test against the latest released Xcode version.
- "..."
# Ensure that we're still supporting Xcode 10.1.
platform: macos
xcode_version: "10.1"
- "..."

Take care to quote the version number, otherwise YAML will interpret it as a floating point number.

### Running Buildifier on CI

For each pipeline you can enable [Buildifier]( to check all WORKSPACE, BUILD, BUILD.bazel and .bzl files for lint warnings and formatting violations. Simply add the following code to the top of the particular pipeline configuration:
@@ -393,6 +393,9 @@
# The platform used for various steps (e.g. stuff that formerly ran on the "pipeline" workers).
DEFAULT_PLATFORM = "ubuntu1804"

XCODE_VERSION_REGEX = re.compile(r"^\d+\.\d+(\.\d+)?$")

@@ -641,6 +644,37 @@ def execute_commands(
tmpdir = tempfile.mkdtemp()
sc_process = None
# Activate the correct Xcode version on macOS machines.
if platform == "macos":
# Get the Xcode version from the config.
xcode_version = task_config.get("xcode_version", DEFAULT_XCODE_VERSION)
print_collapsed_group("Activating Xcode {}...".format(xcode_version))

# Ensure it's a valid version number.
if not isinstance(xcode_version, str):
raise BuildkiteException(
"Version number '{}' is not a string. Did you forget to put it in quotes?".format(
if not XCODE_VERSION_REGEX.match(xcode_version):
raise BuildkiteException(
"Invalid Xcode version format '{}', must match the format X.Y[.Z].".format(

# Check that the selected Xcode version is actually installed on the host.
xcode_path = "/Applications/Xcode {}.app".format(xcode_version)
if not os.path.exists(xcode_path):
raise BuildkiteException("Xcode not found at '{}'.".format(xcode_path))

# Now activate the specified Xcode version and let it install its required components.
# The CI machines have a sudoers config that allows the 'buildkite' user to run exactly
# these two commands, so don't change them without also modifying the file there.
execute_command(["/usr/bin/sudo", "/usr/bin/xcode-select", "--switch", xcode_path])
execute_command(["/usr/bin/sudo", "/usr/bin/xcodebuild", "-runFirstLaunch"])

# If the CI worker runs Bazelisk, we need to forward all required env variables to the test.
# Otherwise any integration test that invokes Bazel (=Bazelisk in this case) will fail.
test_env_vars = ["LocalAppData"] if platform == "windows" else ["HOME"]
@@ -1029,6 +1063,20 @@ def remote_caching_flags(platform):
return []

platform_cache_key = [platform.encode("utf-8")]
if platform == "macos":
# macOS version:
platform_cache_key.append(subprocess.check_output(["/usr/bin/sw_vers", "-productVersion"]))
# Path to Xcode:
platform_cache_key.append(subprocess.check_output(["/usr/bin/xcode-select", "-p"]))
# Xcode version:
platform_cache_key.append(subprocess.check_output(["/usr/bin/xcodebuild", "-version"]))

platform_cache_digest = hashlib.sha256()
for key in platform_cache_key:

flags = [
# TODO(ulfjack): figure out how to resolve
@@ -1037,7 +1085,7 @@ def remote_caching_flags(platform):
'--host_platform_remote_properties_override=properties:{name:"platform" value:"%s"}'
% platform,
% platform_cache_digest.hexdigest(),

if platform == "macos":

0 comments on commit 380f1e6

Please sign in to comment.
You can’t perform that action at this time.