Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
16 contributors

Users who have contributed to this file

@michelle-luna @marcomorain @appplemac @endocrimes @smart-alek @teesloane @zzak @drazisil @sararead @rosieyohannan @ndintenfass @li3n3 @eric-hu @glenjamin @gmegidish @ceyhun
673 lines (530 sloc) 22.6 KB
layout title short-title categories description order
classic-docs
Testing iOS Applications on macOS
Testing iOS Applications on macOS
platforms
Testing iOS applications on macOS
30

This document describes how to set up and customize testing for an iOS application with CircleCI in the following sections:

  • TOC {:toc}

Note: There is also documentation for [an iOS example project]({{ site.baseurl}}/2.0/ios-tutorial/) and [getting started on MacOS]({{ site.baseurl }}/2.0/hello-world-macos/).

Overview

{:.no_toc}

CircleCI offers support for building and testing iOS and macOS projects. Refer to the manifest of the software installed on CircleCI macOS build images in the Using a macOS Build Image document.

macOS Build Containers

Each macos job is run a fresh container, running macOS. We build a new container each time a new version of Xcode is released by Apple. The contents of a particular build container remain unchanged (in very exceptional circumstances we might be forced to re-build a container). Our goal is to keep your builds environement stable, and to allow you to opt-in to newer containers by setting the xcode key in your config.yml file.

We announce the availability of new macOS containers in the annoucements section of our Discuss site.

Supported Xcode Versions

The currently available Xcode versions are:

Getting Started

Select a macOS project you would like to build on the Add Projects page of the CircleCI application. Note: Changing build environment is no longer needed in 2.0. If your project is not listed as macOS, choose Linux project and then select macOS in the Operating System section.

Basic Setup

After enabling macOS builds for your project, share the scheme that is going to be built on CircleCI so that CircleCI runs the correct build actions. Complete the following steps to share an existing scheme in Xcode:

  1. Choose Product > Scheme > Manage Schemes.
  2. Select the Shared option for the scheme to share, and click Close.
  3. Choose Source Control > Commit.
  4. Select the Shared Data folder.
  5. Enter your commit message in the text field.
  6. Select the "Push to remote" option (if your project is managed with Git).
  7. Click the Commit Files button. A new .xcscheme file is located in the xcshareddata/xcschemes folder under your Xcode project.
  8. Commit this file to your git repository so that CircleCI can access it.

Simple projects should run with minimal configuration. You can find an example of a minimal config in the [iOS Project Tutorial]({{ site.baseurl }}/2.0/ios-tutorial/).

Best Practices

{:.no_toc}

In addition to the basic setup steps, it is best practice to include downloading CocoaPods specs from the CircleCI mirror (up to 70% faster) and linting the Swift code together with the build-and-test job:

# .circleci/config.yml
version: 2
jobs:
  build-and-test:
    macos:
      xcode: "10.2.0"
    environment:
      FL_OUTPUT_DIR: output
    steps:
      - checkout
      - run:
          name: Fetch CocoaPods Specs
          command: |
            curl https://cocoapods-specs.circleci.com/fetch-cocoapods-repo-from-s3.sh | bash -s cf
      - run:
          name: Install CocoaPods
          command: pod install --verbose

      - run:
          name: Build and run tests
          command: fastlane scan
          environment:
            SCAN_DEVICE: iPhone 8
            SCAN_SCHEME: WebTests

      - store_test_results:
          path: output/scan
      - store_artifacts:
          path: output

workflows:
  version: 2
  build-and-test:
    jobs:
      - build-and-test

Advanced Setup

For advanced setup, it is possible to run a lint job together with your build and test job, and potentially also run tools like Danger.

The recommended configuration can be extended to add a lint job and a Danger job as follows:

version: 2
jobs:
  build-and-test:
  swiftlint:
    docker:
      - image: dantoml/swiftlint:latest
    steps:
      - checkout
      - run: swiftlint lint --reporter junit | tee result.xml
      - store_artifacts:
          path: result.xml
      - store_test_results:
          path: result.xml
  danger:
    docker:
      - image: dantoml/danger:latest
    steps:
      - checkout
      - run: danger

workflows:
  version: 2
  build-test-lint:
    jobs:
      - swiftlint
      - danger
      - build-and-test

Using Fastlane

Fastlane is a set of tools for automating the build and deploy process of mobile apps. We encourage the use of Fastlane on CircleCI as that allows for parity of build / deploy locally and on CircleCI, and simplifies the setup process.

Adding a Gemfile

{:.no_toc}

It is recommended to add a Gemfile to your repository to make sure that the same version of Fastlane is used both locally and on CircleCI. The simplest Gemfile could look like this:

# Gemfile
source "https://rubygems.org"
gem 'fastlane'

After you have created a Gemfile locally, you will need to run bundle install and check both Gemfile and Gemfile.lock into your repository.

Setting up Fastlane for use on CircleCI

{:.no_toc}

When using Fastlane in your CircleCI project, we recommend adding the following to your Fastfile:

# fastlane/Fastfile

...
platform :ios do
  before_all do
    setup_circle_ci
  end
  ...
end

The setup_circle_ci Fastlane action must be in the before_all block to perform the following actions:

  • Create a new temporary keychain for use with Fastlane Match (see the code signing section for more details).
  • Switch Fastlane Match to readonly mode to make sure CI does not create new code signing certificates or provisioning profiles.
  • Set up log and test result paths to be easily collectible.

Example Configuration for Using Fastlane on CircleCI

{:.no_toc}

A basic Fastlane configuration that can be used on CircleCI is as follows:

# fastlane/Fastfile
default_platform :ios

platform :ios do
  before_all do
    setup_circle_ci
  end

  desc "Runs all the tests"
  lane :test do
    scan
  end

  desc "Ad-hoc build"
  lane :adhoc do
    match(type: "adhoc")
    gym(export_method: "ad-hoc")
  end
end

This configuration can be used with the following CircleCI config file:

# .circleci/config.yml
version: 2
jobs:
  build-and-test:
    macos:
      xcode: "10.2.0"
    environment:
      FL_OUTPUT_DIR: output
      FASTLANE_LANE: test
    shell: /bin/bash --login -o pipefail
    steps:
      - checkout
      - run: bundle install
      - run:
          name: Fastlane
          command: bundle exec fastlane $FASTLANE_LANE
      - store_artifacts:
          path: output
      - store_test_results:
          path: output/scan

  adhoc:
    macos:
      xcode: "10.2.0"
    environment:
      FL_OUTPUT_DIR: output
      FASTLANE_LANE: adhoc
    shell: /bin/bash --login -o pipefail
    steps:
      - checkout
      - run: bundle install
      - run:
          name: Fastlane
          command: bundle exec fastlane $FASTLANE_LANE
      - store_artifacts:
          path: output

workflows:
  version: 2
  build-test-adhoc:
    jobs:
      - build-and-test
      - adhoc:
          filters:
            branches:
              only: development
          requires:
            - build-and-test

The environment variable FL_OUTPUT_DIR is the artifact directory where FastLane logs should be stored. Use this to set the path in the store_artifacts step to automatically save logs such as Gym and Scan.

Reducing Testing Time

By default, Fastlane Scan generates test output reports in html and junit formats. If your tests are taking a long time and you do not need these reports, consider disabling them by altering the output_type parameter as described in the fastlane docs.

Using CocoaPods

{:.no_toc}

If you are using CocoaPods, then we recommend that you check your Pods directory into source control. This will ensure that you have a deterministic, reproducible build.

Supported Build and Test Tools

In CircleCI 2.0 it is possible to customize your build as needed to satisfy almost any iOS build and test strategy.

XCTest-based tools

{:.no_toc}

The following test tools are known to work well on CircleCI (though many others should work just fine):

Other Tools

{:.no_toc}

Popular iOS testing tools like Appium and Frank should also work normally and are installed and called using run commands.

Pre-Starting the Simulator

{:.no_toc}

Pre-start the iOS simulator before building your application to make sure that the simulator is booted in time. Doing so generally reduces the number of simulator timeouts observed in builds.

To pre-start the simulator, add the following to your config.yml file, assuming that you are running your tests on an iPhone 7 simulator with iOS 10.2:

    steps:
      - run:
          name: pre-start simulator
          command: xcrun instruments -w "iPhone 7 (10.2) [" || true

Note: the [ character is necessary to uniquely identify the iPhone 7 simulator, as the phone + watch simulator is also present in the build image:

  • iPhone 7 (10.2) [<uuid>] for the iPhone simulator.
  • iPhone 7 Plus (10.2) + Apple Watch Series 2 - 42mm (3.1) [<uuid>] for the phone + watch pair.

Creating a config.yml File

{:.no_toc}

The most flexible means to customize your build is to add a .circleci/config.yml file to your project, which allows you to run arbitrary bash commands at various points in the build process. See the [Configuring CircleCI]( {{ site.baseurl }}/2.0/configuration-reference/) document for a detailed discussion of the structure of the config.yml file. Note: A number of options in the document will not work for macOS builds.

Installing Custom Packages

{:.no_toc}

Homebrew is pre-installed on CircleCI, so you can simply use brew install to add nearly any dependency required in your build VM. Here's an example:

    steps:
      - run:
          name: Install cowsay
          command: brew install cowsay
      - run:
          name: cowsay hi
          command: cowsay Hi!

It is also possible to use the sudo command if necessary to perform customizations outside of Homebrew.

Using Custom Ruby Versions

{:.no_toc}

Our macOS containers contain multiple versions of Ruby. The default version is the system-installed Ruby. The containers also include the latest stable versions of Ruby at the time that the container is built. We determine the stable versions of Ruby using the Ruby-Lang.org downloads page. The version of Ruby that are installed in each image are listed in the software manifests of each container.

If you want to run steps with a version of Ruby that is listed as "available to chruby" in the manifest, then you can use chruby to do so. To activate chruby, you must change the shell parameter of your job to be a login shell (adding --login).

version: 2
jobs:
  build:
    macos:
      xcode: "10.2.0"
    shell: /bin/bash --login -eo pipefail

To specify a version of Ruby to use, there are two options. You can create a file named .ruby-version and commit it to your repository, as documented by chruby. If you do not want to commit a .ruby-version file to source control, then you can create the file from a job step:

run:
  name: Set Ruby Version
  command:  echo "ruby-2.4" > ~/.ruby-version # Replace 2.4 with the specific version of Ruby here.

Note: The version of Ruby that you select must be one of the versions listed in the software manifests of your macOS container.

To run a job with a version of Ruby that is not pre-installed, you must install the required version of Ruby. We use the ruby-install tool to install the required version. After the install is complete, you can select it using the technique above.

Using Custom Versions of CocoaPods and Other Ruby Gems

{:.no_toc}

To make sure the version of CocoaPods that you use locally is also used in your CircleCI builds, we suggest creating a Gemfile in your iOS project and adding the CocoaPods version to it:

source 'https://rubygems.org'

gem 'cocoapods', '= 1.3.0'

Then you can install these using bundler:

{% raw %}

    steps:
      - restore_cache:
          key: 1-gems-{{ checksum "Gemfile.lock" }}

      - run: bundle check || bundle install --path vendor/bundle

      - save_cache:
          key: 1-gems-{{ checksum "Gemfile.lock" }}
          paths:
            - vendor/bundle

{% endraw %}

You can then ensure you are using those, by prefixing commands with bundle exec:

    steps:
      - run: bundle exec pod install

Configuring Deployment

After you have a signed app you are ready to configure deployment. Distributing the app is easy with one of the following:

Then you should set up environment variables for your service of choice:

Hockey App

{:.no_toc}

  1. Log in to Hockey app and create a new API token on the Tokens page. Your token will need at least upload permission to upload new builds to Hockey App.

  2. Give your new API token a name specific to CircleCI such as "CircleCI Distribution".

  3. Copy the token, and log into CircleCI and go to the Project Settings page for your app.

  4. Create a new Environment Variable with the name HOCKEY_APP_TOKEN and paste the token as the value. You can now access this token in any job.

Beta By Crashlytics

{:.no_toc}

  1. Log in to Fabric.io and visit your organization's settings page. ![Fabric.io loging image]( {{ site.baseurl }}/assets/img/docs/fabric-org-settings-page.png)

  2. Click your organization (CircleCI in the image above), and click the API key and Build Secret links to reveal the items. ![Fabric.io org image]( {{ site.baseurl }}/assets/img/docs/fabric-api-creds-page.png)

  3. Navigate to your App's Project Settings page in the CircleCI app, and under Environment Variables add two new items named CRASHLYTICS_API_KEY and CRASHLYTICS_SECRET, with the values you find on Crashlytics website.

TestFairy

{:.no_toc}

To set up your app on TestFairy, follow these steps:

![TestFairy preferences image]( {{ site.baseurl }}/assets/img/docs/testfairy-open-preferences.png)

  1. On the TestFairy dashboard, navigate to the Preferences page.
  2. On the Preferences page, go to the API Key section.
  3. Copy your API key and go to your application's project settings within the CircleCI application.
  4. To deploy, add a job to your configuration using fastlane or curl (example below).

{% raw %}

jobs:
  build:
    #  insert build code here...
  deploy:
    steps:
      - checkout
      - run:
          name: Deploy to TestFairy
          command: |
            curl \
              -A "CircleCI 2.0" \
              -F api_key="$TESTFAIRY_API_KEY" \
              -F comment="CircleCI build $CIRCLE_BUILD_URL" \
              -F file=@path/to/ipafile.ipa \
              https://upload.testfairy.com/api/upload/

workflows:
  version: 2
  build-and-deploy:
    jobs:
      - build
      - deploy:
        requires:
          - build
        filters:
          branches:
            only: master

{% endraw %}

For a complete list of available options, please visit the TestFairy Upload API documentation

Resolving Common Simulator Issues

{:.no_toc}

A series of simulator-related issues are known to happen on some projects. Here are the most frequent of those:

  • Xcode version is not available. We install a few different versions of Xcode in each build image and keep those updated with the latest point releases. For version 10.0.0, you must specify the full version, down to the point release number. However, to use the latest Xcode 8.3, for example, which is 8.3.3, it is sufficient to specify 8.3 in your config.yml. If a newer point release of 8.3 comes out, we will make that one available under the same 8.3 version on CircleCI.

  • Dependency version mismatches. If you see that the version of the dependencies used in a job are not the expected ones, please try rebuilding without cache — chances are an older dependency got stuck in the cache and is not allowing for the newer version to get installed.

  • Cryptic compilation errors. If you see compile-time errors that do not really make sense, please check if the version of Xcode you are using in your build is the same one you are using locally. When the config.yml of the project does not specify an Xcode version, we default to an older Xcode which might not support the necessary features.

  • Ruby segfaults. We have seen cases where some of the Ruby gems used during a job would produce a segmentation fault in Ruby. This might happen because of the mismatch of Ruby version used to build the gem and the Ruby version used to run it. Please make sure that the Ruby version used locally is the same as the one used on CircleCI. You can install a newer version Ruby in the container by following this guide.

  • Inconsistent timeouts during test runs. If your UI tests are timing out, try running them before the rest of your tests. You can also try using the raw xcodebuild command or the xctool command. Some issues are only present in one of these tools.

  • Errors while installing code signing certificates. Please check out the iOS Code Signing document.

  • Many iOS app developers use tools that generate substantial amounts of code. In such cases CircleCI may not correctly detect the Xcode workspace, project, or scheme. Instead, you can specify these through environment variables.

Constraints on macOS-based Builds

{:.no_toc}

Splitting tests between parallel containers on macOS is currently not supported. We suggest using a workflow with parallel jobs to build with different Xcode versions, or a workflow with parallel jobs to run different test targets. Please check [this doc]({{ site.baseurl }}/2.0/workflows/#workflows-configuration-examples) for examples of workflows with parallel jobs.

Sample Configuration with Multiple Executor Types (macOS + Docker)

It is possible to use multiple executor types in the same workflow. In the following example each push of an iOS project will be built on macOS, and additional iOS tools (SwiftLint and Danger) will be run in Docker.

{% raw %}

version: 2
jobs:
  build-and-test:
    macos:
      xcode: "10.2.0"
    working_directory: /Users/distiller/project
    environment:
      FL_OUTPUT_DIR: output

    steps:
      - checkout
      - run:
          name: Fetch CocoaPods Specs
          command: |
            curl https://cocoapods-specs.circleci.com/fetch-cocoapods-repo-from-s3.sh | bash -s cf
      - run:
          name: Install CocoaPods
          command: pod install --verbose

      - run:
          name: Build and run tests
          command: fastlane scan
          environment:
            SCAN_DEVICE: iPhone 8
            SCAN_SCHEME: WebTests

      - store_test_results:
          path: output/scan
      - store_artifacts:
          path: output

  swiftlint:
    docker:
      - image: dantoml/swiftlint:latest
    steps:
      - checkout
      - run: swiftlint lint --reporter junit | tee result.xml
      - store_artifacts:
          path: result.xml
      - store_test_results:
          path: result.xml

  danger:
    docker:
      - image: dantoml/danger:latest
    steps:
      - checkout
      - run: danger

workflows:
  version: 2
  build-test-lint:
    jobs:
      - swiftlint
      - danger
      - build-and-test

{% endraw %}

React Native projects

{:.no_toc}

React Native projects can be built on CircleCI 2.0 using macos and docker executor types. Please check out this example React Native application on GitHub for a full example of a React Native project.

See Also

{:.no_toc}

  • See the circleci-demo-ios GitHub repository for a full example of how to build, test, sign and deploy an iOS project using Fastlane on CircleCI 2.0.
  • See the [iOS Project Tutorial]( {{ site.baseurl }}/2.0/ios-tutorial/) for a config walkthrough.
You can’t perform that action at this time.