iOS CI CD
The main repo for SimplyE and Open eBooks for iOS takes advantage of Continuous Integration / Continuous Delivery (CI/CD) to ensure product consistency and deliver builds to stakeholders. We use GitHub Actions and bash scripts to achieve that. This document explains in detail how these pieces work together.
We want to ensure that SimplyE and Open eBooks can be built successfully at all times in a predictable and reproducible way. To this end we have configured 2 GitHub Actions workflows that are run every time a PR is published:
- Unit Tests: This workflow builds both apps and runs their related unit tests on the simulator.
- Non-DRM build: This builds SimplyE without DRM support. This is important because the code to enable DRM support is not publicly available, while SimplyE is, and we want to ensure everyone can pick up the code and build a usable version of the app.
We want to make available the most up-to-date version of SimplyE to our QA teams every time there's a change made to its code, and to the App Store for wide distribution. To this end we have implemented the following workflow:
-
SimplyE and Open eBooks Build: after first verifying whether we need to build a new binary for either app, it archives SimplyE and/or Open eBooks, exports them for Ad-Hoc, and uploads them to the iOS-binaries repo. For release branches, it also uploads them to TestFlight for App Store distribution. This workflow is triggered for every commit to
developand for feature and release branches.
Let's take a look at what we have automated. BTW you can see past runs of all workflows here.
Since this is the simplest of our workflows, let's use it to introduce some of the assumptions shared with the others.
SimplyE has various dependencies from other NYPL's and 3rd party projects. We express these via git submodules and via Carthage. We use a Personal Access Token (PAT; see secrets.IOS_DEV_CI_PAT) to authenticate on GitHub: this is not needed on a developer machine configured with SSH keys, but since it's more difficult to configure SSH in a CI context (where you don't directly control the build machine) we resort to using a token an authentication token.
This PAT is defined on the Simplified-CI GitHub user: it's named "iOS Dev CI" and it has the minimum permissions needed, i.e. "repo" and "workflow" scope.
After fetching the code we force usage of a specific version of the Xcode toolchain, because each new version of Xcode may break the build.
The rest of the workflow follows pretty closely the manual build process that we have in the README.
This workflow builds the apps with DRM support for the simulator and runs the unit tests. In addition to the non-DRM build we also checkout the private repos, and build the apps / unit tests with a similar script.
You'll notice that this workflow explicitly checks out the NYPLAEToolkit dependency, which is a private repo normally managed via Carthage in a non-CI environment. It also uses a third-party GitHub Action to build Carthage instead of our own build-carthage.sh script. Our script in fact fails for 2 reasons in a CI context:
-
it seems impossible to pass the authentication token "inside" the Carthage environment so that it can fetch the private repos. In a local dev machine we'd use SSH keys but for GitHub Actions this is not recommended (except for Deploy Keys, which however require access to a server, which I did not have). Luckily, a 3rd party developer built a custom Carthage action that can take care of doing the Carthage bootstrap, handling authentication once it's provided with a PAT.
-
But... that's not enough. As you can see from our
build-carthage.shscript, our Carthage build first has to fetch the binary for the AudioEngine library, symlink it so that both the main application project and the private NYPLAEToolkit framework can use it, and only after that we can bootstrap carthage.
This workflow is triggered every time a change is pushed to either the develop branch, or a feature/** or release/** branch. It is divided into 2 steps.
This first step determines if we actually need to build anything. For each app, it compares the Xcode version and build numbers of the code we are about to build with the previous commit in the history for the same app. If they are identical, it skips building that app. Then it checks if the iOS-binaries repo already contains a matching build number, and if it does, it skips building that app.
This means that the developers are always in charge of what's happening on the CI pipeline, and requires them to manually increase the build number in Simplified.xcodeproj if they do want a new build to be sent out. The PR template on Simplified-iOS reminds them to bump the build number or not. E.g. you may not want to rebuild the app if you're just changing some documentation files or scripts, of if you know that the QA team is currently busy doing regression on a RC build.
We've considered to automatically increase the build number for every commit to develop. However this would require the CI scripts to add a commit as part of the SimplyE Build workflow (or amend an existing one), which is something that could potentially cause issues in the git history affecting everyone working on the project. There are also PRs that don't require a new build, as explained earlier. So, at best, the advantages of a fully automated solution seem minimal for the effort involved in making them and the associated risks. For these reasons, we decided to avoid these and other edge cases by simply putting the developer in charge.
If the version check passes, the archive step is triggered.
The archive step contains a duplication of steps for SimplyE and Open eBooks, so that we can build both apps if needed. Below we'll refer to SimplyE, but the same exact things apply to Open eBooks.
The first part of the archive step is similar to the Unit Tests workflow except it builds an archive (i.e. for physical devices) and it requires credentials to code-sign the build for distribution within the Apple ecosystem. We then export it for Ad Hoc and App Store distribution, with the latter happening only if the push event was received on a release/** branch.
The code-signing requirements are satisfied by the decode-install-secrets.sh script. For this to work we need:
-
The distribution certificate from the Apple developer portal. Since this rarely changes, we exported the cert locally, protected it with a passphrase, encoded it in Base64 (
base64 -i cert+key.p12 -o cert+key.p12.base64), and pasted the Base64 string into a Secret (IOS_DISTR_IDENTITY) on the Simplified-iOS repo. The passphrase is also a repo secret (IOS_DISTR_IDENTITY_PASSPHRASE). The decode-install-secrets.sh script takes care of decoding it, installing it into a new keychain, and enabling that keychain so that we can use its keys to codesign our builds. -
The Ad Hoc and App Store provisioning profiles. To fetch these dynamically (differently from certificates, these can frequently change) we use Fastlane
sigh(a name that says it all ;)). This is a fantastic tool that is also capable of installing the provisioning profiles so that Xcode can see them when it's time to use them. Simply copying them to the$HOME/Library/MobileDevice/Provisioning Profilesdirectory did not work in the context of GitHub Actions.
Once the build is signed, it can be uploaded to either Firebase and/or TestFlight.
The upload to Firebase is performed by the firebase-upload-sh script. Note that this script uses a token that we store in the Certificates repo. This token is long-lived but may expire after a year, and in that case the upload step of the build will error out (example). To regenerate it, install the Firebase CLI tools on your local machine and generate a new token using your google account, as described here: https://firebase.google.com/docs/cli#cli-ci-systems
The upload to TestFlight is performed by fastlane deliver.
Obviously, fastlane tools need credentials with Apple to do their work. These are provided via the APPLEID_USERNAME and APPLEID_PASSWORD repo secrets. The values of these secrets are related to an Apple ID that we use only for CI. With it, we created an App Store Connect user ("Library Simplified-CI") giving it the minimum set of permissions ("Developer" role and "access to certs and provisioning profiles") so that Fastlane can work.
At the time of this writing, Apple enforces 2FA automatically on all accounts, and it's not possible to disable it. This makes things more difficult in a CI context where there's no UI involved. To solve this problem Fastlane provides a mechanism to easily create a login session with a given Apple ID. This generates a session string that you need to save into the FASTLANE_SESSION secret in the Simplified-iOS repo.
IMPORTANT: This login session will expire after 1 month, so one will need to periodically update the FASTLANE_SESSION secret with a new session value. To do so, on your local dev machine run:
fastlane spaceauth -u APPLEID_USERNAMEwhere APPLEID_USERNAME is the same value used for the homonymous secret (if this hasn't been shared with you, ask your mobile lead). Fastlane will then ask to enter the 2FA code with the registered mobile phone (again, ask your mobile lead for it). After that it will produce a session string, which you need to copy and enter in the FASTLANE_SESSION secret, which you can find in the Simplified-iOS repo's Settings section, under "Secrets".
Fastlane deliver also requires an "app-specific password", which is something one can create after signing in to https://appleid.apple.com. The value of this password is stored as the APPLEID_APP_SPECIFIC_PASSWORD repo secret, and shouldn't need to change. It is passed to fastlane deliver via the FASTLANE_APPLE_APPLICATION_SPECIFIC_PASSWORD env variable.