Add automation and configuration for Prototype Builds #1769

mokagio · 2024-05-17T08:42:43Z

Internal ref pdnsEh-1D4-p2

This PR replaces the unused "Staging" build configuration with a "Prototype" one.

To get there:

All the app identifiers and their attributes have been configured in the Enterprise account
New .entitlements files have been added to the targets that need them
The way bundle identifiers and provisioning profiles are defined for each target has been made hierarchical and DRYed at the xcconfig level, to avoid having to edit the project file via the error prone Xcode GUI
Building on top of Automation refactors to better support Enterprise builds #1749, new lanes for code signing and building have been added

There is a minor naming asymmetry that I'd like your thoughts on. At the Xcode level, I referred to the new build configuration as "Prototype", in Fastlane as "Enterprise". I did that because the Xcode level is closer to the dev-side of things, where I think it's good to identify the scheme and configuration as "Prototype" because devs will "consume" the build in the established "Prototype Build" pattern within Automattic. But in Fastlane, where we deal with infra and ASC, I found it useful to think with the vocabulary Apple uses.

What do you think?

If the asymmetry is annoying, I'd then use "Prototype" name in Fastlane, too.

Side note, while working on this, I opened the cleanups #1768 and #1767.

To test

Run bundle exec fastlane build_enterprise

Checklist

I have considered if this change warrants user-facing release notes and have added them to CHANGELOG.md if necessary.
I have considered adding unit tests for my changes.
I have updated (or requested that someone edit) the spreadsheet to reflect any new or changed analytics.

As in, the configuration for prototype builds

mokagio · 2024-05-17T08:47:23Z

fastlane/Fastfile

+      # Notice the `in_house: true` parameter.
+      # Unfortunately, even using it, the API does not allow modifying certs or profiles via API key.
+      # It's here for reference and future-proofing.
+      api_key = app_store_connect_api_key(in_house: true)


I discovered the in_house parameter in the docs while working on something else. I had my hopes up it would have spared us the trouble of authenticating locally when running for Enterprise in readonly: false but it didn't work...

mokagio · 2024-05-17T08:48:40Z

podcasts.xcodeproj/xcshareddata/xcschemes/Pocket Casts Prototype Build.xcscheme

@@ -120,7 +103,7 @@
      buildConfiguration = "Debug">
   </AnalyzeAction>
   <ArchiveAction
-      buildConfiguration = "Release"
+      buildConfiguration = "Prototype"
      revealArchiveInOrganizer = "YES">
   </ArchiveAction>
 </Scheme>


This scheme looks like and edit of an existing in this diff because of how textual diffs work, but once #1767 lands it'll be a new one.

mokagio · 2024-05-17T08:50:12Z

podcasts/podcasts.prototype.entitlements

+		<string>group.au.com.shiftyjelly.pocketcasts.prototype</string>
+	</array>
+</dict>
+</plist>


I don't like how the entitlements are in the root of the target but I didn't change it to avoid more noise in the diff. I'd like to:

Have them in podcasts/entitlements/

For the debug one to follow the .<config>.entitlement naming convention

mokagio · 2024-05-20T11:49:55Z

.buildkite/commands/build.sh

@@ -1,9 +1,6 @@
 #!/bin/bash -u

 echo "--- Set up SPM"
-# We'd like to use this, but it doesn't work yet. See
-# https://github.com/Automattic/bash-cache-buildkite-plugin/issues/20 ...
-#


This comment was a leftover. But looking at the rest of the code it seems like all the following SSH code is unnecessary now that 1) everything is private and 2) we have a CI pluging command for SPM

leandroalonso

@mokagio the Staging is actually used though. 😬

Basically, we always build this configuration when we want to test stuff using the staging backend. You can see some #if STAGING macros in the code.

We've been discussing moving that to a toggle because the change in configuration is not obvious and Xcode rebuilds the whole app when we do so — so this is not a blocker.

However, I'm having issues testing the Prototype configuration:

Showing All Errors Only
/Users/leandroalonso/Documents/pocketcasts/alpha/podcasts.xcodeproj:1:1: Unable to open base configuration reference file '/Users/leandroalonso/Documents/pocketcasts/alpha/Pods/Target Support Files/Pods-podcasts/Pods-podcasts.prototype.xcconfig'.
Unable to load contents of file list: '/Target Support Files/Pods-podcasts/Pods-podcasts-frameworks-Prototype-input-files.xcfilelist'
Unable to load contents of file list: '/Target Support Files/Pods-podcasts/Pods-podcasts-frameworks-Prototype-output-files.xcfilelist'

There's an additional step I need to do to build it?

mokagio · 2024-05-21T04:07:03Z

@leandroalonso Hey there!

@mokagio the Staging is actually used though. 😬

Ouch! 🤦‍♂️

We've been discussing moving that to a toggle because the change in configuration is not obvious and Xcode rebuilds the whole app when we do so — so this is not a blocker.

That would also make it easier to test WIP features in the staging backend on a prototype build 😄

I'll open a PR to restore the Staging build configuration 👍

mokagio · 2024-05-21T04:09:07Z

@leandroalonso

However, I'm having issues testing the Prototype configuration: [...] There's an additional step I need to do to build it?

From the logs, I think you might need to run bundle exec pod install (which I didn't consider when writing the instructions).

CocoaPods generates xcconfigs for the different build configurations. So now that we the new prototype configuration, we need to run CocoaPods to generate those new xcconfigs.

See conversation at #1769 (review)

dangermattic · 2024-05-21T07:01:06Z

	1 Warning
⚠️	This PR is larger than 500 lines of changes. Please consider splitting it into smaller PRs for easier and faster reviews.

Generated by 🚫 Danger

mokagio · 2024-05-21T07:07:54Z

I'll open a PR to restore the Staging build configuration 👍

I didn't realize on which branch I was and pushed to this PR instead of opening a new one.

Anyway, the Staging build configuration is back in place. It seems better to have it available than to have to remember about the build flag until the day a toggle in the settings will be implemented.

config/PocketCasts.prototype.xcconfig

leandroalonso

@mokagio I can now build the Prototype but I can't run it:

Tested the Staging and all good!

mokagio · 2024-05-29T09:51:53Z

🤔 that's interesting...

~~It looks like there's a difference in how the PRODUCT_BUNDLE_IDENTIFIER is computed for Info.plist and at launch time.~~

~~It might have something to do with the fact that PRODUCT_BUNDLE_IDENTIFIER is set as $(PRODUCT_BUNDLE_IDENTIFIER_ROOT).watchkitapp...~~

Looking into it...

Update 🤦‍♂️ I was looking at the wrong key. PRODUCT_BUNDLE_IDENTIFIER is computed at runtime and fine. The key that was giving problem was WKCompanionAppBundleIdentifier which was hardcoded in the Info.plist.

mokagio · 2024-05-29T10:27:09Z

@leandroalonso I removed the hardcoded value that resulted in the failure.

The prototype build runs on my end. And so do the staging and debug builds.

leandroalonso

@mokagio now it works for me!

However when running bundle exec fastlane build_enterprise it fails:

[!] pocket-casts-ios.env not found in /Users/leandroalonso/.a8c-apps!

Please copy fastlane/example.env to /Users/leandroalonso/.a8c-apps/pocket-casts-ios.env and fill in the values for the automation you require.

mkdir -p /Users/leandroalonso/.a8c-apps && cp fastlane/example.env /Users/leandroalonso/.a8c-apps/pocket-casts-ios.env

Here I saw you mentioning:

I'll followup tomorrow with a PR that sets up the .env pattern to account for local builds.

Was that done? Do I need to run any command to have this set up?

mokagio · 2024-05-30T01:51:30Z

Was that done? Do I need to run any command to have this set up?

Yes, via #1761

The prompt suggests to run:

mkdir -p /Users/leandroalonso/.a8c-apps && cp fastlane/example.env /Users/leandroalonso/.a8c-apps/pocket-casts-ios.env

Which should create the file in the expected location. Then, one needs to add the correct env vars in there. You can find the values in mobile-secrets.

leandroalonso · 2024-05-30T14:22:30Z

@mokagio if I just execute the command, the build fails. Then I copied the env file from .mobile-secrets it was still failing because some values come from the shared-secrets.yaml. I then copied each value one by one, however it failed on build_app :(

[11:12:13]: ▸ Clean Succeeded
[11:12:25]: ▸     Unable to read contents of XCFileList '/Target Support Files/Pods-podcasts/Pods-podcasts-frameworks-Prototype-output-files.xcfilelist' (in target 'podcasts' from project 'podcasts')
[11:12:25]: ▸ ❌  error: Unable to load contents of file list: '/Target Support Files/Pods-podcasts/Pods-podcasts-frameworks-Prototype-input-files.xcfilelist' (in target 'podcasts' from project 'podcasts')
[11:12:25]: ▸ ❌  error: Unable to load contents of file list: '/Target Support Files/Pods-podcasts/Pods-podcasts-frameworks-Prototype-output-files.xcfilelist' (in target 'podcasts' from project 'podcasts')
[11:12:25]: ▸     Run script build phase '[CP] Embed Pods Frameworks' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'podcasts' from project 'podcasts')
[11:12:25]: ▸ ** ARCHIVE FAILED **
▸ Clean Succeeded
    Unable to read contents of XCFileList '/Target Support Files/Pods-podcasts/Pods-podcasts-frameworks-Prototype-output-files.xcfilelist' (in target 'podcasts' from project 'podcasts')

❌  error: Unable to load contents of file list: '/Target Support Files/Pods-podcasts/Pods-podcasts-frameworks-Prototype-input-files.xcfilelist' (in target 'podcasts' from project 'podcasts')



❌  error: Unable to load contents of file list: '/Target Support Files/Pods-podcasts/Pods-podcasts-frameworks-Prototype-output-files.xcfilelist' (in target 'podcasts' from project 'podcasts')


    Run script build phase '[CP] Embed Pods Frameworks' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'podcasts' from project 'podcasts')
** ARCHIVE FAILED **
[11:12:25]: Exit status: 65
[11:12:25]:
[11:12:25]: Maybe the error shown is caused by using the wrong version of Xcode
[11:12:25]: Found multiple versions of Xcode in '/Applications/'
[11:12:25]: Make sure you selected the right version for your project
[11:12:25]: This build process was executed using '/Applications/Xcode.app'
[11:12:25]: If you want to update your Xcode path, either
[11:12:25]:
[11:12:25]: - Specify the Xcode version in your Fastfile
[11:12:25]: ▸ xcversion(version: "8.1") # Selects Xcode 8.1.0
[11:12:25]:
[11:12:25]: - Specify an absolute path to your Xcode installation in your Fastfile
[11:12:25]: ▸ xcode_select "/Applications/Xcode8.app"
[11:12:25]:
[11:12:25]: - Manually update the path using
[11:12:25]: ▸ sudo xcode-select -s /Applications/Xcode.app
[11:12:25]:

+-----------------------------------------+
|            Build environment            |
+---------------+-------------------------+
| xcode_path    | /Applications/Xcode.app |
| gym_version   | 2.220.0                 |
| export_method | enterprise              |
| sdk           | iPhoneOS17.4.sdk        |
+---------------+-------------------------+

[11:12:25]: ▸ error: Unable to load contents of file list: '/Target Support Files/Pods-podcasts/Pods-podcasts-frameworks-Prototype-output-files.xcfilelist' (in target 'podcasts' from project 'podcasts')
[11:12:25]: ▸ warning: Run script build phase '[CP] Embed Pods Frameworks' will be run during every build because it does not specify any outputs. To address this warning, either add output dependencies to the script phase, or configure it to run in every build by unchecking "Based on dependency analysis" in the script phase. (in target 'podcasts' from project 'podcasts')
[11:12:25]: ▸ note: Run script build phase 'Modify Plist' will be run during every build because the option to run the script phase "Based on dependency analysis" is unchecked. (in target 'podcasts' from project 'podcasts')
[11:12:25]: ▸ note: Run script build phase 'L10n' will be run during every build because the option to run the script phase "Based on dependency analysis" is unchecked. (in target 'PreBuildActions' from project 'podcasts')
[11:12:25]: ▸ note: Run script build phase 'SwiftLint' will be run during every build because the option to run the script phase "Based on dependency analysis" is unchecked. (in target 'PreBuildActions' from project 'podcasts')
[11:12:25]:
[11:12:25]: ⬆️  Check out the few lines of raw `xcodebuild` output above for potential hints on how to solve this error
[11:12:25]: 📋  For the complete and more detailed error log, check the full log at:
[11:12:25]: 📋  /Users/leandroalonso/Library/Logs/gym/podcasts-Pocket Casts Prototype Build.log
[11:12:25]:
[11:12:25]: Looks like fastlane ran into a build/archive error with your project
[11:12:25]: It's hard to tell what's causing the error, so we wrote some guides on how
[11:12:25]: to troubleshoot build and signing issues: https://docs.fastlane.tools/codesigning/getting-started/
[11:12:25]: Before submitting an issue on GitHub, please follow the guide above and make
[11:12:25]: sure your project is set up correctly.
[11:12:25]: fastlane uses `xcodebuild` commands to generate your binary, you can see the
[11:12:25]: the full commands printed out in yellow in the above log.
[11:12:25]: Make sure to inspect the output above, as usually you'll find more error information there
[11:12:25]:
+-------------------------------------------------------------------------------------+
|                                    Lane Context                                     |
+------------------------------------+------------------------------------------------+
| DEFAULT_PLATFORM                   | ios                                            |
| PLATFORM_NAME                      | ios                                            |
| LANE_NAME                          | ios build_enterprise                           |
| SIGH_PROFILE_TYPE                  | enterprise                                     |
| MATCH_PROVISIONING_PROFILE_MAPPING | {"au.com.shiftyjelly.podcasts.prototype"=>"ma  |
|                                    | tch InHouse                                    |
|                                    | au.com.shiftyjelly.podcasts.prototype",        |
|                                    | "au.com.shiftyjelly.podcasts.prototype.Notifi  |
|                                    | cationContent"=>"match InHouse                 |
|                                    | au.com.shiftyjelly.podcasts.prototype.Notific  |
|                                    | ationContent",                                 |
|                                    | "au.com.shiftyjelly.podcasts.prototype.Notifi  |
|                                    | cationExtension"=>"match InHouse               |
|                                    | au.com.shiftyjelly.podcasts.prototype.Notific  |
|                                    | ationExtension",                               |
|                                    | "au.com.shiftyjelly.podcasts.prototype.Podcas  |
|                                    | tsIntents"=>"match InHouse                     |
|                                    | au.com.shiftyjelly.podcasts.prototype.Podcast  |
|                                    | sIntents",                                     |
|                                    | "au.com.shiftyjelly.podcasts.prototype.Podcas  |
|                                    | tsIntentsUI"=>"match InHouse                   |
|                                    | au.com.shiftyjelly.podcasts.prototype.Podcast  |
|                                    | sIntentsUI",                                   |
|                                    | "au.com.shiftyjelly.podcasts.prototype.Share-  |
|                                    | Extension"=>"match InHouse                     |
|                                    | au.com.shiftyjelly.podcasts.prototype.Share-E  |
|                                    | xtension",                                     |
|                                    | "au.com.shiftyjelly.podcasts.prototype.watchk  |
|                                    | itapp"=>"match InHouse                         |
|                                    | au.com.shiftyjelly.podcasts.prototype.watchki  |
|                                    | tapp",                                         |
|                                    | "au.com.shiftyjelly.podcasts.prototype.Widget  |
|                                    | Extension"=>"match InHouse                     |
|                                    | au.com.shiftyjelly.podcasts.prototype.WidgetE  |
|                                    | xtension"}                                     |
+------------------------------------+------------------------------------------------+
[11:12:25]: Error building the application - see the log above

Am I doing something wrong?

mokagio · 2024-06-04T03:41:33Z

@leandroalonso sorry for the late followup

❌ error: Unable to load contents of file list: '/Target Support Files/Pods-podcasts/Pods-podcasts-frameworks-Prototype-input-files.xcfilelist' (in target 'podcasts' from project 'podcasts')

Suggest to me that something is missing or misconfigured in your local pods installation. I say "your" because CI builds fine once instructed to bypass the pods cache (i.e. generated Pods from scratch including the xcfilelist for the Prototype configuration) and bundle exec pod install && bundle exec fastlane build_enterprise works on my end.

leandroalonso

Now it worked!

Sorry for the back and forth @mokagio, good to go!

mokagio · 2024-06-04T23:36:43Z

Sorry for the back and forth @mokagio, good to go!

No worries at all. I found it helped me better understand what CocoaPods does in terms of its xcconfigs.

mokagio added 14 commits May 17, 2024 13:47

Rename unused (?) Staging build configuration to Prototype

73dbaed

As in, the configuration for prototype builds

Move bundle id definition to xcconfig

ad872c3

Parameterize and differentiate bundle ids between AS and Prototype

33c5d8d

Add scheme and lane for Enterprise/Prototype Build

f166314

Push code sign identity definitions at project level (DRY)

90f853d

Push development team definition to base.xcconfig (DRY)

da34412

Use Enterprise dev team for Prototype build configuration

2cc0ef5

Order extensions alphabetically like Developer Portal does

fc4235c

Remove legacy Today Extension scheme

b86f6a2

Remove dead references to Today Extension target

f257d72

Add entitlements files for Prototype bulid

65d64d6

Add prototype entitlements for PodcastsIntents, Share, and Widget ext.

76774f7

DRY provisioning profile definition and remove cruft

2aa1986

DRY and update provisioning profiles for Prototype Build

c34b76a

mokagio requested a review from a team as a code owner May 17, 2024 08:42

mokagio requested review from bjtitus and removed request for a team May 17, 2024 08:42

mokagio changed the base branch from trunk to mokagio/fastlane-env May 17, 2024 08:42

mokagio commented May 17, 2024

View reviewed changes

mokagio added this to the 7.65 milestone May 17, 2024

mokagio added the tooling Issues related to tooling: build tools, ruby, scripts, etc. label May 17, 2024

mokagio force-pushed the mokagio/prototype-builds-local branch from bc058f0 to 3bf522d Compare May 20, 2024 11:31

mokagio commented May 20, 2024

View reviewed changes

mokagio force-pushed the mokagio/prototype-builds-local branch from 3258f46 to c34b76a Compare May 20, 2024 12:07

leandroalonso self-requested a review May 20, 2024 16:56

leandroalonso requested changes May 20, 2024

View reviewed changes

mokagio requested review from leandroalonso and a team May 21, 2024 02:37

mokagio added 5 commits May 21, 2024 16:43

Remove -D STAGING flag from Prototype build configuration

379f558

Remove -D DEBUG from Prototype build configuration

ce45265

Move OTHER_SWIFT_FLAGS = -D DEBUG to the xcconfig level

96dd4c9

Reintroduce Staging build configuration

2068f14

See conversation at #1769 (review)

Add dedicated scheme to run Staging app

d7df0e7

mokagio commented May 21, 2024

View reviewed changes

config/PocketCasts.prototype.xcconfig Show resolved Hide resolved

leandroalonso requested changes May 21, 2024

View reviewed changes

SergioEstevao modified the milestones: 7.65 ❄️, 7.66 May 27, 2024

Base automatically changed from mokagio/fastlane-env to trunk May 29, 2024 01:50

Make prototype build run with -D DEBUG

049f84a

mokagio added 2 commits May 29, 2024 19:59

Compute WKCompanionAppBundleIdentifier for watch app from xcconfig

151204b

Add bundle identifier to dev menu for reference

d3b6d7a

mokagio requested a review from leandroalonso May 29, 2024 10:27

leandroalonso reviewed May 29, 2024

View reviewed changes

Extract lane to build for ASC without uploading

2b02c50

leandroalonso approved these changes Jun 4, 2024

View reviewed changes

mokagio merged commit feeb220 into trunk Jun 4, 2024
4 checks passed

mokagio deleted the mokagio/prototype-builds-local branch June 4, 2024 23:35

mokagio mentioned this pull request Jun 5, 2024

Prototype Builds #1771

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add automation and configuration for Prototype Builds #1769

Add automation and configuration for Prototype Builds #1769

mokagio commented May 17, 2024 •

edited

mokagio May 17, 2024

mokagio May 17, 2024

mokagio May 17, 2024

mokagio May 20, 2024

leandroalonso left a comment •

edited

mokagio commented May 21, 2024

mokagio commented May 21, 2024

dangermattic commented May 21, 2024

mokagio commented May 21, 2024

leandroalonso left a comment

mokagio commented May 29, 2024 •

edited

mokagio commented May 29, 2024

leandroalonso left a comment

mokagio commented May 30, 2024

leandroalonso commented May 30, 2024

mokagio commented Jun 4, 2024

leandroalonso left a comment

mokagio commented Jun 4, 2024

Add automation and configuration for Prototype Builds #1769

Add automation and configuration for Prototype Builds #1769

Conversation

mokagio commented May 17, 2024 • edited

To test

Checklist

mokagio May 17, 2024

Choose a reason for hiding this comment

mokagio May 17, 2024

Choose a reason for hiding this comment

mokagio May 17, 2024

Choose a reason for hiding this comment

mokagio May 20, 2024

Choose a reason for hiding this comment

leandroalonso left a comment • edited

Choose a reason for hiding this comment

mokagio commented May 21, 2024

mokagio commented May 21, 2024

dangermattic commented May 21, 2024

mokagio commented May 21, 2024

leandroalonso left a comment

Choose a reason for hiding this comment

mokagio commented May 29, 2024 • edited

mokagio commented May 29, 2024

leandroalonso left a comment

Choose a reason for hiding this comment

mokagio commented May 30, 2024

leandroalonso commented May 30, 2024

mokagio commented Jun 4, 2024

leandroalonso left a comment

Choose a reason for hiding this comment

mokagio commented Jun 4, 2024

mokagio commented May 17, 2024 •

edited

leandroalonso left a comment •

edited

mokagio commented May 29, 2024 •

edited