Support for more build configurations

[enhorn]

Currently, Tuist only supports debug and release build configurations.
In my current project we handle preview builds with a separate build configuration.

What I would find useful would be to either have an optional array of further build configurations, or if all build configurations where set as an array instead of the debug and release that are there now.

[pepicrft]

Supporting only debug and release was done on purpose to reduce complex configurations setup. I'm ok with supporting more configurations, but I'd like to first understand how you use that third configuration. Perhaps we can think of an alternative approach.

[enhorn]

For us we have a setup with debug, preview and release.
These three have different bundle identifiers so they can be installed in parallel on the same device, have their own separate set of keychain credentials, authentication sessions agains different backens, crash reporting etc.

Debug is as expected only available when building through Xcode.
Preview is built each night, as well as a separate build when doing releases for testing purposes, and are distributed through a third party solution as an AdHoc build.
Release builds are as expected built for AppStore.

Both Debug and Preview builds have special developer settings where the user can switch backend environment, look at logs etc.

Both Preview and Release builds are done by our CI server.

We have fully disabled any special developer functionality for the Release build and locked it to the production backend environment.

[pepicrft]

What do you think about supporting it from the manifest with the following syntax?:

let project = Project(configurations: [
  Configuration("Preview", extends: .release, settings: [:])
])

By indicating that a configuration extends from .debug or .release it'd inherit some base settings that are specific to those configurations.

[enhorn]

@pepibumur Yeah, that would work!
Anything set in the settings dict would override any inherited values, right?

[pepicrft]

Anything set in the settings dict would override any inherited values, right?

Right @enhorn

[kwridan]

Multiple configurations would be a great addition to tuist!

If I understood it correctly a full project description would look something like this:

let debug = Configuration("Debug",
                          settings: ["Debug": "Debug"],
                          xcconfig: "/path/to/debug.xcconfig")
let release = Configuration("Release",
                            settings: ["Release": "Release"],
                            xcconfig: "/path/to/debug.xcconfig")
let preview = Configuration("Preview",
                            settings: ["Preview": "Preview"],
                            extends: .release,
                            xcconfig: "/path/to/preview.xcconfig")

let settings = Settings(base: ["Base": "Base"],
                        debug: debug,
                        release: release)
let project = Project(name: "MyProject",
                      settings: settings,
                      configurations: [preview])

Perhaps there would need to be another type for the additional configurations (e.g. AdditionalConfiguration) instead of re-using Configuration ? This will help avoid the following odd case:

let debug = Configuration("Debug",
                          settings: ["Debug": "Debug"],
                          extends: .release,
                          xcconfig: "/path/to/debug.xcconfig")

At first glance, I wondered why the BuildConfiguration enum was suggested as the extends type instead of Configuration, however looking closer at the code, I realised that would be needed to determine the variant and ensure those additional configurations end up with the appropriate default settings (e.g. DEBUG, GCC_OPTIMIZATION_LEVEL etc...).

[ollieatkinson]

In addition to the above it would be nice to provide a way to specify the creation of different build schemes for the different configurations.

For example, I have an App with a single target:

• App

With the following configurations:

• Automation
• User Acceptance Test
• Private Production
• Production
• Environment A
• Environment B

It would be nice to generate schemes for each of them, so that is is easy to create a build or test using those configurations.

The setup I'm talking about is quite similar to the one described here: https://thoughtbot.com/blog/let-s-setup-your-ios-environments. This setup is fairly common and best-practice for a lot of teams.

If we have a good idea of what we want to go for I'm happy to try push it forward. What are your thoughts and opinions? @pepibumur @enhorn @kwridan @marciniwanicki

[pepicrft]

My only concern regarding having a more configurable support for schemes is that we might end up replicating Xcode project's structure. If we can come up with an approach that keeps the interface simple and covers most projects use case that'd be ideal.
Do you have any idea of how you'd like Tuist to support the setup that you referenced?

[ollieatkinson]

Providing support for additional configurations are usually to support different environments like I stated above, and you want schemes so that you are able to launch the application target with different configurations.

I need to think about it more... but I have some ideas to go and play with.

[ollieatkinson]

How about the following.

Define the settings at the top level inside the project:

let project = Project(
    name: "MyProject",
    settings: Settings(configurations: [
      .configuration(name: "Automation", type: .debug, xcconfig: "Configs/Automation.xcconfig"),
      .configuration(name: "UAT", type: .debug, xcconfig: "Configs/UAT.xcconfig"),
      .configuration(name: "Private Production", type: .debug, xcconfig: "Configs/PrivateProduction.xcconfig"),
      .configuration(name: "A05", type: .debug, xcconfig: "Configs/A05.xcconfig"),
      .configuration(name: "E05", type: .debug, xcconfig: "Configs/E05.xcconfig"),
      .configuration(name: "Production", type: .release, xcconfig: "Configs/Production.xcconfig"),
    ]),
    targets: [
       ...
    ]
)

or

let project = Project(
    name: "MyProject",
    settings: Settings(configurations: [
      .debug(xcconfig: "/path/to/debug.xcconfig"),
      .release(xcconfig: "/path/to/release.xcconfig"),
    ]),
    targets: [
       ...
    ]
)

The default implementation will just just have debug and release if no configurations are specified.

The implementation for Settings looks something along the lines of:

public class Configuration: Codable {
    public let name: String
    public let type: BuildConfiguration
    public let settings: [String: String]
    public let xcconfig: String?

    public enum CodingKeys: String, CodingKey {
        case name
        case type
        case settings
        case xcconfig
    }

    public init(name: String, type: BuildConfiguration, settings: [String: String] = [:], xcconfig: String? = nil) {
        self.name = name
        self.type = type
        self.settings = settings
        self.xcconfig = xcconfig
    }

    public static func debug(_ settings: [String: String] = [:], xcconfig: String? = nil) -> Configuration {
        return Configuration(name: "Debug", type: .debug, settings: settings, xcconfig: xcconfig)
    }
    
    public static func release(_ settings: [String: String] = [:], xcconfig: String? = nil) -> Configuration {
        return Configuration(name: "Release", type: .release, settings: settings, xcconfig: xcconfig)
    }
    
    public static func configuration(name: String, type: BuildConfiguration = .debug, settings: [String: String] = [:], xcconfig: String? = nil) -> Configuration {
        return Configuration(name: name, type: type, settings: settings, xcconfig: xcconfig)
    }
    
}

// MARK: - Settings

public class Settings: Codable {
    public let base: [String: String]
    public let configurations: [Configuration]

    public init(base: [String: String] = [:], debug: Configuration = .debug(), release: Configuration = .release()) {
        self.base = base
        self.configurations = [ debug, release ]
    }
    
    public init(base: [String: String] = [:], configurations: [Configuration]) {
        self.base = base
        self.configurations = configurations
    }
    
}

We will struggle to maintain backwards compatibility because the old Settings struct enforced it by having two different properties.

By implementing this we then need to actually make it reflect in the project. I propose that the configuration for the project spec you initially call tuist generate defines the configuration for the workspace. This is similar to how CocoaPods achieves it's goal.

Specifying configurations does overwrite the defaults, but if you are wanting to provide your own then I think it's safe to assume you want to take control of the system and want that flexibility.

Second to this, I think it would be safe to automatically generated shared schemes for the application targets for each of the configurations.

Using the example I posted above, the following would be generated:

• MyProject (Automation)
• MyProject (UAT)
• MyProject (Private Production)
• MyProject (A05)
• MyProject (E05)
• MyProject (Production)

This approach is really light touch, and should only affect people who are wanting to introduce more configuration for their targets.

@kwridan @enhorn Would the above solution fit your needs?

[ollieatkinson]

I am stuck for a solution on a particular issue, if anyone is able to chime in with an idea that would be great.

With the above solution it's going to be fairly tedious to specify custom xcconfig configuration files for dependencies. Is there an elegant way to describe those?

e.g. This is what previously worked, but with specifying more configuration options not sure it will work.

import ProjectDescription

let project = Project(
    name: "LayoutView",
    settings: Settings(
      debug: Configuration(xcconfig: "Module.Debug.xcconfig"),
      release: Configuration(xcconfig: "Module.Release.xcconfig")
    ),
    targets: [

In the new world it would look something like this:

let project = Project(
    name: "LayoutView",
    settings: Settings(configurations: [
      .debug(xcconfig: "Module.Debug.xcconfig"),
      .release(xcconfig: "Module.Release.xcconfig")
    ]),
    targets: [

Is it safe enough to assume and attach .debug and .release to their appropriate configurations when in the workspace. e.g. Inside the LayoutView.xcodeproj

• MyProject (Automation) = Module.Debug.xcconfig
• MyProject (UAT) = Module.Debug.xcconfig
• MyProject (Private Production) = Module.Debug.xcconfig
• MyProject (A05) = Module.Debug.xcconfig
• MyProject (E05) = Module.Debug.xcconfig
• MyProject (Production) = Module.Release.xcconfig

[CognitiveDisson]

There is no need to specify the release configuration. Same as in ruby xcodeproj.

public final class XCBuildConfiguration: PBXObject {

    public var baseConfiguration: PBXFileReference? 

    /// A map of build settings.
    public var buildSettings: BuildSettings // just dictionary

    /// The configuration name.
    public var name: String

    public var isDebug: Bool {
       guard let preprocessorDefinitions = buildSettings['GCC_PREPROCESSOR_DEFINITIONS'] else { return false}
        return buildSettings.contains("DEBUG=1")
    }
}

[ollieatkinson]

Thanks for that @CognitiveDisson - that should simplify the usage even further.

Edit:
However, thinking about it... for us to generate the configuration in the first instance, we still need to know if the configuration is DEBUG or RELEASE - since we are the ones specifying the build settings from BuildSettingProvider in xcodeproj.

[pepicrft]

We will struggle to maintain backwards compatibility because the old Settings struct enforced it by having two different properties.

I'd not worry about figuring out how to build this with backwards compatibility. I'm strong believer in the value of following semantic versioning when releasing software, but considering that we are still iterating the API, I'd introduce the breaking change. We need to state clearly that the change is breaking.

By implementing this we then need to actually make it reflect in the project. I propose that the configuration for the project spec you initially call tuist generate defines the configuration for the workspace. This is similar to how CocoaPods achieves it's goal.

Could you rephrase this one?

Specifying configurations does overwrite the defaults, but if you are wanting to provide your own then I think it's safe to assume you want to take control of the system and want that flexibility.

Agreed 👍

Second to this, I think it would be safe to automatically generated shared schemes for the application targets for each of the configurations.

I'm not sure about this one because you'd end up with a huge list of schemes. I'd rather keep it short and generate a scheme per target by default. Once we introduce more flexibility in the schemes area, developers will have more control to decide how they want their schemes to look. Just dumping my thoughts here. I'm curious about what @kwridan / @marciniwanicki think about this.

This approach is really light touch, and should only affect people who are wanting to introduce more configuration for their targets.

We should make sure that when no configuration is provided we generate a valid project that works with the default configurations Debug and Release.

With the above solution it's going to be fairly tedious to specify custom xcconfig configuration files for dependencies. Is there an elegant way to describe those?

Don't fully get what would be the issue here 😬

I just had an idea that I'd like to through here. I recently came across the @ dynamicCallable feature from Swift, which I found very interesting. I think we can benefit from it here. Instead of having .debug, .release, and .configuration as Configuration options we could define a callable in the Configuration type that would allow doing something like:

[
    .debug(xcconfig: "/path/to/debug.xcconfig"),
    .release(xcconfig: "/path/to/release.xcconfig"),
    .beta(type: .debug, settings: [:])
]

And we could infer the name by taking the dynamic method name and uppercasing the first string :love:

[kwridan]

A bit of a long reply I drafted with @marciniwanicki - grab a ☕️:

How about the following.
Define the settings at the top level inside the project:

let project = Project(
    name: "MyProject",
    settings: Settings(configurations: [
      .debug(xcconfig: "/path/to/debug.xcconfig"),
      .release(xcconfig: "/path/to/release.xcconfig"),
    ]),
    targets: [
       ...
    ]
)

More in favor of this style (it’s mostly syntactic sugar 🙂)

Actually, you can go a step further:

public static func debug(name: String = "Debug", _ settings: [String: String] = [:], xcconfig: String? = nil) -> Configuration {
        return Configuration(name: name, type: .debug, settings: settings, xcconfig: xcconfig)
    }
    
    public static func release(name: String = "Release", _ settings: [String: String] = [:], xcconfig: String? = nil) -> Configuration {
        return Configuration(name: name, type: .release, settings: settings, xcconfig: xcconfig)
    }

That way, your definition becomes:

let project = Project(
    name: "MyProject",
    settings: Settings(configurations: [
      .debug(name: "Automation", xcconfig: "Configs/Automation.xcconfig"),
      .debug(name: "UAT", xcconfig: "Configs/UAT.xcconfig"),
      .debug(name: "Private Production", xcconfig: "Configs/PrivateProduction.xcconfig"),
      .debug(name: "A05", xcconfig: "Configs/A05.xcconfig"),
      .debug(name: "E05", xcconfig: "Configs/E05.xcconfig"),
      .release(name: "Production", xcconfig: "Configs/Production.xcconfig"),
    ]),
    targets: [
       ...
    ]
)

The default implementation will just just have debug and release if no configurations are specified.

Makes sense

We will struggle to maintain backwards compatibility because the old Settings struct enforced it by having two different properties.

With what you proposed we still have some backward compatibility as the user facing API won’t be impacted unless someone was explicitly referencing settings.debug or settings.release within the manifest.

Specifying configurations does overwrite the defaults, but if you are wanting to provide your own then I think it's safe to assume you want to take control of the system and want that flexibility.

Makes sense.

If we later find that is not the case, we could add something like this to allow extending the defaults

public extension Settings {
    static func defaultsWith(base: [String: String],
                             configurations: [Configuration]) -> Settings {
        return Settings(base: base, configurations: [.debug, .release] + configurations)
    }
}

Let’s go for the simple option first.

Second to this, I think it would be safe to automatically generated shared schemes for the application targets for each of the configurations.

This is a tough one, speaking from some of our use cases, there isn’t a 1-1 mapping.

We have several schemes that use the same configs, and some configs not in any scheme. For the schemes, we often use a similar set of configs however vary launch arguments / environment variables. Additionally, a single scheme can reference different configs for the different actions (Run, Profile, Analyze & Archive).

As for the configs, many of them are used from the command line build scripts but not directly in Xcode via schemes.

With the above solution it's going to be fairly tedious to specify custom xcconfig configuration files for dependencies. Is there an elegant way to describe those?

The code snippet you posted looks reasonable - is the issue with the repetition of the Settings definition in dependency projects. I guess they would need to do that, otherwise a default or incorrect config may be chosen to build those projects.

There is an additional layer of complexity even more local to a single project - that is the individual targets! They currently allow specifying Settings which is fair enough, there can be target level build setting overrides, though not sure if there can be a configurations discrepancy (e.g. Project has Config A and Config B, one its target has Config C and Config D)

We’d probably need a new Settings structure like TargetSettings which omit the ability to specify configurations

struct TargetSettings {

    // This prevents inlineing the definition, as you'd need to caspture it in a variable first :(
   var buildSettingsOverride: [Configuration: [String: String]]
    // or maybe "Configuration Name: [String: String]"
    // and rely on linting to ensure they exist
   var buildSettingsOverride: [String: [String: String]]
}

I'm not sure about this one because you'd end up with a huge list of schemes. I'd rather keep it short and generate a scheme per target by default. Once we introduce more flexibility in the schemes area, developers will have more control to decide how they want their schemes to look. Just dumping my thoughts here. I'm curious about what @kwridan / @marciniwanicki think about this.

Perhaps it’s simplest to decouple schemes generations from the configurations definitions, I know this is entering more configuration over convention territory but may be simpler over the long run.

I just had an idea that I'd like to through here. I recently came across the @ dynamicCallable feature from Swift, which I found very interesting. I think we can benefit from it here. Instead of having .debug, .release, and .configuration as Configuration options we could define a callable in the Configuration type that would allow doing something like:

And we could infer the name by taking the dynamic method name and uppercasing the first string

Cool idea, it may come at the cost of losing auto complete, but could be an addition ontop of some defaults.

[ollieatkinson]

Could you rephrase this one?

Badly worded, my error.

What I mean here is that I would not expect each project to define a list of configurations that the main app supports. I would expect it supports only the ones it cares about.

I like the suggestion @kwridan demonstrated above with TargetSettings - this is something I actually started implementing on my proof of concept. Nice to see we are on the same page here.

Don't fully get what would be the issue here 😬

Similar to the issue above, I would not want to define the configurations at each level. However one of the dependencies might require to override some configuration or specify their own xcconfig file.

I'm not sure about this one because you'd end up with a huge list of schemes. I'd rather keep it short and generate a scheme per target by default. Once we introduce more flexibility in the schemes area, developers will have more control to decide how they want their schemes to look. Just dumping my thoughts here. I'm curious about what @kwridan / @marciniwanicki think about this.

We have several schemes that use the same configs, and some configs not in any scheme. For the schemes, we often use a similar set of configs however vary launch arguments / environment variables. Additionally, a single scheme can reference different configs for the different actions (Run, Profile, Analyze & Archive).

It sounds like the scheme's side of things is still a little unknown, so I don't think we should solve that here.

Making it declarative about what schemes have which build configuration, arguments and pre/post actions may introduce more configuration - but it seems like with the multitude of different requirements that we need between ourselves that there is a clear need for it to be configurable.

[ollieatkinson]

We’d probably need a new Settings structure like TargetSettings which omit the ability to specify configurations

I prefer using the name so you can inline it and then catch it at linting.

public class TargetSettings: Codable {
    
    public let buildSettings: [Configuration.Name: BuildSettings]
    
    public init(buildSettings: [Configuration.Name: BuildSettings]) {
        self.buildSettings = buildSettings
    }
    
}

public typealias BuildSettings = [String: String]

public class Configuration: Codable {
    
    public typealias Name = String
....

[pepicrft]

A bit of a long reply I drafted with @marciniwanicki - grab a ☕️:

What I mean here is that I would not expect each project to define a list of configurations that the main app supports. I would expect it supports only the ones it cares about.

Agree. I think the app (entry point to the graph) should be the one determining which configurations the project supports. Dependencies and transitive dependencies will have to adhere to that. They should be able to override project configurations as they wish. If any of the dependencies includes a configuration that is not supported by the project I'd fail letting the developer know about the mismatch.

I've been pondering the idea of leveraging types and generics more in the manifest so that we can define restrictions leveraging the type system instead of using linting rules. For example if the project is generic with the configuration types debug and release, we could limit any target to only being able to define the same configurations.
Although I think it's powerful, I think it'd make the manifest harder to reason about and the auto-completion not working as expected.

It sounds like the scheme's side of things is still a little unknown, so I don't think we should solve that here.

Agree.I think proper schemes support is a feature on its own and we'll need a lot of input from people with different setups to come up with a good design.