Include support for Project Definitions (reuse code across Project.swift files)

[ollieatkinson]

Is your feature request related to a problem? Please describe.
I want to be able to write some code which can be used in all of the project definitions.

Describe the solution you'd like
A mechanism for including shared code in the Project.swift file, an example @pepibumur mentioned was including //include includes/shared.swift at the top of the file.

We would have to map a graph for all of the includes and pass them to the compiler in the right order, from the most dependent to the least.

[pepicrft]

❤️ this idea. This is a clear advantage compared to other manifest formats like json or yaml. In modular apps where most of the targets have a similar structure, this will be very useful.

[enhorn]

+1 on this. Would be a big win for my setup as well.

[ollieatkinson]

@pepibumur I've actually thought about this more and not sure it's the best idea. Having the project files as declarative as possible should be our goal, introducing this might take us away from that.

I can see how sharing build settings might be useful, but we should look at doing that through another mechanism

[enhorn]

There are ups-and-downs, sure.

Currently I have a 50 line chunk that I need to keep in sync in 12 separate Project.swift files, and any new sub-project for my repo.

Some of it (like extended Carthage handling) is thankfully redundant now with recent updates to Tuist, but I haven't had the time to update yet.

[ollieatkinson]

@enhorn Do you have an example of what the repeated lines are?

[pepicrft]

Yeah, I'm also curious to see what are the most repeated pieces of code. With the information we could think of other potential solutions to tackle the reusability problem.

[enhorn]

First some context. Hehe.

So the whole project only uses one Cartfile that is shared between all projects right now, and I haven't had time to update my manual handling of Carthage dependencies to the one introduced to Tuist yet, but it's planned 😇.
In the meanwhile, I kept a Carthage.dependecies file in in any sub-project, to keep track in which module depended och witch Carthage framework.

Some code is to keep redundant relative paths to a minimum.

As the app has extensions, (action extension & rich notifications), I haven't been able to fully convert our project yet, and have been forced to run the Tuist generation side-by-side with the full project, and currently only uses it to generate the project structure, and then infuses my created app-modules in the the existing Workspace file.

A rough description of the module architecture of the app:

And finally, the actual code I copy to each module to pull this off. 😅
This is the code I need to copy to each project.

enum Setup {

    enum BuildConfiguration: String { case base, debug, preview, release }

    enum Paths {
        static let dependenciesFile: String = "Carthage.dependencies"
        static func dependencies(_ dependent: String) -> String {
            return "../\(dependent)/\(dependenciesFile)"
        }
        static func carthage(framework: String) -> String {
            return "../../MainApp/Carthage/Build/iOS/\(framework).framework"
        }
        static let baseConfig = "../Configurations/base.json"
        static func xcconfig(_ configuration: BuildConfiguration) -> String {
            return "../Configurations/\(configuration.rawValue).xcconfig"
        }
    }

    static func baseConfig() -> [String:String] {
        guard let data = try? Data(contentsOf: URL(fileURLWithPath: Paths.baseConfig), options: .mappedIfSafe) else { return [:] }
        guard let json = (try? JSONSerialization.jsonObject(with: data, options: .mutableLeaves)) as? [String:String] else { return [:] }
        return json
    }

    private static func dependencies(_ path: String) -> Set<String> {
        return Set<String>(((try? String(contentsOfFile: path, encoding: .utf8)) ?? "").split(separator: "\n").map(String.init))
    }

    static func carthage(dependants: [String] = []) -> [String] {
        var carthageDependencies: Set<String> = []
        carthageDependencies.formUnion(dependencies(Paths.dependencies(ProjectName)))
        carthageDependencies.formUnion(dependants.map(Paths.dependencies).map(dependencies).reduce([], +))
        return carthageDependencies.sorted()
    }

    static func settings(extraBase: [String:String] = [:], extraDebug: [String:String] = [:], extraRelease: [String:String] = [:]) -> Settings {
      return Settings(
        base: Setup.baseConfig().merging(extraBase, uniquingKeysWith: { $1 }),
        debug: Configuration(settings: extraDebug, xcconfig: Paths.xcconfig(.debug)),
        release: Configuration(settings: extraRelease, xcconfig: Paths.xcconfig(.release))
      )
    }

    static let moduleDependencyNames: [String] = ["NetworkAndModels", "UIComponents", "CoreComponents"]

    static func moduleDependencies() -> [TargetDependency] {
      return moduleDependencyNames.map { dependency in
        return .project(target: dependency, path: "../\(dependency)")
      }
    }

}

Which I then use to define the module in a project structure I generate when creating a new module:

let ProjectName = "AppModuleName"

let carthageFrameworks: [TargetDependency] = Setup.carthage(dependants: Setup.moduleDependencyNames).map({ .framework(path: Setup.Paths.carthage(framework: $0)) })

let project = Project(
  name: ProjectName,
  settings: Setup.settings(),
  targets: [
    Target(name: ProjectName,
      platform: .iOS,
      product: .framework,
      bundleId: "com.MyApp.\(ProjectName)${BUNDLE_SUFFIX}",
      infoPlist: "Info.plist",
      sources: "Sources/**",
      resources: "Resources/**",
      headers: Headers(public: "Headers/Public/*"),
      actions: [TargetAction.pre(tool: "swiftgen", arguments: [], name: "Swiftgen Generation")],
      dependencies: Setup.moduleDependencies() + carthageFrameworks,
      settings: Setup.settings()
    ),
    Target(name: "\(ProjectName)Tests",
      platform: .iOS,
      product: .unitTests,
      bundleId: "com.MyApp.\(ProjectName)Tests${BUNDLE_SUFFIX}",
      infoPlist: "Tests.plist",
      sources: "Tests/**",
      resources: "Resources/**",
      dependencies: [
        .target(name: ProjectName)
      ] + carthageFrameworks,
      settings: Setup.settings()
    ),
    Target(name: "\(ProjectName)App",
      platform: .iOS,
      product: .app,
      bundleId: "com.MyApp.\(ProjectName)App${BUNDLE_SUFFIX}",
      infoPlist: "App.plist",
      sources: "AppSources/**",
      resources: "AppResources/**",
      entitlements: "App.entitlements",
      dependencies: [
        .target(name: ProjectName)
      ] + carthageFrameworks,
      settings: Setup.settings(extraBase: [
        "CODE_SIGN_STYLE": "Automatic",
        "DEVELOPMENT_TEAM": "..."
      ])
    )
  ]
)

[enhorn]

Perhaps I've made this more complex than necessary. Updating my Carthage handling will definitely ease some of this. 😅

[ollieatkinson]

Thanks for taking the time to get that example written down with some diagrams.
I've noticed in your use-case being able to specify Relative and Absolute paths would be a benefit, so you could loose the relative pathing and just be able to specify its location from the root of the project.

e.g.

.framework(path: "//Carthage/iOS/Build/FancyButton.framework")

similarly for everywhere else which defines a path.

with just the above you should be able to reduce the amount of work you are required to do inside of each Project.swift file, and In addition to that the work I am doing to provide different configuration options will only aid you more.

I would really advise against doing any code inside of the Project.swift file, you should prefer declarative syntax as it allows to make easier assumptions about the Project file and could even help with migrations in the future.

Take a watch of the SwiftPM video from the WWDC 2018 session "Getting to know SwiftPM" https://developer.apple.com/videos/play/wwdc2018/411/ - at about 13:40 he talks in more detail about why you should prefer declarative.

Do you think with the above changes it will help tuist fit your requirements?

[enhorn]

Yeah. Relative paths would look cleaner. :-)

The issue I've solved right now is that we have about ~30 targets (defined in 11 Project.swift files) located in sub-projects, plus the main app target with it's extensions (That can't be generated with Tuist yet), so having all Carthage dependencies listed flat in all projects would mean we either have all targets linking agains all frameworks all the time (not good, as not all our Carthage dependencies are extension safe), or we would need to keep an up to date™ documentation of which dependencies that is needed at which level.

As it is now, targets inherit their Carthage dependencies through their "Carthage-dependencies" file, and they only need to add any additional they them self use in their own dependencies file.

It's generally a workaround for: #253

[Rag0n]

@enhorn
Main issue is tuist doesn't support transitive dependencies for frameworks(only for static frameworks).

[enhorn]

Indeed.

And with the recent fix for transitive dependencies, my need for this has been met. :-)

No more enum Setup 👏.

[kwridan]

Glad to hear the issue you were facing has been resolved @enhorn.

I was curious about the technical feasibility of this feature so I pushed a proof of concept.

My findings so far:

• It is technically feasible to support includes, in the proof of concept it was achieved as follows

include("relative/path/to/Helpers.swift")

let project = myCustomProjectHelper(name: "MyProject")

• Includes sadly introduce / surface some additional challenges, especially related to paths.

e.g.

// Helpers.swift

let sharedSettings = Settings(base: ["base": "base"],
                               debug: Configuration(settings: ["debug": "debug"],
                                                    xcconfig: "path/debug.xcconfig"),
                               release: Configuration(settings: ["release": "release"],
                                                      xcconfig: "path/release.xcconfig"))

// Project.swift

include("relative/path/to/Helpers.swift")

let project = Project(name: "MyProject",
                                   settings: sharedSettings)

Ideally one would expect the path specified in Helpers.swift is relative to the helpers file however, sadly that won't be the case as once used via include it acts as it would when defined within the project manifest. Out of the box this won't work without solving #249 & #293 somehow.

• Not sure if anyone relied on #file to introspect the current path to provide names or paths within the manifest - that will cease to work correctly, as the technique I followed in the proof of concept has all files copied to a temporary directory.

• The steps required to get this working are effectively compiling/linking/running a whole executable which can introduce a performance penalty - though it's hard to really judge it without some numbers (I've added Performance metrics - time the generation process #329 for that 🙂).

• The additional steps may reduce the "it just works" factor of Tuist. For example, during my prototyping I noticed I had to add special cases for scenarios when running via Xcode vs the command line. Also I had to ensure I installed the Swift 5 runtime (as I was on an older Mojave version). Of course these aren't necessarily blockers but something to consider.

If the only use case we currently have is configurations, perhaps the Environments concept may be a better fit (even though less flexible than full fledged includes).

[kwridan]

Resolved via #668

[pepicrft]