Iterating on Dependencies.swift

[pepicrft]

Dependencies.swift proved to be valuable for users because they can cache them using Tuist's caching capabilities and allows developers to overcome some challenges that they face when using Xcode's SPM integration, which resolves packages at startup time and invalidates them when users clean the environment.

Because of this, I think we should further invest in it, but I think we should revisit some aspects of it. The following are some problems that I've noticed with our approach with proposed solutions.

Drop Carthage support

Carthage is not actively maintained since January. Moreover, a lot of teams are migrating or have fully migrated to SPM. Because of this, I think we should consider dropping support for it and reducing the support surface to narrow the focus to Swift Packages. Since Tuist provides interfaces to link against system binaries, developers can continue using Carthage, but they'd have to wrap the tuist fetch command in a script that also runs carthage build (or the equivalent).

I believe this change will be beneficial for the maintenance of the project because we have very limited resources and the focus can scatter easily

Dependency auto-updating doesn't work

Tools like Dependabot which have support for Swift Packages are not able to update dependencies because we don't follow the convention of Swift Packages. This prevents Tuist users from users dependency auto-updating tools. Because of this, I think we should revisit our approach. If we drop support for Carthage frameworks, we can adopt Package.swift as the interface for defining dependencies. It'd be a Package.swift with no targets, only dependencies. The current logic would require a few changes because it uses the SPM and a Package.swift internally. The only difference is that we'd make a public thing instead of an internal implementation detail.

[thedavidharris]

+1 to both of these. The ecosystem has definitely moved to SPM and I think Tuist can reflect that safely.

We were longtime users of Carthage but dropped it multiple years ago due to the support situation, and once SPM started supporting binary artifacts some of the benefits are now no longer even present.

We keep a copy of the Package.swift and Package.resolved in the repo as well so that we can do checks like https://fossa.com/. Being able to have things like auto-updating and other tools that work on the Package ecosystem would bring a lot of convenience.

[pepicrft]

Hey folks 👋🏼
I gave this more thought, wondering if we could drop the concept altogether from Tuist.
The benefit of using this approach to dependencies is that:

1. Developers can separate dependency fetching and caching from project generation, meaning they generate a project ready to compile.
2. They can cache dependencies as binaries so the cleaning of derived data doesn't cause the purge of the Swift Packages that have been resolved.

However, it comes at a huge maintenance cost for us. Every time there's a new Xcode version, our integration breaks, which is unfortunate and time-consuming. I think Apple will adjust the CMD+K workflow not to delete the resolved graph locally, so that alleviates some of the pains a bit. If we stick to Apple's approach, Tuist users won't be able to cache the dependencies as binaries. Still, the dependencies will be cached transitively through the targets that depend on them.

I'm curious about what everyone thinks.

[danieleformichelli]

IMO caching, and in particular dependencies caching is a fundamental piece of the puzzle, so we cannot lose it.

Are you sure about this?

"Still, the dependencies will be cached transitively through the targets that depend on them.

[pepicrft]

Agree. Whatever low-maintenance alternative we'd develop must be integrated into the caching solution.
My current thinking is that when caching is not used, dependencies are integrated using Xcode's standard approach by default. This will cover most of the users of Tuist, which might not be using caching. For those using caching, we'll need to read the project and the packages graph to:

• Compile packages as .xcframeworks or frameworks.
• Mutate the graph to replace references to Swift Packages with references to binaries.

The biggest challenge will be understanding how SPM will integrate those packages at runtime (static or dynamic). This is a decision that Xcode, with the help of SPM, made at build-time and that we'd need to align with. We already have logic in place to detect whether a graph is valid, so we could leverage that logic to make a similar decision from the caching and generation logic.

Of course, this requires some prototyping to validate the ideas, but I like that for most developers, the integration of dependencies aligns with Xcode's proposal. We only deviate when you use caching.

[shahzadmajeed]

Does this proposal mean decentralizing dependencies and getting rid of Dependencies.swift? Would we still be able to detect cross-project dependency (version) conflicts?

[fortmarek]

We would still need to have a Package.swift which we already generate anyway based on Dependencies.swift as we need to retain the Package.resolved. Basically, we would remove our own format but most of the functionality should stay the same (apart from integrating non-cached packages as actual packages instead of library modules as we do now)