Binary downloads from 3rd party sources

[BernardGatt]

I know this topic has been raised multiple times but since we heavily depend on Carthage I need to know if there's an official stand, we depend on Carthage so much that we ended up creating a whole ecosystem of tools to support it.

• Rome - An asset server (to store binaries)
• Rome Build - Builds & uploads dependencies on Rome using Carthage
• Our own Carthage fork - A quick and dirty integration of Rome into Carthage

These tools helped us reduce dependency build time from 30min to around 2minutes (as we only build what's required), unfortunately the downside with having a fork is that we need to keep it up to date with the latest fixes/changes because we obviously don't want to lose compatibility with Carthage.

My question is this, would you consider adding the ability to support binary downloads from alternate sources instead of just GitHub? I'm not asking you to add individual support for Rome but to separate dependency resolution & building from the downloading of binaries.

[mdiep]

My question is this, would you consider adding the ability to support binary downloads from alternate sources instead of just GitHub?

Yes. But: it needs be automatic, discoverable, and decentralized.

I'm willing to review and consider pull requests and/or proposals. (But I won't have time to do the work myself.)

But if your primary interest is in caching for CI builds, then I think #1222 would be more useful. I can easily see that being extended to a global LRU cache of dependencies.

The idea of caching built products seems very different than distributing pre-built binaries, although there is some overlap. I'd encourage you to focus on the former because I think that's an easier, more generable, and more widely usable problem.

[BernardGatt]

I don't think people have issues with github for the distribution of binaries on public repositories, but as soon as you start building projects with many internal dependencies the need of a caching mechanism becomes evident.

In fact the utilities we buit are only intended to solve every day issues in big teams / projects.

Carthage already has the functionality to build and archive dependencies, this enables everyone to do what they want with their binaries.

Do you like the idea of adding a --skip command to update, build & bootstrap? This will open up the possibility to plug in utilities like RomeBuild which read the Cartfile.resolved and know which binaries are already cached, these utilities can then extract the cached binaries in a folder where Carthage is building the depenendencies and leave Carthage to build the rest.

[mdiep]

Do you like the idea of adding a --skip command to update, build & bootstrap? This will open up the possibility to plug in utilities like RomeBuild which read the Cartfile.resolved and know which binaries are already cached, these utilities can then extract the cached binaries in a folder where Carthage is building the depenendencies and leave Carthage to build the rest.

You can do that today by passing the dependencies that you want to build. You should be able to calculate that easily by reading the list of resolved dependencies, removing the ones you want to skip, and passing the remaining to Carthage.

[BernardGatt]

Does it handle transitive dependencies as well? So if I need to build A and A depends on B & C does it build A and use pre-built binaries for B & C? if that's the case I imagine the pre-built frameworks would have to be in the Carthage/Build/platform directory right?

[mdiep]

Currently, building A will also build B & C. But I'm definitely open to adding an option for that.

The pre-built frameworks would need to be in the right directory, yes.

[jeeftor]

I would really like to see this work -> we've come up with a hacky solution of our own where i basically am building the entire carthage directory on our CI server and then distributing the carthage directory as an artifact to projects.

[BernardGatt]

RomeBuild works great if you have a CI server cos you can run it on every commit (it will only build the dependencies that have changed), for now that's the best setup we've got, i'll most probably start looking at this issue later this month.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.