RFC: Per dependancy platform setting

[mcfedr]

I want to make a suggestion, but looking for comments before working on an implementation, I couldn't find any existing discussion of this.

The idea is to be able to specify the build platforms in the Cartfile, for each dependency. I see two main reasons for doing so:

• When you first get a project, hit carthage bootstrap, by default it does builds for 4 platforms, but in most cases, you are only interested in one of these. If it was specified in the Cartfile you wouldn't have to know which platforms to include using --platforms at bootstrap.
• When you have mixed project, e.g. iOS app with watchOS embedded, you often have some dependencies that only need to be built for iOS, so that only need to be built for watchOS, and some for both. When using --platforms all your deps get built for iOS and watchOS.

It would require extending the Cartfile syntax, something like:

github "jdhealy/PrettyColors" ~> 5.0, platforms: iOS, watchOS
github "ReactiveCocoa/ReactiveSwift" ~> 3.0, platforms: iOS

Most likely the --platforms argument would override this, and build for everything specified

I think this can improve build times quite significantly.

[mdiep]

This has been discussed in #356.

[mcfedr]

So it looks like Carthage might be open to accepting such a pull request?

[tmspzz]

shoudn't the dependency have 3 dimensions? Version, Target, Platform?

[mcfedr]

What would target refer to?

[tmspzz]

Inside an Xcode Project there can be multiple targets.

[mdiep]

The big issue here is a design question: how should Cartfiles be extended?

If we're going to extend them, they should be extended with an eye toward the future, recognizing that we will probably want other additions in the future. Previously, some work was done to implement an OGDL parser that was compatible with existing Cartfiles. That was quite a while ago, and I'm not sure if it would be a good direction for today.

Once we have a format designed, then we can tackle a specific addition like this.

[tmspzz]

@mdiep Why OGDL rather than a yaml ?

The problem is that Carthage dependencies have no way to describe themselves.

Why not something like:

 Curry: &Curry # this is a package, describing a dependency
    name: Curry
    type: scm # or binary
    url: https://github.com/thoughtbot/Curry.git
    platforms: [macos, ios]
    version: 3.0
    tag: 3.0-golden-master
    macos: 
      targets:
          - name: Curry-Mac
    ios: 
     targets:
        - name: Curry-iOS


Fabric: &Fabric # this is a package, describing a dependency
    name: Fabric
    type: binary # one of binary, scm
    url: file:///tmp/Fabric.zip
    platforms: [macos, ios]
    version: 2.1
    ios:
        targets:
            - name: Fabric-iOS
    macos:
        targets:
            - name: Fabric-Mac

 
package: # this is a package, describing itself and it's dependencies
    name: Carthage
    type: scm # or binary
    url: https://github.com/Carthage/Carthage.git
    platforms: [macos]
    version: 0.27.0
    tag: 0.27.0
    macos: 
        targets:
           - name: carthage
             dependencies:
                 - package: *Curry #or URL to the package file
                   lock:
                       type: version # version of branch
                       resolution-strategy: strict # strict or compatible
                       identifier: 3.0 # or name and commit of the branch
                       targets: [Curry-Mac]
                 - package: *Fabric #or URL to the package file
                   lock:
                       type: version # version of branch
                       resolution-strategy: compatible #
                       identifier: 2.0
                       targets: [Fabric-iOS]

see in JSON at: http://yaml-online-parser.appspot.com/

A few advantages:

• If you know what a dependency looks like, you can fully describe it yourself locally as a package.
• For dependency resolutions only the package description is needed (not a checkout of the entire repo)
• Packages auto link to other packages
• Packages are searchable
• Extensible. New Platform? Add a section

If you are open to this approach I can submit a full proposal a-la swift-evolution.

Otherwise, do you have something else in mind?

[mdiep]

Why OGDL rather than a yaml ?

From the OGDL FAQ:

What are the differences between YAML and OGDL ?
Both use space to structure text, but they are quite different in syntax and underlying structure. YAML is a data serialization language that supports two complex types: maps (hashes) and lists (vectors). OGDL on the other hand, needs a different complex type, one that preserves order and can hold duplicate entries: a named list (Graph, in the APIs). YAML is complexer than OGDL, in terms of number of productions. For more information on YAML, see yaml.org.

OGDL was born, in fact, while trying to simplify the YAML syntax during the implementation of a Java parser for it. Similarities between both languages should be credited to the authors of YAML (Clark Evans, Brian Ingerson and Oren Ben-Kiki).

Why not something like

Because that's considerably more complex than what we have and, I think, more complex than what we need. Everything doesn't need to be configurable, but we want room for more than one piece of configuration.

[tmspzz]

I personally still don't buy OGDL, parsers for YAML are already there. I also don't see the the need for order preservation and duplicate entries. However if this is the preferred format I guess the same structure can be translated to OGDL.

The dependency format that I have sketched is very verbose and it can be reduce by adopting some syntactic sugar but it contains all the information that is needed for:

• representing a package
• building a dependency graph

It might look more complex to the eye at a first glance but the contents of it are very obvious.

What is lacking for dependency management on Apple platforms are the following things:

• Ability to describe a package
• Ability to describe the package's version
• Ability to describe what platforms the package is for
• Ability to describe what targets are for what platforms
• Ability to describe a dependency to another combination of package/platform/version/targets

@mdiep what would be the ideal next steps to move forward?

[mdiep]

I personally still don't buy OGDL, parsers for YAML are already there. I also don't see the the need for order preservation and duplicate entries. However if this is the preferred format I guess the same structure can be translated to OGDL.

I'm not personally tied to OGDL. It's just what was previously being considered.

What is lacking for dependency management on Apple platforms are the following things:

Ability to describe a package
Ability to describe the package's version
Ability to describe what platforms the package is for
Ability to describe what targets are for what platforms
Ability to describe a dependency to another combination of package/platform/version/targets

It's a little unclear whether you mean to specify these things for dependencies only or to have a project define information about itself. But we want to avoid the latter. Versions, targets, etc. all exist already in Git and Xcode files; we don't want to duplicate that information.

what would be the ideal next steps to move forward?

I think a concrete proposal would be the next step. This should happen in a new issue or possibly in a PR that updates documentation.

[tmspzz]

@mdiep Thanks for the clarification.

It's a little unclear whether you mean to specify these things for dependencies only or to have a project define information about itself.

Both are possible. A "package" described above is a subset of a dependency. A "dependency" contains more detailed "lock" information.

Versions, targets, etc. all exist already in Git and Xcode files; we don't want to duplicate that information.

They do, but the problem is that their relation to each others does not exist anywhere. That is what should be encoded. It's not possible to encode the relation between two targets without duplicating the the information from Xcode & Git.

How do you encode that Carthage depends on "Curry-Mac.framework" in project "Curry" at version 3.0 for platform "macos" without mentioning explicitly all 4?

The same information must be also available in some sort of manifest file Curry. Currently the manifest file is the sum of Git information + Xcode Project file. This requires a full checkout before anything can be said of the manifest itself.

I believe that this worked well so far but it's not sustainable for the future. Even Apple with SPM migrated away and duplicates information in the package file.

Now, I'm not saying that to support Carthage people will have to provide "package" information. In the solution above the user of the dependency himself can describe the package it depends upon, relieving the maintainers from this burden.

[mcfedr]

Most of these options seem to be ideas to replace the current Cartfile. Some of the options

• A standard data format, YAML, JSON, OGDL
  • Would allow future extensions to be added
  • Many package managers take this route, such as composer, npm
  • I would probably prefer YAML over other formats, its so widely used and understood, and easily readable. Also many existing parsers exist.
• Another option I see is to follow the route of SPM, Cocoapods, which use a compiled file format, Swift and Ruby respectively.
  • This allows for lots of flexibility as the code is run and come can calculate the results.
  • The disadvantages are more complexity in using the format, and potential security risks.
• A simple approach for Carthage users might be just to extend the current format
  • Easy for users to upgrade
  • custom format with just the required fields
  • requires maintaining custom parser
  • maybe less clear for future extensions

Backwards compatibility

• Extending the Cartfile breaks compatibility, the current parser throws on any extra content it doesn't expect.
• Additional files would be ignored by older Carthage
  • One could keep the existing Cartfile, and just specify new options in a separate file, seems undesirable to have to maintain two files
• It would be possible to generate a current version Cartfile from a new file, such as Cartfile.yml
  • this would keep old versions working
  • No manual work
  • Easily removed in the future
    Is this even an issue?

What are the desired fields in a new format?

• Everything in the current file
  • location of dep
  • version
• Required platform
• Required target?
  • Does that conflict with platform?
• Describe the current package?
  • What would go here? I tend to agree with people that the use of Xcode and Git to describe this is generally more desirable, why repeat it?

[mdiep]

The important bit is that we have backwards compatibility with the Cartfile format. i.e. future versions of Carthage should be able to read old Cartfiles; it's not necessary that older versions of Carthage continue to read new Cartfiles.

OGDL was looked into specifically because existing Cartfiles were valid OGDL.

The current format has worked well for us IMO, so we should strive to stay as close to that as possible.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[KyleLeneau]

Another option I see is to follow the route of SPM, Cocoapods, which use a compiled file format, Swift and Ruby respectively.

I like this idea, has this been explored more? The tools like Sourcery being more popular generating a template and format would be much easier and I think you’d be able to find a way to provide backwards support.

One thing not being mentioned is that if a new format is chosen then I think it needs to support Versioning of the schema so it’s allowed to grow with the tool. We could consider the current files and formats version: 0 and anything new needs to specify it on the first line of the file. This is common with file based CI tools that use yaml.

Either way, I am still more of a fan of yaml over OGDL cause I want to be able to extend the format to someday support a pre and post action for dependencies in Carthage (example: call the Makefile in a project rather then let xcodebuild happen automatically if it’s possible).