Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Carthage doesn't cache build across projects (or CI builds) #2400

Open
kenji21 opened this Issue Apr 1, 2018 · 43 comments

Comments

Projects
None yet
6 participants
@kenji21
Copy link
Contributor

kenji21 commented Apr 1, 2018

  • carthage install method: [ ] .pkg, [X] homebrew, [ ] source
  • which carthage: /usr/local/bin/carthage
  • carthage version: 0.29.0
  • xcodebuild -version: Xcode 9.3 Build version 9E145
  • Are you using --no-build? no
  • Are you using --no-use-binaries? yes
  • Are you using --use-submodules? no
  • Are you using --cache-builds? yes
  • Are you using --new-resolver? no

Cartfile

github "SVProgressHUD/SVProgressHUD" ~> 2.0

Carthage Output

10:35:30 kenji@macboo:~/Desktop/CarthageCacheTest
cat Cartfile
github "SVProgressHUD/SVProgressHUD" ~> 2.0

10:35:33 kenji@macboo:~/Desktop/CarthageCacheTest
cat Cartfile.resolved
github "SVProgressHUD/SVProgressHUD" "2.2.4"

10:35:34 kenji@macboo:~/Desktop/CarthageCacheTest
rm -Rf Carthage ; time carthage bootstrap --no-use-binaries --cache-builds --platform iOS
*** Checking out SVProgressHUD at "2.2.4"
*** No cache found for SVProgressHUD, building with all downstream dependencies
*** xcodebuild output can be found in /var/folders/7v/dfg0p2vj1z58n65d99l2n1lh0000gp/T/carthage-xcodebuild.5CyxXz.log
*** Building scheme "SVProgressHUD-Framework" in SVProgressHUD.xcodeproj

real	0m25.314s
user	0m24.218s
sys	0m7.705s

10:36:21 kenji@macboo:~/Desktop/CarthageCacheTest
ls -l /Users/kenji/Library/Caches/org.carthage.CarthageKit/DerivedData/9.3_9E145/SVProgressHUD/
total 0
drwxrwxr-x@ 6 kenji  staff  192 Apr  1 10:36 2.2.4
drwxrwxr-x@ 6 kenji  staff  192 Apr  1 10:19 2.2.5

10:36:28 kenji@macboo:~/Desktop/CarthageCacheTest
rm -Rf Carthage ; time carthage bootstrap --no-use-binaries --cache-builds --platform iOS
*** Checking out SVProgressHUD at "2.2.4"
*** No cache found for SVProgressHUD, building with all downstream dependencies
*** xcodebuild output can be found in /var/folders/7v/dfg0p2vj1z58n65d99l2n1lh0000gp/T/carthage-xcodebuild.fpieKP.log
*** Building scheme "SVProgressHUD-Framework" in SVProgressHUD.xcodeproj

real	0m25.840s
user	0m25.477s
sys	0m6.808s

Actual outcome
Carthage did not use cached build if I rm the Carthage directory of my working copy (what our CI does before each build)

Expected outcome
Carthage should use the previously compiled framework stored in ~/Library/Caches/org.carthage.CarthageKit/DerivedData when running the second bootstrap command

I used SVProgressHUD here to have a "quickly recompiled" framework, but it is also the other dependencies (and our CI jobs are taking ages to recompile each time all frameworks)

@blender blender added the question label Apr 1, 2018

@blender

This comment has been minimized.

Copy link
Member

blender commented Apr 1, 2018

Thanks for reporting, I'm not sure this behavior is expected. Further investigation is needed.

@kenji21 for time being you this might help you.

@mdiep

This comment has been minimized.

Copy link
Member

mdiep commented Apr 1, 2018

This is the expected behavior. --cache-builds caches build products, but does not alter the behavior of using a clean vs. a dirty build.

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Apr 2, 2018

@blender, thanks for pointing me to Rome, this is an interesting tool to share builds via the CI server (I'll need to think a bit more about what to do with this tool)

Usage of submodules have the same behavior :

$ carthage update --no-use-binaries --cache-builds --platform iOS --use-submodules
$ rm -Rf Carthage/Build/ ; time carthage bootstrap --no-use-binaries --cache-builds --platform iOS
*** Checking out SVProgressHUD at "2.2.5"
*** No cache found for SVProgressHUD, building with all downstream dependencies
*** xcodebuild output can be found in /var/folders/7v/dfg0p2vj1z58n65d99l2n1lh0000gp/T/carthage-xcodebuild.HYI0Hr.log
*** Building scheme "SVProgressHUD-Framework" in SVProgressHUD.xcodeproj

it is the Build directory which is "key" to cache (deleting this folder or the framework subfolder lead to a "Invalid cache found for SVProgressHUD, rebuilding with all downstream dependencies")

@olejnjak

This comment has been minimized.

Copy link
Contributor

olejnjak commented Apr 2, 2018

@kenji21 on our Jenkins I do something similar like Rome, but less sophisticated, when Jenkins workspace is wiped, then I look into predefined cache directory (e.g. ~/Library/Caches/ci/<project_name>) and then sync it to the workspace. After dependency artifacts are built, I sync them back to the cache directory.

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Apr 3, 2018

I like the way Carthage caches (e.g. using Xcode/swift version and dependency version in the path), does a PR adding an option like --use-cached-builds (only on bootstrap command, and why not on update, let's say when 2 projects have similar dependencies and are updated the same day) has a chance to be merged @mdiep ? I think it can because it explicitly shows that it will use cached build (if available, reusing the ProjectEvent.buildingUncached if not already compiled)

@mdiep

This comment has been minimized.

Copy link
Member

mdiep commented Apr 3, 2018

I'm not sure what you're asking.

@olejnjak

This comment has been minimized.

Copy link
Contributor

olejnjak commented Apr 3, 2018

What I see in this is that @kenji21 would like to cache build artifacts in ~/Library/Caches/<carthagekit>. And the question would be if it makes any sense to you 🤷‍♂️

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Apr 3, 2018

With the new Xcode / swift version (4.1), we have to "bootstrap or update" our dependencies on different projects having common dependencies (Alamofire, RxSwift...), same could occurs if Alamofire post a security update

I want to share carthage builds across projects (or when runing CI builds with Carthage/Build not checked in, which have a simpler way to reproduce the cache usage / non-usage)

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Apr 4, 2018

One more use case: if a compilation fails during update (let's say the 7th out of 10), relaunching carthage update will recompile all dependencies, with --use-cached-builds it won't be the case and fixing the failing depency will be easier (real use case as it happened to me with a swiftlint script phase making a xcodebuild command fail)

@ikesyo

This comment has been minimized.

Copy link
Member

ikesyo commented Apr 4, 2018

We does not specify clean action explicitly but Xcode may silently do clean build for archive action ( carthage uses archive action for device builds).

https://stackoverflow.com/questions/46211606/xcodebuild-archive-always-performs-a-clean-build

So this will not be an our fault I think.

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Apr 4, 2018

I think I understand what carthage means by "Valid cache":

carthage bootstrap --no-use-binaries --cache-builds --platform iOS
*** Checking out SVProgressHUD at "2.2.5"
*** xcodebuild output can be found in /var/folders/7v/dfg0p2vj1z58n65d99l2n1lh0000gp/T/carthage-xcodebuild.MYO3mI.log
*** Valid cache found for SVProgressHUD, skipping build

It is the $PWD/Carthage/Build/iOS/SVProgressHUD.framework*** files that is called "cache" by carthage, I was thinking of cache to be located at ~/Library/Caches/org.carthage.CarthageKit/DerivedData/9.3_9E145/SVProgressHUD/2.2.5/

@ikesyo

This comment has been minimized.

Copy link
Member

ikesyo commented Apr 4, 2018

Yes, since you are deleting Carthage directory at all, Carthage can't find any cached builds. So this is the expected behavior.

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Apr 4, 2018

So the aim of the PR adding option --use-cached-builds will be to copy ~/Library/Caches/org.carthage.CarthageKit/DerivedData/9.3_9E145/SVProgressHUD/2.2.5/....framework + dSYM to $PWD/Carthage/Build/iOS/ instead of running xcodebuild archive

As I assume that --cache-builds do the opposite (copy the builded framework + dSYM to ~/Library/Caches/Carthage...)

@stale

This comment has been minimized.

Copy link

stale bot commented Jun 30, 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jun 30, 2018

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Jul 2, 2018

Am I right when I say:

I assume that --cache-builds do the opposite (copy the builded framework + dSYM to ~/Library/Caches/Carthage...)

I still like to have the --use-cached-builds (e.g. make a PR) in comment #2400 (comment) but I don't want to spend time on such a "non-mergeable request" PR

@stale stale bot removed the stale label Jul 2, 2018

@olejnjak

This comment has been minimized.

Copy link
Contributor

olejnjak commented Jul 3, 2018

Well, I recently switched to Rome and now I think that such PR would be redundant as it is much more powerful than behavior described above 🤷‍♂️

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Jul 4, 2018

To sum up:

--cache-build doesn't use files from ~/Library/Caches/org.carthage.CarthageKit/DerivedData/, but only compares .version files in $PWD/Carthage/Builds (

guard options.cacheBuilds && projects.isDisjoint(with: projectsToBeBuilt) else {
return dependenciesIncludingNext
}
guard let versionFileMatches = matches else {
self._projectEventsObserver.send(value: .buildingUncached(nextDependency.0))
return dependenciesIncludingNext
}
if versionFileMatches {
self._projectEventsObserver.send(value: .skippedBuildingCached(nextDependency.0))
)

When building, the DerivedData path used is generated:

let baseURL = options.derivedDataPath.flatMap(URL.init(string:)) ?? Constants.Dependency.derivedDataURL
let derivedDataPerXcode = baseURL.appendingPathComponent(self.xcodeVersionDirectory, isDirectory: true)
let derivedDataPerDependency = derivedDataPerXcode.appendingPathComponent(dependency.name, isDirectory: true)
let derivedDataVersioned = derivedDataPerDependency.appendingPathComponent(version.commitish, isDirectory: true)
options.derivedDataPath = derivedDataVersioned.resolvingSymlinksInPath().path

As xcodebuild archive makes a clean build, even specifying same DerivedData path make all projects being rebuild from scratch...

So, you're right @olejnjak, my needs are what Rome provides, but as I only want to keep it "local" (no need for S3, minio or ceph), I was thinking by error that Carthage with --cache-builds provides this feature, as carthage help bootstrap (with latest 0.30.1) states:

[--cache-builds]
	use cached builds when possible

My bad here, because I haven't RTFM enough the README.md#caching-builds (maybe a link to this can be added to the help?)

And Rome doesn't know which Xcode (so, swift version) build your framework, so we should always use [--cache-prefix](https://github.com/blender/Rome#cache-prefix) $(xcodebuild -version | xargs | sed -E 's# #_#g') to prevent, (Carthage know this). And Rome doesn't fix the issue of #issuecomment-378474277 with the way it tell us to update (using xargs to tell carthage to rebuild all missing one in one step, it should be done in a shell while loop)

I still think that a local cache can easily be included within Carthage, maybe copying built frameworks/dSYMs in a directory, with same path generation as DerivedData, but prefixed by SharedCache instead of DerivedData, plus platform-commitish of .version, with a new option like --share-builds which doesn't interfere with dev workflow, and makes frameworks not being recompiled for a single platform if already in the "SharedCache"

Exemple:

$ carthage bootstrap --no-use-binaries --cache-builds --platform iOS
*** No Cartfile.resolved found, updating dependencies
*** Fetching SVProgressHUD
*** Checking out SVProgressHUD at "2.2.5"
*** No cache found for SVProgressHUD, building with all downstream dependencies
*** xcodebuild output can be found in /var/folders/7v/dfg0p2vj1z58n65d99l2n1lh0000gp/T/carthage-xcodebuild.JzLTs2.log
*** Building scheme "SVProgressHUD-Framework" in SVProgressHUD.xcodeproj
$ ls Carthage/Build/iOS/
BBE05204-6206-30DE-B075-9009DFB73858.bcsymbolmap  FF12DFCB-55B3-38C3-BDBB-F1CF41A02B7E.bcsymbolmap  SVProgressHUD.framework/                          SVProgressHUD.framework.dSYM/
$ ls ~/Library/Caches/org.carthage.CarthageKit/SharedCache/9.3_9E145/SVProgressHUD/2.2.5/iOS/
BBE05204-6206-30DE-B075-9009DFB73858.bcsymbolmap FF12DFCB-55B3-38C3-BDBB-F1CF41A02B7E.bcsymbolmap SVProgressHUD.framework                          SVProgressHUD.framework.dSYM

Important command here is : ls ~/Library/Caches/org.carthage.CarthageKit/SharedCache/9.3_9E145/SVProgressHUD/2.2.5/iOS/

Second "clean" bootstrap (as jenkins would do):

$ rm -Rf Carthage
$ carthage bootstrap --no-use-binaries --cache-builds --platform iOS
*** Checking out SVProgressHUD at "2.2.5"
*** xcodebuild output can be found in /var/folders/7v/dfg0p2vj1z58n65d99l2n1lh0000gp/T/carthage-xcodebuild.HYzyWv.log
*** Valid shared cache found for SVProgressHUD, skipping build

And all files from ~/Library/Caches/org.carthage.CarthageKit/SharedCache/9.3_9E145/SVProgressHUD/2.2.5/ have been copied to $PWD/Carthage/Build/ (depends on --platform filter)

@blender

This comment has been minimized.

Copy link
Member

blender commented Jul 4, 2018

--cache-build doesn't use files from ~/Library/Caches/org.carthage.CarthageKit/DerivedData/, but only compares .version files in $PWD/Carthage/Builds

Correct

my needs are what Rome provides, but as I only want to keep it "local"

You can only specify a local cache to keep it local. You don't need to provide S3 or other caches.

#issuecomment-378474277

Can you fix this link please so that I can read the comment?

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Jul 4, 2018

Fixed, I was thinking anchor link on "local page" was working

@blender blender changed the title Jenkins CI doesn't cache build across projects Carthage doesn't cache build across projects Jul 4, 2018

@blender

This comment has been minimized.

Copy link
Member

blender commented Jul 4, 2018

From #2400 (comment)

carthage update will recompile all dependencies

It won't if you use --cache-builds. That's the point of it.

This case is already covered. Your problem is a cold bootstrap.

From #2400 (comment)

And Rome doesn't know which Xcode (so, swift version) build your framework, so we should always use --cache-prefix $(xcodebuild -version | xargs | sed -E 's# #_#g') to prevent, (Carthage know this)

Carthage doesn't know either, it uses the current xcode-selected version.

I'll follow up with another comment formalizing your problem.

@blender

This comment has been minimized.

Copy link
Member

blender commented Jul 4, 2018

So, to formalize a little for @mdiep

Problem

  • Project A uses carthage
  • Project B uses carthage
  • Project A and B don't depend on each other, they are separate, for example two distinct apps
  • Project A and B happen to have a common dependency, Framework C (same version in both projects)

When running carthage boostrap on Project A, Framework C is built.
When running carthage boostrap on Project B, Framework C is built.

Current Status

  • Carthage does not share Framework C across boostraps in different projects
  • Extra tools are needed to share frameworks across boostraps like Rome
  • Carthage only provides a cache for rebuilds in the same project if bootstrap or update are called with. --cache-builds

Improvement Suggestion

Add a feature to share artefacts across boostraps (and updates)

--cache-builds is still relevant and needed for binary artifacts.

Implementation

  • Introduce a new flag --share-builds to active the feature
  • When the flag is active, check a known location on disk (cache) for previous artifacts
  • Copy artifacts from the known location

About the bootstrap cache

  • Location: ~/Library/Caches/org.carthage.CarthageKit/SharedCache
  • Structure: <xcode-version>/<repository-name>/<comittish-or-tag>/<platform>/
@blender

This comment has been minimized.

Copy link
Member

blender commented Jul 4, 2018

My personal comment:

  • What you need is already covered by the extra tool (the cache structure is almost the same), however I can see how this can be a nice addition
  • I'm not a big fan of adding yet another flag to the command line. It's becoming a jungle imho.
  • I'm worried about the confusion between this and --cache-builds, so I would rather improve that instead to achieve what is proposed here
@ikesyo

This comment has been minimized.

Copy link
Member

ikesyo commented Jul 29, 2018

This would be a duplicate of #1588.

@stale

This comment has been minimized.

Copy link

stale bot commented Aug 28, 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@blender blender added the enhancement label Sep 13, 2018

kenji21 added a commit to openium/Carthage that referenced this issue Sep 18, 2018

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Sep 18, 2018

Just committed on my fork : openium@a9ac3dc

Refactored many times to do less possible modifications. I reused the "copyFramework" from binaries to put them in and out of the "shared cache"

Bigger one is that I move method from Project to more global functions (they were only using directoryURL attribute, so I moved it to a parameter)

Tested with multi-platforms dependencies (and having nested deps):

git "https://github.com/radex/SwiftyUserDefaults" ~> 3.0
git "https://github.com/tristanhimmelman/AlamofireObjectMapper" ~> 5.0

I tried to respect the "reactive" way the more I can, but as it's relatively new to me... maybe I didn't make things in a correct fashion

Maybe a message should be shown to the user when a dependency frameworks are copied to the "shared cache" (e.g. ~/Library/Caches/org.carthage.CarthageKit/SharedCache)

What I tried to do but rollbacked, is to remove the "Carthage/Build" folder from the shared cache path of a dependency, because it is added in many points (so I prefer don't break any thing, searching for Constants.binariesFolderPath in Xcode shows this)

Here is an example of the shared cached version of ObjectMapper:

.../SharedCache/9.4.1_9F2000/
└── ObjectMapper
    └── 2.2.9
        └── Carthage
            └── Build
                ├── .ObjectMapper.version
                ├── Mac
                │   ├── ObjectMapper.framework
                │   │   ├── Headers -> Versions/Current/Headers
                │   │   ├── Modules -> Versions/Current/Modules
                │   │   ├── ObjectMapper -> Versions/Current/ObjectMapper
                │   │   ├── Resources -> Versions/Current/Resources
                │   │   └── Versions
                │   │       ├── A
                │   │       │   ├── Headers
                │   │       │   │   ├── ObjectMapper-Swift.h
                │   │       │   │   └── ObjectMapper.h
                │   │       │   ├── Modules
                │   │       │   │   ├── ObjectMapper.swiftmodule
                │   │       │   │   │   ├── x86_64.swiftdoc
                │   │       │   │   │   └── x86_64.swiftmodule
                │   │       │   │   └── module.modulemap
                │   │       │   ├── ObjectMapper
                │   │       │   └── Resources
                │   │       │       └── Info.plist
                │   │       └── Current -> A
                │   └── ObjectMapper.framework.dSYM
                │       └── Contents
                │           ├── Info.plist
                │           └── Resources
                │               └── DWARF
                │                   └── ObjectMapper
                ├── iOS
                │   ├── 3EAF3E6D-9382-3FAD-AD87-8AAEFE14AB30.bcsymbolmap
                │   ├── CCF90AA3-2D89-3FC0-949F-797C8F3F5864.bcsymbolmap
                │   ├── ObjectMapper.framework
                │   │   ├── Headers
                │   │   │   ├── ObjectMapper-Swift.h
                │   │   │   └── ObjectMapper.h
                │   │   ├── Info.plist
                │   │   ├── Modules
                │   │   │   ├── ObjectMapper.swiftmodule
                │   │   │   │   ├── arm.swiftdoc
                │   │   │   │   ├── arm.swiftmodule
                │   │   │   │   ├── arm64.swiftdoc
                │   │   │   │   ├── arm64.swiftmodule
                │   │   │   │   ├── i386.swiftdoc
                │   │   │   │   ├── i386.swiftmodule
                │   │   │   │   ├── x86_64.swiftdoc
                │   │   │   │   └── x86_64.swiftmodule
                │   │   │   └── module.modulemap
                │   │   └── ObjectMapper
                │   └── ObjectMapper.framework.dSYM
                │       └── Contents
                │           ├── Info.plist
                │           └── Resources
                │               └── DWARF
                │                   └── ObjectMapper
                ├── tvOS
                │   ├── B8E64EDB-1077-30E4-AF82-01C752B58F51.bcsymbolmap
                │   ├── ObjectMapper.framework
                │   │   ├── Headers
                │   │   │   ├── ObjectMapper-Swift.h
                │   │   │   └── ObjectMapper.h
                │   │   ├── Info.plist
                │   │   ├── Modules
                │   │   │   ├── ObjectMapper.swiftmodule
                │   │   │   │   ├── arm64.swiftdoc
                │   │   │   │   ├── arm64.swiftmodule
                │   │   │   │   ├── x86_64.swiftdoc
                │   │   │   │   └── x86_64.swiftmodule
                │   │   │   └── module.modulemap
                │   │   └── ObjectMapper
                │   └── ObjectMapper.framework.dSYM
                │       └── Contents
                │           ├── Info.plist
                │           └── Resources
                │               └── DWARF
                │                   └── ObjectMapper
                └── watchOS
                    ├── E84BC4E9-0225-3546-B7D2-B0CC82D77C42.bcsymbolmap
                    ├── ObjectMapper.framework
                    │   ├── Headers
                    │   │   ├── ObjectMapper-Swift.h
                    │   │   └── ObjectMapper.h
                    │   ├── Info.plist
                    │   ├── Modules
                    │   │   ├── ObjectMapper.swiftmodule
                    │   │   │   ├── arm.swiftdoc
                    │   │   │   ├── arm.swiftmodule
                    │   │   │   ├── i386.swiftdoc
                    │   │   │   └── i386.swiftmodule
                    │   │   └── module.modulemap
                    │   └── ObjectMapper
                    └── ObjectMapper.framework.dSYM
                        └── Contents
                            ├── Info.plist
                            └── Resources
                                └── DWARF
                                    └── ObjectMapper
@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Sep 26, 2018

using it on our CI since few days, it works nicely and allow us to not rebuild all dependencies (only new ones or new versions of current ones) at each integrations :-)

@Max0u

This comment has been minimized.

Copy link

Max0u commented Sep 26, 2018

I'm not sure to understand what happened, I also use Carthage to our Gitlab CI but it don't find any Cache, what can I do ?

@blender

This comment has been minimized.

Copy link
Member

blender commented Sep 26, 2018

Wait, this feature is under development.

For a more sophisticated set up you can use https://github.com/blender/Rome

kenji21 added a commit to openium/Carthage that referenced this issue Oct 3, 2018

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Oct 3, 2018

Found an unrelated issue while copying some from shared cache (our CI job randomly fails, command carthage bootstrap --no-use-binaries --cache-builds --log-path carthage-boostrap.log --new-resolver --platform iOS):

A shell task (/usr/bin/xcrun dwarfdump --uuid /Users/hudson/Desktop/project/Carthage/Build/iOS/RxCocoa.framework/RxCocoa) failed with exit code 1:
error: unable to open ''

Which is due to multiple versions of RxCocoa copied because of "dependency of dependency", I added a debug print("copying product from \(from) to \(to)") to be able to determine what was happening (in Xcode.swift: copyProduct(from:,to:) func):

copying product from file:///Users/hudson/Library/Caches/org.carthage.CarthageKit/SharedCache/10.0_10A255/RxSwift/4.3.1/Carthage/Build/iOS/RxCocoa.framework/ to file:///Users/hudson/Desktop/project/Carthage/Build/iOS/RxCocoa.framework
copying product from file:///Users/hudson/Library/Caches/org.carthage.CarthageKit/SharedCache/10.0_10A255/RxDataSources/3.1.0/Carthage/Build/iOS/RxCocoa.framework/ to file:///Users/hudson/Desktop/zastuces.ios/Carthage/Build/iOS/RxCocoa.framework/

The Cartfile.resolved (with minimal dependencies):

github "ReactiveX/RxSwift" "4.3.1"
github "RxSwiftCommunity/RxDataSources" "3.1.0"
github "RxSwiftCommunity/RxGesture" "1.2.1"
github "RxSwiftCommunity/RxSwiftExt" "3.3.0"

Having the print in the copyProduct function allowed me to validate that what is copied out of the SharedCache is correct), so now I need to make it "not copying in parallel"

kenji21 added a commit to openium/Carthage that referenced this issue Oct 3, 2018

kenji21 added a commit to openium/Carthage that referenced this issue Oct 3, 2018

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Oct 3, 2018

Fixed the issue (updated the commit in previous comment by amending it my clone & rebased to current master)

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Oct 5, 2018

Here is an example of output produced during a carthage update:

*** Building scheme "ImagePicker-iOS" in ImagePicker.xcodeproj
copying product from file:///Users/kenji/Library/Caches/org.carthage.CarthageKit/DerivedData/10.0_10A255/ImagePicker/2.1.1/Build/Intermediates.noindex/ArchiveIntermediates/ImagePicker-iOS/IntermediateBuildFilesPath/UninstalledProducts/iphoneos/ImagePicker.framework/ to file:///Users/kenji/source-cache/project/Carthage/Build/iOS/ImagePicker.framework
copying product from file:///Users/kenji/Library/Caches/org.carthage.CarthageKit/DerivedData/10.0_10A255/ImagePicker/2.1.1/Build/Intermediates.noindex/ArchiveIntermediates/ImagePicker-iOS/BuildProductsPath/Release-iphoneos/8E23C5C1-CA32-34FE-AB30-631545FA2B14.bcsymbolmap to file:///Users/kenji/source-cache/project/Carthage/Build/iOS/8E23C5C1-CA32-34FE-AB30-631545FA2B14.bcsymbolmap
copying product from file:///Users/kenji/Library/Caches/org.carthage.CarthageKit/DerivedData/10.0_10A255/ImagePicker/2.1.1/Build/Intermediates.noindex/ArchiveIntermediates/ImagePicker-iOS/BuildProductsPath/Release-iphoneos/E898DF97-9979-3792-8C31-FAF37A332F96.bcsymbolmap to file:///Users/kenji/source-cache/project/Carthage/Build/iOS/E898DF97-9979-3792-8C31-FAF37A332F96.bcsymbolmap
copying product from file:///Users/kenji/source-cache/project/Carthage/Build/iOS/ImagePicker.framework to file:///Users/kenji/Library/Caches/org.carthage.CarthageKit/SharedCache/10.0_10A255/ImagePicker/2.1.1/Carthage/Build/iOS/ImagePicker.framework
copying product from file:///Users/kenji/source-cache/project/Carthage/Build/iOS/ImagePicker.framework.dSYM/ to file:///Users/kenji/Library/Caches/org.carthage.CarthageKit/SharedCache/10.0_10A255/ImagePicker/2.1.1/Carthage/Build/iOS/ImagePicker.framework.dSYM
copying product from file:///Users/kenji/source-cache/project/Carthage/Build/iOS/E898DF97-9979-3792-8C31-FAF37A332F96.bcsymbolmap to file:///Users/kenji/Library/Caches/org.carthage.CarthageKit/SharedCache/10.0_10A255/ImagePicker/2.1.1/Carthage/Build/iOS/E898DF97-9979-3792-8C31-FAF37A332F96.bcsymbolmap
copying product from file:///Users/kenji/source-cache/project/Carthage/Build/iOS/8E23C5C1-CA32-34FE-AB30-631545FA2B14.bcsymbolmap to file:///Users/kenji/Library/Caches/org.carthage.CarthageKit/SharedCache/10.0_10A255/ImagePicker/2.1.1/Carthage/Build/iOS/8E23C5C1-CA32-34FE-AB30-631545FA2B14.bcsymbolmap

Can I send a PR (without the log of course) ?

@blender

This comment has been minimized.

Copy link
Member

blender commented Oct 5, 2018

@kenji21 yep

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Oct 8, 2018

Travis is failing (due to shared cache) I'm fixing this and will update the PR

kenji21 added a commit to openium/Carthage that referenced this issue Oct 8, 2018

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Oct 8, 2018

CI now runs fine

kenji21 added a commit to openium/Carthage that referenced this issue Oct 29, 2018

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Oct 29, 2018

Updated the PR to current master : #2608

kenji21 added a commit to openium/Carthage that referenced this issue Oct 31, 2018

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Oct 31, 2018

Updated the PR again as Xcode 10.1 is out (and our CI was failing building dependencies with the simulator not found issue, so I have run it with PR #2631):

$ ls -1 ~/Library/Caches/org.carthage.CarthageKit/SharedCache/
10.0_10A255
10.1_10B61
9.4.1_9F2000
$ ls -1 ~/Library/Caches/org.carthage.CarthageKit/SharedCache/10.1_10B61/Alamofire
4.7.3

kenji21 added a commit to openium/Carthage that referenced this issue Jan 9, 2019

@kenji21 kenji21 changed the title Carthage doesn't cache build across projects Carthage doesn't cache build across projects (or CI builds) Feb 4, 2019

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Feb 22, 2019

Here is an example of CI clean build times (on an iMac Pro) with a project having 17 "first level" dependencies (RxSwift, AlamofireObjectMapper...):

carthage 0.31.2 with the "SharedCache" :
ios-app - #646 Success after 3 min 11 sec (Open)
ios-app - #647 Success after 3 min 15 sec (Open)

carthage 0.32:
ios-app - #648 Success after 15 min (Open)

@kenji21

This comment has been minimized.

Copy link
Contributor Author

kenji21 commented Mar 1, 2019

Created a new PR (#2716) since I moved my commit to a branch (in order to send another PR fixing issue #2677)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.