New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XCode10 (new build system) - if incremental build, Embed script doesn't run #8073

Closed
mkubista opened this Issue Sep 3, 2018 · 28 comments

Comments

Projects
None yet
@mkubista
Copy link

mkubista commented Sep 3, 2018

Report

What did you do?

ℹ I made changes in some code files in local development pod

What did you expect to happen?

ℹ After build and run in xCode, the changes are incorporated into the resulting package.

What happened instead?

ℹ The old binary is in package. The [CP] Embed Pods Frameworks script doesnt run. Its because of build optimalisation in new xCode - when it detects no change in input or output files specified for script, the script is not started.
When we did enter the binary file from the content of the .framework folder into the input files list, everything is OK. Clean build is also OK.
So the solution would be to have no input and out files at all, or during pod install add also content files of frameworks into the input files list.

Strange thing - in the build log in Report navigator, the output of the [CP] Embed Pods Frameworks appeared even when it didn't actually run (I modified the script to echo current time, so I can see the same time stamps)

CocoaPods Environment

   CocoaPods : 1.6.0.beta.1
        Ruby : ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-darwin16]
    RubyGems : 2.6.11
        Host : Mac OS X 10.14 (18A371a)
       Xcode : 10.0 (10L232m)
         Git : git version 2.17.1 (Apple Git-112)
Ruby lib dir : /usr/local/Cellar/ruby/2.4.1_1/lib
Repositories : artsy - https://github.com/Artsy/Specs.git @ 8c00e26ca1bd8ba163587f0eb592e9cd5a241d0e
               master - https://github.com/CocoaPods/Specs.git @ 058895fdadffd84d53c3ddc27a31591ea2f89ce7
               SSCPWrapper - https://tfs.kswr.cz/tfs/DefaultCollection/Jablotron/_git/MyCompany-iOS @ 518f383f1524b77aad7ddaa1d50eb72e4b824594

Project that demonstrates the issue

ℹ Unfortunately, project cannot be shown in public.

@Legoless

This comment has been minimized.

Copy link

Legoless commented Sep 3, 2018

I have the same issue. This causes me having to rebuild the entire project, which takes about 15 minutes. But I only changed a single line in code in one of the development pods.

@mkubista

This comment has been minimized.

Copy link
Author

mkubista commented Sep 3, 2018

I have a possible temporary solution
I added this Run script phase before Embed Pods Framework
touch "${PODS_ROOT}/*SCRIPT_DIRECTORY*frameworks.sh"
The real dirctory can be found in XCODE log of the build.
It forces xcode to run the script everytime. Because inside is used rsync command, the time to do this every-time isn't as much long

@dnkoutso

This comment has been minimized.

Copy link
Contributor

dnkoutso commented Sep 3, 2018

Even in the current build system the input/output files are used to speed up builds. If nothing has changed then there is no need to run these scripts.

Please include a sample app demonstrating this issue as I cannot reproduce locally.

@mkubista

This comment has been minimized.

Copy link
Author

mkubista commented Sep 3, 2018

Here is the sample project https://github.com/mkubista/CocoapodsTest
The problem can be simulated by this steps:

  1. Build and run the project - the viewcontroller with "Test 2" label should appear.
  2. Change the code in cocoapods module in CocoapodsTest.Submodule/CocoapodsTestSubmodule/Utils.swift for example to return "Test 1" instead.
  3. Build and run again - one would think the "Test 1" will appear but "Test 2" is here. Clean build will solve this.

Project is in git with all necessary files.
I hope the problem is not on my computer only (if so I have no clue what could cause this)

@stale stale bot removed the s1:awaiting input label Sep 3, 2018

@djbe

This comment has been minimized.

Copy link

djbe commented Sep 4, 2018

I have the same issue in a large project with around 6 development pods.

Whenever we modify anything in a development pod, we are forced to perform a full clean & build before the changes are propagated on device (or in simulator).

@dnkoutso dnkoutso removed the s6:need sample label Sep 4, 2018

@dnkoutso

This comment has been minimized.

Copy link
Contributor

dnkoutso commented Sep 4, 2018

Thanks for sample will take a look when time permits.

@dnkoutso

This comment has been minimized.

Copy link
Contributor

dnkoutso commented Sep 4, 2018

Confirmed.

@dnkoutso

This comment has been minimized.

Copy link
Contributor

dnkoutso commented Sep 4, 2018

Quick debugging shows its a new build system issue only and despite the file timestamp changing Xcode does not treat the resulting framework as "dirty" and therefore skips the script phase...

Will try to see if I can work around it but unfortunately this might need to be a radar instead.

@dnkoutso

This comment has been minimized.

Copy link
Contributor

dnkoutso commented Sep 4, 2018

This is an unfortunate regression in Xcode 10 new build system. Switching to legacy system makes it work just fine.

I believe Apple is aware of this. Switching the input/output paths to point to the actual executable inside the framework makes the problem go away but we may not want to do that. This means that the script phases will re-run correctly as long as the sources of the framework change but it won't catch resource changes or other things inside the .framework folder.

For now it seems that Xcode 10 build system will always require a clean build to ensure latest changes are present which sucks.

We could maybe add an option to not use input/output paths which would "fix" this but it is ultimately a workaround.

@mkubista

This comment has been minimized.

Copy link
Author

mkubista commented Sep 5, 2018

I will try to report it in Apple Bug reporter. Will see, what they will say...

@xaphod xaphod referenced this issue Sep 13, 2018

Closed

XCode 10 Caching Pod builds (New build system) #7966

1 of 1 task complete
@maxkattner

This comment has been minimized.

Copy link

maxkattner commented Sep 13, 2018

I created a radar (41126633) for that in June. They are aware, got a sample project and are working on fixing it. Unfortunately, I do not have any more information.
Feel free to created dupes as it helps them in prioritising!

@dnkoutso

This comment has been minimized.

Copy link
Contributor

dnkoutso commented Sep 13, 2018

Thank you!!

@dnkoutso dnkoutso removed the t2:defect label Sep 13, 2018

jverkoey added a commit to jverkoey/material-components-ios that referenced this issue Sep 18, 2018

[Xcode10] Enable the legacy build system.
The new Xcode 10 build system does not play well with CocoaPods, resulting in iterative builds not picking up changes in source files. This results in a remarkably frustrating debugging experience because breakpoints will often not be associated with the lines you expect them to be, and functionality that you're testing won't actually be testable until you perform a clean build.

See CocoaPods/CocoaPods#8073 for additional details.
@dmcgloin

This comment has been minimized.

Copy link

dmcgloin commented Sep 18, 2018

FYI: Tested on Xcode 10.1 beta (10O23u) and issue still occurs.

@dnkoutso

This comment has been minimized.

Copy link
Contributor

dnkoutso commented Sep 18, 2018

I think the best thing for us to do is to provide an option not to use input/output paths. This will cause the script phase to run always but at least folks wont have to do clean builds.

@dnkoutso

This comment has been minimized.

Copy link
Contributor

dnkoutso commented Sep 18, 2018

I opened up this PR (#8105) to provide an option to disable usage of input/output paths.

jverkoey added a commit to material-components/material-components-ios that referenced this issue Sep 19, 2018

[Xcode10] Enable the legacy build system. (#5160)
The new Xcode 10 build system does not play well with CocoaPods, resulting in iterative builds not picking up changes in source files. This results in a remarkably frustrating debugging experience because breakpoints will often not be associated with the lines you expect them to be, and functionality that you're testing won't actually be testable until you perform a clean build.

As such, we cannot use the new Xcode 10 build system until this is resolved in upstream Xcode. As of this writing, the Xcode 10 GM version also is known to exhibit this undesirable behavior.

See CocoaPods/CocoaPods#8073 for additional details.

@dnkoutso dnkoutso removed the d2:moderate label Sep 21, 2018

@brunophilipe

This comment has been minimized.

Copy link

brunophilipe commented Sep 24, 2018

Expanding on what @mkubista said, another solution is to add this as a run script phase before the "Embed Pods Frameworks" phase to automatically touch all framework shell scripts:

find "${SRCROOT}/Pods" -type f -name *frameworks.sh -exec bash -c "touch \"{}\"" \;
@dnkoutso

This comment has been minimized.

Copy link
Contributor

dnkoutso commented Sep 24, 2018

This issue has been closed since we now merged #8105 which will ship with beta.2.

For anyone who is taking a look here, #8105 is not fixing the issue it just provides an option to disable input/output paths.

If you do use this option please note that you will get a performance hit on incremental builds since the script phases will constantly re-run but at least your build output should be correct.

There are a few workarounds present here but ultimately we hope Apple fixes the issue.

@dnkoutso

This comment has been minimized.

Copy link
Contributor

dnkoutso commented Oct 8, 2018

@maxkattner do you have an open radar link to the issue you filed do you know the status of your issue if it has been updated or being worked on?

@brunophilipe

This comment has been minimized.

Copy link

brunophilipe commented Oct 8, 2018

@ralfebert

This comment has been minimized.

Copy link

ralfebert commented Oct 15, 2018

Still happening with Xcode 10.1 beta 2.

Removed the in/output files in the 'Embed Pods Frameworks' manually as a workaround, this works for now.

Any hints how to install a cocoapod with the installation option from #8105 included? Do I need to checkout the repo and do 'rake install' or is there a smarter way? Is there a way to enable this globally as a default until the Xcode bug is fixed?

@maxkattner Do you have a link to the example project you attached? I'd like to create a dupe for the issue.

@amorde

This comment has been minimized.

Copy link
Member

amorde commented Oct 15, 2018

@ralfebert I recommend using Bundler to pin CocoaPods to the commit SHA or version that you need for your project

@maxkattner

This comment has been minimized.

Copy link

maxkattner commented Oct 16, 2018

Hey @ralfebert, nice seeing you here! :D
I created a repository to check out the example project and dupe the radar: https://github.com/maxkattner/new-bs-bug-example

@ralfebert

This comment has been minimized.

Copy link

ralfebert commented Oct 16, 2018

What strikes me as odd is that even with the in-/output files present for the "Embed Pods Frameworks" phase, I see the build phase being run in the build log and the files of the frameworks being rsynced to the app bundle (with the framework clearly not being up-to-date when running app the app). Any clues what in particular is causing this?

@jgavris

This comment has been minimized.

Copy link
Contributor

jgavris commented Oct 16, 2018

@ralfebert I've found that very confusing, but it seems Xcode 'replays' the last output even in the incremental / no-op case.

@ralfebert

This comment has been minimized.

Copy link

ralfebert commented Oct 16, 2018

Indeed, good to know! It helps to add a date command to the script to see if it's actually executed.

I had the suspicion that the issue might be that it doesn't pick up changes from files in a folder. I just did a quick experiment in a standalone project and if I use a folder as input file, the run script phase is not run when I change a file in that folder. But for this simple example, I get the same behaviour with the old build system (while in the Cocoapods project switching to the legacy build system clearly fixed the issue).

Any ideas what might make this work in the Cocoapods context but not in a simple project for the old build system?

I think it would help to get this fixed to show the regression in the most straightforward way. Unfortunately I could not find any documentation that specifies that this is supposed to work with a folder as input file.

@grigorye

This comment has been minimized.

Copy link

grigorye commented Oct 22, 2018

@ralfebert

What strikes me as odd is that even with the in-/output files present for the "Embed Pods Frameworks" phase, I see the build phase being run in the build log and the files of the frameworks being rsynced to the app bundle (with the framework clearly not being up-to-date when running app the app). Any clues what in particular is causing this?

Do you mean xcodebuild or the build log at Xcode report tab? If the latter, there's "All" and "Recent" buttons at the top left of build log window - I believe that "Recent" shows just what actually happened in the last build, no?

@dnkoutso

This comment has been minimized.

Copy link
Contributor

dnkoutso commented Oct 22, 2018

Correct. Xcode will output in "All" the full script even though it actually didn't run it. The "Recent" tab will actually not include the script phase if it didn't run.

MarioIannotta added a commit to MarioIannotta/PullUpController that referenced this issue Nov 1, 2018

k0nserv added a commit to Skyscanner/backpack-ios that referenced this issue Jan 22, 2019

[No JIRA] Fix build issue for framework changes
Previously the new Xcode build system caused development pods not to
correctly rebuild when changes were made to the source. This required a
clean build for each change.
See: CocoaPods/CocoaPods#8073

With this chagne we adopt a new CocoaPods option
`disable_input_output_paths: true` which resolves this issue while still
staying on the new build system.

See: CocoaPods/CocoaPods#8105

georgegillams added a commit to Skyscanner/backpack-ios that referenced this issue Jan 22, 2019

[No JIRA] Fix build issue for framework changes
Previously the new Xcode build system caused development pods not to
correctly rebuild when changes were made to the source. This required a
clean build for each change.
See: CocoaPods/CocoaPods#8073

With this chagne we adopt a new CocoaPods option
`disable_input_output_paths: true` which resolves this issue while still
staying on the new build system.

See: CocoaPods/CocoaPods#8105
@ralfebert

This comment has been minimized.

Copy link

ralfebert commented Feb 4, 2019

I documented the issue here:
https://www.ralfebert.de/ios/blog/cocoapods-clean-input-output-files/

I wondered, has it been considered to disable the input/output folders in the build phase by default until the Xcode bug is fixed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment