Copy Pods Resources phase wipes .lproj directories completely, removing localized resources of app target since 1.3.0

[nikolaykasyanov]

Report

What did you do?

Updated to CocoaPods 1.3.0, did pod install, run xcodebuild

What did you expect to happen?

All resources (pods' and target's) are properly copied into app bundle

What happened instead?

Target's resources are wiped during Copy Pods Resources phase, here's the excerpt from the build log:

PhaseScriptExecution [CP]\ Copy\ Pods\ Resources /Users/nkasyanov/Library/Developer/Xcode/DerivedData/CocoaPods13Bug-fwgmofrkgchvrufcqskhrpiwysqx/Build/Intermediates/CocoaPods13Bug.build/Debug-iphonesimulator/CocoaPods13Bug.build/Script-8ED8F990C2E1683A2F7CBAF1.sh
    cd /Users/nkasyanov/Desktop/CocoaPods13Bug
    /bin/sh -c /Users/nkasyanov/Library/Developer/Xcode/DerivedData/CocoaPods13Bug-fwgmofrkgchvrufcqskhrpiwysqx/Build/Intermediates/CocoaPods13Bug.build/Debug-iphonesimulator/CocoaPods13Bug.build/Script-8ED8F990C2E1683A2F7CBAF1.sh
/Users/nkasyanov/Desktop/CocoaPods13Bug/Pods/Accengage-iOS-SDK/BMA4SSDK/Resources/Localizables/Base.lproj
/Users/nkasyanov/Desktop/CocoaPods13Bug/Pods/Accengage-iOS-SDK/BMA4SSDK/Resources/Localizables/en.lproj
/Users/nkasyanov/Desktop/CocoaPods13Bug/Pods/Accengage-iOS-SDK/BMA4SSDK/Resources/Localizables/fr.lproj
building file list ... done
deleting Base.lproj/Main.storyboardc/UIViewController-BYZ-38-t0r.nib
deleting Base.lproj/Main.storyboardc/Info.plist
deleting Base.lproj/Main.storyboardc/BYZ-38-t0r-view-8bC-Xf-vdC.nib
deleting Base.lproj/Main.storyboardc/
deleting Base.lproj/LaunchScreen.storyboardc/UIViewController-01J-lp-oVM.nib
deleting Base.lproj/LaunchScreen.storyboardc/Info.plist
deleting Base.lproj/LaunchScreen.storyboardc/01J-lp-oVM-view-Ze5-6b-2t3.nib
deleting Base.lproj/LaunchScreen.storyboardc/
deleting Base.lproj/Localizable.strings
Base.lproj/
Base.lproj/BMA4S.strings
en.lproj/
en.lproj/BMA4S.strings
fr.lproj/
fr.lproj/BMA4S.strings

Please note deleting ... lines.

CocoaPods Environment

Stack

   CocoaPods : 1.3.0
        Ruby : ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-darwin16]
    RubyGems : 2.6.8
        Host : Mac OS X 10.12.6 (16G29)
       Xcode : 8.3.3 (8E3004b)
         Git : git version 2.13.3
Ruby lib dir : /Users/nkasyanov/.rvm/rubies/ruby-2.3.1/lib
Repositories : anodamobi - https://github.com/anodamobi/ANODA-CocoaPods.git @ efe834d1d1d590a6978919b86d6242b9e6922651
               master - https://github.com/CocoaPods/Specs.git @ 14451c67238e007cc9f725eae36df40ee02aeb68
               voltmobi - git@github.com:VoltMobi/CocoaPodsSpecs.git @ cd81270cee87ae81d2078c3632cbdd02c8864c6e

Installation Source

Executable Path: /Users/nkasyanov/.rvm/gems/ruby-2.3.1/bin/pod

Plugins

claide-plugins        : 0.9.2
cocoapods-deintegrate : 1.0.1
cocoapods-packager    : 1.5.0
cocoapods-plugins     : 1.0.0
cocoapods-search      : 1.0.0
cocoapods-stats       : 1.0.0
cocoapods-trunk       : 1.2.0
cocoapods-try         : 1.1.0
slather               : 2.4.2

Podfile

target 'CocoaPods13Bug' do
  pod 'Accengage-iOS-SDK', :podspec => 'Accengage-iOS-SDK.podspec'
end

Project that demonstrates the issue

https://github.com/nikolaykasyanov/CocoaPods13Bug

How to reproduce:

1. Clone
2. Run pod install (make sure it's CocoaPods 1.3.0)
3. Run xcodebuild -scheme CocoaPods13Bug -workspace CocoaPods13Bug.xcworkspace -sdk iphonesimulator build
4. Inspect the build log and resulting app bundle

Notes

1. It happens consistently when using xcodebuild. Doesn't always happen in Xcode.

[nikolaykasyanov]

I believe this PR is the culprit: #6694 (5cd2c97), namely --delete flag.

[dnkoutso]

@nikolaykasyanov thanks for the report and project. I'll need to check your project but why do you expect an *.lproj file in the DerivedData?

[nikolaykasyanov]

@dnkoutso I don't expect anything in derived data, rather in the actual application bundle, but unfortunately it's not there, everything except pod resources is wiped by [CP] Copy Pods Resources phase.

Expected bundle contents:

Actual bundle contents:

Thanks for looking into it!

[nikolaykasyanov]

Here's another weird thing: when I do Build in Xcode, it behaves exactly as command line xcodebuild call (i.e. files are missing), but when I use Run, files suddenly appear.

[dnkoutso]

@nikolaykasyanov if you wipe out your derived data and re-build do you see the files? I did this and I correctly see the same file list as in your first screenshot.

[dnkoutso]

@nikolaykasyanov keep in mind from the Copy Resources script phase its supposed to be copying only the strings. The rest of the resources like LaunchScreen.storyboard come from the app target.

[nikolaykasyanov]

@dnkoutso I understand this, and the problem is that [CP] Copy Pods Resources removes what has been previously copied by Copy Bundle Resources phase (the latter deals only with target resources, while the former is supposed only to copy pod strings a you've mentioned)

Here's my clean build results:

$ rm -rf ~/Library/Developer/Xcode/DerivedData/
$ xcodebuild -scheme CocoaPods13Bug -workspace CocoaPods13Bug.xcworkspace -sdk iphonesimulator build
... log omitted...
** BUILD SUCCEEDED **
$ ls /Users/nkasyanov/Library/Developer/Xcode/DerivedData/CocoaPods13Bug-fwgmofrkgchvrufcqskhrpiwysqx/Build/Products/Debug-iphonesimulator/CocoaPods13Bug.app/Base.lproj/
BMA4S.strings

It also happens on CI (here's how I found this issue in the first place) and on my colleagues's laptop.

[dnkoutso]

Hm I believe you here's my output

ls /Users/dimitris/Library/Developer/Xcode/DerivedData/CocoaPods13Bug-aqehmikltgljodhjpkwjwdyujcrs/Build/Products/Debug-iphonesimulator/CocoaPods13Bug.app/Base.lproj 
BMA4S.strings            
LaunchScreen.storyboardc
Localizable.strings     
Main.storyboardc

[dnkoutso]

@nikolaykasyanov can you manually inspect "./Pods/Target Support Files/Pods-CocoaPods13Bug/Pods-CocoaPods13Bug-resources.sh" and remove the --delete flag, save the file, wipe out DD and try the same command again?

[dnkoutso]

@nikolaykasyanov wait a second I think I got it.

[dnkoutso]

@nikolaykasyanov confirmed. This is a rather large issue. Working on it.

[nikolaykasyanov]

@dnkoutso thank you! If you need any more input or help with testing — just let me know.

[dnkoutso]

/cc @jgavris

[jgavris]

🤔

[dnkoutso]

@nikolaykasyanov this is a bit weird, still investigating. I have two large apps that use tons of resources that do not exhibit this behavior

[jgavris]

Same, I am not seeing any 'deleting' in my copy resources build phase.

[nikolaykasyanov]

@dnkoutso @jgavris do your apps have pods with localized resources, though? Also, I believe this won't happen if dynamic frameworks are used or if pods use resources bundles.

[dnkoutso]

I believe this won't happen if dynamic frameworks are used or if pods use resources bundles.

I believe we are getting closer. I think I found it.

From https://github.com/Accengage/ios-pod-specs/blob/master/Accengage-iOS-SDK/5.3.5/Accengage-iOS-SDK.podspec#L30

You are right its not using a resource bundle. This causes the $RESOURCE_PATH in the script to be the same as the resource path instead of a bundle path therefore, rsync will delete these files.

Investigating a bit more to confirm this and will update with it with a fix.

[dnkoutso]

I am a bit wrong in the original assumption, still looking.

[nikolaykasyanov]

@dnkoutso thanks for the update.

A small correction: podspec I'm using in the sample project is derived from this file, I believe: https://github.com/Accengage/ios-pod-specs/blob/master/Accengage-iOS-SDK/5.5.0/Accengage-iOS-SDK.podspec#L30

UPD: I've updated sample project to use upstream podspec. Same behaviour.

[dnkoutso]

mkdir -p "${TARGET_BUILD_DIR}/${UNLOCALIZED_RESOURCES_FOLDER_PATH}"
rsync --delete -avr --copy-links --no-relative "${RSYNC_PROTECT_TMP_FILES[@]}" --exclude '*/.svn/*' --files-from="$RESOURCES_TO_COPY" / "${TARGET_BUILD_DIR}/${UNLOCALIZED_RESOURCES_FOLDER_PATH}"
if [[ "${ACTION}" == "install" ]] && [[ "${SKIP_INSTALL}" == "NO" ]]; then
  mkdir -p "${INSTALL_DIR}/${UNLOCALIZED_RESOURCES_FOLDER_PATH}"
  rsync --delete -avr --copy-links --no-relative "${RSYNC_PROTECT_TMP_FILES[@]}" --exclude '*/.svn/*' --files-from="$RESOURCES_TO_COPY" / "${INSTALL_DIR}/${UNLOCALIZED_RESOURCES_FOLDER_PATH}"
fi

This is the section in the .sh that causes the failure. @jgavris is --delete needed there? Probably but I dont think we can keep it. We are uncertain what other files could be there by the app and therefore should not take the risk and delete them.

[jgavris]

Agreed, that one is probably overstepping its bounds. The one for copying .framework should stay though.

[dnkoutso]

And the problem is because Base.lproj matches the same destination folder then rsync deletes all these files.

[dnkoutso]

@jgavris are we certain the --delete is needed for .frameworks? what if the app or some other means have copied a framework in the same folder there manually?

[jgavris]

I'm not sure I follow. It seems wrong that an app would install a resource bundle (.framework with only resources) via CocoaPods but also copy stuff into that bundle, I don't think we should encourage that.

[dnkoutso]

Ah, ok I think you are right just wanted to double check.

[dnkoutso]

@nikolaykasyanov thanks for the report, I believe this will be merged and ship with 1.3.1 of CP.

[nikolaykasyanov]

@dnkoutso thanks you.

[dnkoutso]

@nikolaykasyanov 1.3.1 is out. double check it :)

[nikolaykasyanov]

@dnkoutso works like a charm, thanks!