CocoaPods hangs on unpacking boost 1.59 when executing pod install

[issue2k]

Hello,

when trying to integrate boost library into one of our projects the pod installation process hangs on extracting the downloaded file, nothing happens after the last line:

 $ /usr/bin/tar xfz /var/folders/gc/6x2pdd557zsg_p2lcrhvq8p80000gn/T/d20160127-36204-18s54gb/file.tgz -C /var/folders/gc/6x2pdd557zsg_p2lcrhvq8p80000gn/T/d20160127-36204-18s54gb

pod install --verbose

Installing boost (1.59.0)
 > Http download
   $ /usr/bin/curl -f -L -o /var/folders/gc/6x2pdd557zsg_p2lcrhvq8p80000gn/T/d20160127-36204-18s54gb/file.tgz http://sourceforge.net/projects/boost/files/boost/1.59.0/boost_1_59_0.tar.gz
   --create-dirs
     % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                    Dload  Upload   Total   Spent    Left  Speed
     0   353    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
     0   429    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
     0   343    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
   100 79.8M  100 79.8M    0     0   521k      0  0:02:36  0:02:36 --:--:--  669k
   $ /usr/bin/tar xfz /var/folders/gc/6x2pdd557zsg_p2lcrhvq8p80000gn/T/d20160127-36204-18s54gb/file.tgz -C /var/folders/gc/6x2pdd557zsg_p2lcrhvq8p80000gn/T/d20160127-36204-18s54gb

Any suggestions?

[issue2k]

ok after waiting for around 10-15 minutes it continued. Maybe it makes sense to speed that up somehow? When i execute the same extraction command on another shell it is done within seconds.

[orta]

We're running the commands you're seeing in the log, you're welcome to dig into cocoapods-downloader to see if there's anything that could cause it to go slower?

[dnkoutso]

FWIW running:

curl -f -L -o /var/folders/gc/6x2pdd557zsg_p2lcrhvq8p80000gn/T/d20160127-36204-18s54gb/file.tgz http://sourceforge.net/projects/boost/files/boost/1.59.0/boost_1_59_0.tar.gz

Is slow just by itself.

I dont think its CocoaPods related...

[issue2k]

@orta @dnkoutso please read my post again. Its not related to the download, the speed of the curl of course depends on server and line speed... What takes time is unpacking the tar, but only when using CocoaPods. If i do it manually its unpacked in a couple of seconds.

[segiddins]

Either way, if this is an issue it belongs to cocoapods-downloader, not CocoaPods proper.

[issue2k]

@segiddins ok thanks i will create an issue there

[madhavajay]

This exact thing just happened to me, same version of boost. It took like... 5 minutes or so? Really long to extract. The download in verbose was 16 seconds. When I extracted it myself using the same command it ran, it was a few seconds. Something weird definitely going on.

I even did a full update of everything ruby and cocoapods related, but it didn't help.

[DikeyKing]

Met the same problem ...
Reboot mac and pod update again solve this problem (wait for about 5 minutes after finished downloading)

[ohtangza]

I am also experiencing this...

[philolo1]

@issue2k can you link your newly created issue here?

[issue2k]

@philolo1 i never did that, no longer using cocoapods, feel free to proceed

[sergeyzhukov]

same here, cocoapods 1.3.1

[a-r-d]

Yes if you follow the instructions on the React native getting started guide for integrating with an existing app, you will be able to reproduce this problem.

I opened this issue before I saw this one, I never thought of waiting 15 minutes.

[shaikatz]

I did a test and disabling cocoapods cache solves that issue.
digging around I've saw that it extracts boost to the cache directory ~500MB of data and then it has another copy to do into the target directory, all that copies takes a loot of time and make it look like it hangs.

You can disable it by creating a ~/.cocoapods/config.yaml file with the following contents.

skip_download_cache: true

[chrismcleod]

same thing happened to me but turned out to be bc my internet dropped. probably just me, but who knows

[lakhman]

checkout my answers here:

• CocoaPods hangs on unpacking boost 1.59 when executing pod install cocoapods-downloader#69 (comment)
• Pod install fails after ejecting to ExpoKit expo/create-react-native-app#436 (comment)

[Pranit-Harekar]

Same cocoapods 1.3.1
@lakhman How would you do that on travis ? The only way I can think of is writing a sed substitute script to replace the download url. Is there any better way of changing that url ?

[lakhman]

I think it should be done via the repository somewhere.
Maybe: https://github.com/CocoaPods/Specs?

I think the url has changed, but the repository hasn't been updated, or maybe I had a version mismatch locally.

# Notice the '9/9/d/' - my version didn't have that
# but I had an older version that threw a 404.
# repos/master/Specs/9/9/d/boost/1.59.0/boost.podspec.json

I don't know how to do this on travis, this is just a "quick fix" - hopefully someone can figure out a permanent solution.

[itinance]

The issue is caused by an invalid tar.gz-file for boost that only contains plain text with html-code of SourceForge:

/usr/bin/curl -f -L -o /private/tmp/ZizzleApp/ios/Pods/boost/file.tgz http://sourceforge.net/projects/boost/files/boost/1.59.0/boost_1_59_0.tar.gz --create-dirs --netrc-optional

less /private/tmp/ZizzleApp/ios/Pods/boost/file.tgz

the less command will list something like:

<!doctype html>
<!-- Server: sfs-consume-5 -->

<!--[if lt IE 7 ]> <html lang="en" class="no-js ie6"> <![endif]-->
<!--[if IE 7 ]>    <html lang="en" class="no-js ie7"> <![endif]-->
<!--[if IE 8 ]>    <html lang="en" class="no-js ie8"> <![endif]-->
<!--[if IE 9 ]>    <html lang="en" class="no-js ie9"> <![endif]-->
<!--[if (gt IE 9)|!(IE)]>--> <html lang="en" class="no-js"> <!--<![endif]-->
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /><script type="text/javascript">(window.NREU

[itinance]

Apparently the people behind boost aren't interested in fixing their issue. Is it possible to install boost from another location where i can fix it by myself?

[orta]

Could you PR this to boost? CocoaPods/Specs#14361

[Pranit-Harekar]

** This is only for Travis builds.
If you are facing this issue on Travis like I mentioned here, you can wait for boost tarfile to unpack using travis_wait. It takes almost 20-25 minutes to unpack for the first time, but if you have pod cache enabled like this

cache: 
    - cocoapods

then it will take few seconds from second time onwards. If you don't do travis_wait then build will timeout after 10 minutes causing it to fail.

[vjpr]

Use @lakhman's instructions but use url: https://github.com/boostorg/boost/archive/boost-1.59.0.tar.gz

sourceforge didn't work for me.

---

I manually downloaded it from sourceforge then did:

~/.cocoapods/repos/master/Specs/9/9/d/boost/1.59.0/boost.podspec.json

  "source": {
    "http": "file:///Users/Vaughan/Downloads/boost_1_59_0.tar.gz"
  },

But now this hangs when its untarring.

---

Using @shaikatz tip, this speeds up the extract a lot.

Notice it extracts to your project instead of the cache (pod install --verbose):

/usr/bin/tar xfz /dev/app/ios/Pods/boost/file.tgz -C /dev/app/ios/Pods/boost

~/.cocoapods/config.yml

skip_download_cache: true

---

Now this step is really slow (took ~5mins):

-> Using yoga (0.52.0.React)
  - Running pre install hooks

[pisacode]

@vjpr I have waited more than 20 min for the yoga running pre install hooks and it did not finish. I had to cancel pod install

[khiettran]

@pisacode I got the same issue, then I updated RN to 0.55v, it's resolved but I don't prefer this workaround.

[zymxxxs]

Boost's podspec has many regular matching paths, and pods copy to cache operation will take a long time.

[FrederickEngelhardt]

Trying not to necro this topic but to speed up the install, you will need to multithread tar extraction.

On mac by default boost is extracted by tar on a single thread. As a workaround, in bash you can do something like this.

Solution

1. Install pbzip2 which unpacks tar in parallel (multiple threads / cores) brew install pbzip2
2. Add something like this script to your shell environment. Example spot if you use zsh would be the $HOME/.zshrc file.

tar(){  
  if [ $(command -v ) ]; then
    command pbzip2 -dkv file.tbz | tar $1 $2.out $3 $4 ${5:@}
    else
    command tar $@
  fi
}

Noting that this script will only override the tar method if pbzip2 exists. But it may break other tar extraction scripts. Consider adding direnv and adding a .envrc file with this script in your repo.

Side note

• It gets even worse when in a corporate environment and EVERY file is scanned by microsoft endpoint. Having the extraction capped at 1 thread will bottleneck the entire process. (think 90 minutes and it may even timeout...)
• Make sure to exclude tar extraction for boost if you can with whatever security technology is being used.