bower install is stalling/hanging completely

[knalli]

I'm issued with the situation that bower install hangs sometimes without any further message. The issue is new since Monday (on Friday is was still fine).

Because there are several dependencies, I am still investigating whether it is only a connection issue (internet uplink). I have got also read about #2235, but at the moment I don't see this could be the issue. I'm looking for more logging.

Details about the environment:

• RHEL6 (everything running within Jenkins)
• git 1.7.1
• npm 4.3.1 (configured by nvm)
• bower ^1.7.9 installed locally

But I have seen the issue also on my local machine: OS X 10.11.4, npm 4.3.1, bower ^1.7.9

So far, the symptoms are:

• bower install hangs after some time. Interestingly, it is often a the same position by job. Just like a specific threshold would be reached every time.

bower angular#~1.2.x                                                               progress Receiving objects:  94% (20428/21650), 160.50 MiB | 2.39 MiB/s
bower angular#~1.2.x                                                               progress Receiving objects:  95% (20620/21650), 162.84 MiB | 2.34 MiB/s
bower angular#~1.2.x                                                               progress Receiving objects:  96% (20835/21650), 165.30 MiB | 2.38 MiB/s
bower angular#~1.2.x                                                               progress Receiving objects:  97% (21041/21650), 167.90 MiB | 2.41 MiB/s
bower angular#~1.2.x                                                               progress Receiving objects:  99% (21434/21650), 172.39 MiB | 2.63 MiB/s
bower angular#~1.2.x                                                               progress Resolving deltas:  12% (1685/14008)
bower angular#~1.2.x                                                               progress Resolving deltas:  35% (4921/14008)
bower angular#~1.2.x                                                               progress Resolving deltas:  60% (8518/14008)
bower angular#~1.2.x                                                               progress Resolving deltas:  81% (11373/14008)
bower angular#~1.2.x                                                                old-git It seems you are using an old version of git, it will be slower and propitious to errors!
bower angular#~1.2.x                                                               resolved https://github.com/angular/bower-angular.git#1.2.29

# HANG! For hours..

• I also can see following processes

$ ps aux | grep git

jenkins  23303  0.0  0.0   9776   932 ?        S    15:57   0:00 git ls-remote --tags --heads https://github.com/monospaced/angular-elastic.git
jenkins  23304  0.0  0.0  93156  5612 ?        S    15:57   0:00 git-remote-https https://github.com/monospaced/angular-elastic.git https://github.com/monospaced/angular-elastic.git
jenkins  23503  0.0  0.0   9780  1132 ?        S    15:58   0:00 git clone https://github.com/angular/bower-angular.git -b v1.4.7 --progress . --depth 1
jenkins  23504  0.0  0.0  93160  5612 ?        S    15:58   0:00 git-remote-https https://github.com/angular/bower-angular.git https://github.com/angular/bower-angular.git

All these processes will stay. I cannot see any actions anymore. Another job, or another time: Other repositories, but same type of commands.

• If one build job is broken, the next one will be probably also.
• The issue is present on RHEL6, but I could get it on my local OS X, too.
• I've verified the issue without Jenkins being evolved. Actually ridiculous, but I want to be sure not overseen something.
• *If I manually kill the specific stalling processes (git-remote-https), bower install will finish without any notice and error. But the result are partially incomplete checkouts (actually, no surprise because I've killed something).

I've already moved the default storage paths to ensure concurrent builds of different projects will not conflict each other:

{
  "storage" : {
    "packages" : ".bower-cache",
    "registry" : ".bower-registry"
  },
  "tmp" : ".bower-tmp"
}

The local workspaces will be destroyed before each build, so there is nothing left of any prior build. Due the specified storage path settings, there is no "global" cache active. I have verified this: ~/.cache/bower is not being present.

Obviously, you will not get up with a solution right now.. (but still hoping :)). And it is not clear it is an actual issue for bower. But it would help me a lot, if there is any possibility to increase the logging level specifically for this dependency clone/install issue. bower install --verbose does not change anything. :(

[jj4th]

Might not be the solution, but this occurs if one of your bower packages is installed from a private github repository. It's not actually hung, it's just waiting for you to enter the username and password.

[knalli]

Is it possible to log that? The point is: everything is public, no private repositories are required.

[jj4th]

@knalli I don't work on the project, I was actually checking issues because of a weird problem I was having with bower cache (something like 10-12 packages today ended up cached under the wrong versions on my build box), and I happened to see this issue. The symptoms you described sounded EXACTLY like the symptoms we encountered with the private github.com repos and a username/password prompt. One of the packages would prompt for a username/password, but bower would keep on installing other packages, essentially "hiding" the prompt from the log. If you scroll through the log, you might find a username: somewhere at the beginning or end of another line. At least, if this is the issue. If not, I don't know what it might be 😦

[steven-prybylynskyi]

@knalli we have similar issue and it hangs randomly, it can be a repo of angular, jquery, etc. We have noticed that it is mostly a connection problem with git. Speed goes as low as 30-50 Kbit/sec.

Currently we are moving packages to npm because it works better with the locally cached packages, doesn't perform excessive validations (and that means no additional requests to git servers).

[ZeusF1]

@knalli
Here is pretty same issue.

We have VERY long time for get angular, jquery, from bower
Reproduced on different server in different locations,

Issue repeatable for now on:

• Centos 6,7, Ubuntu 15
• Git 1.7.1, and latest master (2.8.0)
• npm 1.3.6

[mitrae]

git config --global url."https://".insteadOf git:// didn't help

[knalli]

@mitrae Well, #2235 is about that this is default now.

[mitrae]

@knalli yep. read it already.

on Ubuntu machine we got following: "gnutls_handshake() failed: Error in the pull function."

[bnjjj]

Same bug with :
GIT 1.7.10.4
Bower 1.7.9
npm 2.11.3

Need help :)

[mitrae]

@sheerun any ideas?

[knalli]

So, I think I have additional information. The freeze is internally a timeout while polling a file descriptor (process/socket maybe?).

I've traced the bower install a little bit. The following output is the result of strace -f -p $bPid where $bPid is the process id. In this specific case, the corresponding process is

jenkins  26159  0.0  0.0  94528  5420 pts/0    S+   22:41   0:00 git-remote-https https://github.com/PascalPrecht/bower-angular-translate-loader-static-files.git https://github.com/PascalPrecht/bower-angular-translate-loader-static-files.git

Fortunately, I'm co-maintaining this particular repository (bower-angular-translate) and I know it is not "large". So the repository size is definitely not a cause.

[pid 21482] stat("/fullpath/.bower-cache/060a9fe0e60a0d3d6c9ed350cde03e61", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 21482] write(8, "\1\0\0\0\0\0\0\0", 8) = 8
[pid 21474] <... epoll_wait resumed> {{EPOLLIN, {u32=8, u64=8}}}, 1024, -1) = 1
[pid 21482] futex(0x1989d24, FUTEX_WAIT_PRIVATE, 30072, NULL <unfinished ...>
[pid 21474] read(8, "\1\0\0\0\0\0\0\0", 1024) = 8
[pid 21474] futex(0x1989d24, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x1989d20, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1} <unfinished ...>
[pid 21481] <... futex resumed> )       = 0
[pid 21474] <... futex resumed> )       = 1
[pid 21481] futex(0x1989ce0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>
[pid 21474] futex(0x1989ce0, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
[pid 21481] <... futex resumed> )       = -1 EAGAIN (Resource temporarily unavailable)
[pid 21474] <... futex resumed> )       = 0
[pid 21481] futex(0x1989ce0, FUTEX_WAKE_PRIVATE, 1) = 0
[pid 21481] rename("/fullpath/.bower-tmp/d18b8624a0f5f721da7b82365fc562dd-21474-DeZbmP", ".bower-cache/060a9fe0e60a0d3d6c9ed350cde03e61/1.2.29") = 0
[pid 21474] epoll_wait(5,  <unfinished ...>
[pid 21481] write(8, "\1\0\0\0\0\0\0\0", 8) = 8
[pid 21474] <... epoll_wait resumed> {{EPOLLIN, {u32=8, u64=8}}}, 1024, -1) = 1
[pid 21481] futex(0x1989d24, FUTEX_WAIT_PRIVATE, 30074, NULL <unfinished ...>
[pid 21474] read(8, "\1\0\0\0\0\0\0\0", 1024) = 8
[pid 21474] unlink(".bower-cache/060a9fe0e60a0d3d6c9ed350cde03e61-1.2.29.lock") = 0
[pid 21474] futex(0x1989d24, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x1989d20, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1
[pid 21480] <... futex resumed> )       = 0
[pid 21480] futex(0x1989ce0, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
[pid 21474] epoll_wait(5,  <unfinished ...>
[pid 21480] <... futex resumed> )       = 0
[pid 21480] lstat("/fullpath/.bower-tmp/d18b8624a0f5f721da7b82365fc562dd-21474-DeZbmP", 0x7fae0e38bd90) = -1 ENOENT (No such file or directory)
[pid 21480] write(8, "\1\0\0\0\0\0\0\0", 8) = 8
[pid 21474] <... epoll_wait resumed> {{EPOLLIN, {u32=8, u64=8}}}, 1024, -1) = 1
[pid 21480] futex(0x1989d24, FUTEX_WAIT_PRIVATE, 30076, NULL <unfinished ...>
[pid 21474] read(8, "\1\0\0\0\0\0\0\0", 1024) = 8
[pid 21474] futex(0x1989d24, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x1989d20, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1} <unfinished ...>
[pid 21479] <... futex resumed> )       = 0
[pid 21474] <... futex resumed> )       = 1
[pid 21479] futex(0x1989ce0, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>
[pid 21474] futex(0x1989ce0, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
[pid 21479] <... futex resumed> )       = -1 EAGAIN (Resource temporarily unavailable)
[pid 21474] <... futex resumed> )       = 0
[pid 21479] futex(0x1989ce0, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
[pid 21474] epoll_wait(5,  <unfinished ...>
[pid 21479] <... futex resumed> )       = 0
[pid 21479] lstat("/fullpath/.bower-tmp/d18b8624a0f5f721da7b82365fc562dd-21474-DeZbmP", 0x7fae0ed8cd90) = -1 ENOENT (No such file or directory)
[pid 21479] write(8, "\1\0\0\0\0\0\0\0", 8) = 8
[pid 21474] <... epoll_wait resumed> {{EPOLLIN, {u32=8, u64=8}}}, 1024, -1) = 1
[pid 21479] futex(0x1989d24, FUTEX_WAIT_PRIVATE, 30078, NULL <unfinished ...>
[pid 21474] read(8, "\1\0\0\0\0\0\0\0", 1024) = 8
[pid 21474] write(9, "bower \33[32mangular#~1.2.x\33[39m  "..., 164) = 164
[pid 21474] epoll_wait(5, {}, 1024, 0)  = 0
[pid 21474] epoll_wait(5,  <unfinished ...>
[pid 26159] <... poll resumed> )        = 0 (Timeout)
[pid 26159] poll([{fd=6, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
[pid 26159] poll([{fd=6, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 1000) = 0 (Timeout)
[pid 26159] poll([{fd=6, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)

# AND SO ON.. only timeout polling lines

And the log of the install itself:

bower angular#~1.2.x                                                               progress Resolving deltas:  90% (12670/14033)
bower angular#~1.2.x                                                               progress Resolving deltas: 100% (14033/14033)
bower angular#~1.2.x                                                               progress Resolving deltas: 100% (14033/14033), done.
bower angular#~1.2.x                                                                old-git It seems you are using an old version of git, it will be slower and propitious to errors!
bower angular#~1.2.x                                                               resolved https://github.com/angular/bower-angular.git#1.2.29

# HANG

[forgulencia]

I can confirm that this issue (what knalli described ) is affecting all of our local vagrant dev environments regardless of the host is Windows or Ubuntu. The guest os is Ubuntu 14.04.4.

Bower version: 1.7.9

poll([{fd=4, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout) select(5, [4], [], [], {0, 50000}) = 0 (Timeout)
I can easily run the command what bower is stuck into on the same machine, meanwhile the above timeouts happen.

[knalli]

We have determined the situation a little bit. Perhaps this is not an issue with the bower lib itself but the decision switching to https endpoints. There are some evidences people have issues with https+git+cloning.

A fast overview in a tcpdump shows hard resets of the tcp stream (when fetching a clone/tar.gz). I have already patched the library locally removing --depth 1 (also a candidate in combination with an older git and clone and https) but this does not change anything.

I'd also noticed such hangs of git processes when configuring private-bower (which uses a similar approach).

Is there a possibility to change the setting locally like git:// instead of https:// ?

[knalli]

I have upgraded the local Git 1.7 with a much fresher Git 2.8.1 (in my case for RHEL6, so look for git-2.8.1-1.el6.x86_64.rpm or similiar). Nope, that does not change anything. So it does look like we can exclude the factor "Git version".

Meanwhile, I have installed a local bower cache proxy. Finally, because this was already the plan for months; I've chosen Nexus3 (because we already use Nexus2 for Maven and NPM). It is running for some hours and until now it looks fine. At the end this is much better than fetching the stuff always and it also reduces the dependency on a full working github.com connection.

However, the issue itself is not resolved.

[mitrae]

Moved from bower to nmp. Solved.

[sheerun]

I'll be addressing this issue soon. Probably by downloading .tar.gz files for https:// instead only for git://

[sheerun]

All it took is 30 minutes to prepare a patch people, please contribute

[matt-bailey]

Just wondered if this has been fixed? I've recently started experiencing the 'hanging' problem in my VM when it tries to download jQuery. It doesn't actually get to the stage of creating the bower_components directory, it just hangs at this point:

// Other components successfully downloaded...
bower jquery#>=1.8.0    resolved https://github.com/jquery/jquery-dist.git#3.1.0

// Then nothing, and I've left it like this for 30 mins or more just to be sure...

Here are my specs:

• Vagrant/Virtual Box
• Ubuntu Trusty
• Git 2.9.0
• NodeJS 6.3.0
• NPM 3.10.5
• Bower 1.7.9

So far I've tried:

• npm cache clean
• Updating Git, NodeJS and NPM (to the versions above)
• Setting Git to use https:// rather than git://

FYI node packages install without any issues.

I'm at a bit of a loss now, do you have any other suggestions?

[preston]

I'm also having this issue with bower v1.7.9 when it tries to install lodash. Any solutions yet?

[holuspokus]

@preston Try bower cache clean on same position you did bower install and try again bower install
@matt-bailey I think it takes to long. You can watch the output of private-bower or see to the cache directory. if there is the jquery folder, but nothing in except the .git folder try it manually:
git clone git://github.com/jquery/jquery.git jquery in your cache directory (defined in bowerConfig.json)

[MWalid]

Any update with this? having the same with bower 1.7.9

[preston]

No luck with cache cleaning. Still not sure what the problem is. 😦

[krish-ram]

Facing this issue with bower 1.7.9 node 4.4.5 git 1.7.1.
Tried cleaning bower cache and npm cache.
Any solution ?

[stephanie-dm]

We had the same problem on a Windows-environment. The solution for us was to remove the environment variable "GIT_SSH" which was set to "C:\Program Files (x86)\Putty\plink.exe". Apparently it didn't work to fetch Bower dependencies from Git through Putty-SSH.

[kblend]

i had the same setup as @knalli in his first post.
For me i figured out it was the version of the git client in rhel6 1.7.x
Just compiled 2.8.1 and the stuff is running smoothly.

For compiling instructions follow this link
http://tecadmin.net/install-git-2-x-on-centos-rhel-and-fedora/