Use of flock can cause problems on some NFS shares #806

Closed
dark-panda opened this Issue Feb 3, 2014 · 6 comments

Comments

Projects
None yet
4 participants

I've recently had a problem with installing gems to home directories that have been mounted over NFS on a cluster of linux servers. It seems that flock does not work over NFS in some situations, namely on older kernels. The recent calls to flock added in the more recent versions of rubygems is thus causing gem installs to hang on some systems where NFS is being used.

Here's my situation and setup:

  • group of linux servers running CentOS 5, Ruby 2.1.0, linux kernel 2.6.18 with home directories mounted off of a central NFS server.
  • the calls to flock in remote_fetcher.rb and in rubygems.rb hang when used against the file systems mounted via NFS.

According to http://nfs.sourceforge.net/#faq_d10 , some support for NFS was added to flock in kernel 2.6.12, although I'm having no luck with the RHEL version 2.6.18 that CentOS 5 ships with.

It might be worth looking into creating separate lock files rather than trying to lock the file itself with flock. This would involve creating and detecting the existence of a separate file and not relying on flock at all.

drbrain added this to the Future milestone Apr 2, 2014

Owner

drbrain commented Apr 2, 2014

It seems that fcntl can be used instead. Unfortunately I don't have an environment to test this in. Can you create a pull request that prefers fcntl if it is available?

Member

luislavena commented Apr 3, 2014

fcntl might not be portable.

How about a separate, exclusive lock file instead?

Sorry for top posting. Sent from mobile.
On Apr 2, 2014 7:36 PM, "Eric Hodel" notifications@github.com wrote:

It seems that fcntl can be used instead. Unfortunately I don't have an
environment to test this in. Can you create a pull request that prefers
fcntl if it is available?

Reply to this email directly or view it on GitHubhttps://github.com/rubygems/rubygems/issues/806#issuecomment-39392748
.

Yeah I looked into using fcntl but I thought that the flock struct varies from platform to platform, so figured a temporary lock file would be the most portable.

Owner

drbrain commented Apr 3, 2014

I was thinking fcntl for Linux (because NFS) and flock everywhere else (because maximum portability). For Linux is seems you need hard links to make separate lock files work on NFS, so the work would be greater but equally unportable.

Contributor

djberg96 commented Aug 26, 2015

See #1327

Contributor

djberg96 commented Oct 5, 2015

Closing, best we can do.

djberg96 closed this Oct 5, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment