Gems with executables installed from git source failed Errno::EPERM: Operation not permitted #1977

Closed
raldred opened this Issue Jun 12, 2012 · 18 comments

Projects

None yet

10 participants

raldred commented Jun 12, 2012

I've spent a while tracking this down. I'm going to run through the whole process.

I'm currently using capistrano to deploy, there are a few different users on my team, we all use our own user accounts to deloy, shared deploy users are a no go in a PCI environment, this is fine for everything apart from this one case, where a gem that has executables is installed directly from git source.

Bundle install was run by user1 first, installing everything successfully.
user2 now cannot bundle install because of a permissions issue only related to gems installed from git source (maybe sources in general) that have executables.

The following error is presented:

Errno::EPERM: Operation not permitted - /var/www/vhosts/domain.org/shared/bundle/ruby/1.9.1/bundler/gems/stiller-c52c2a785a98/bin/stiller
An error occured while installing stiller (0.0.1), and Bundler cannot continue.
Make sure that `gem install stiller -v '0.0.1'` succeeds before bundling.

The problem appears to lie in Bundler::Source::Path::Installer#generate_bin on line 404
Where the source installer calls super this in turn calls Gem::Installer#generate_bin (rubygems core)

Full trace of the actual error raised in Gem::Installer (rubygems)
http://pastebin.com/DGtd9VqU

Gem::Installer#generate_bin chmods the executables listed in the spec so that they are executable (line 302 /usr/lib/ruby/1.9.1/rubygems/installer.rb:302)
This is where it breaks, only a file's owner can chmod, so because user1 ran bundle install first, user2 cannot because the chmod fails.

Gems installed directly from rubygems.org work with no issues regardless of the user.
Could this be solved with Bundler.requires_sudo? instead of calling super in Bundler::Source::Path::Installer#generate_bin maybe have some custom implementation?

Looking forward to your input.

Owner

Maybe this can be fixed by just not trying to chmod if the file is owned by another user?

On Jun 12, 2012, at 4:05 AM, Rob Aldredreply@reply.github.com wrote:

I've spent a while tracking this down. I'm going to run through the whole process.

I'm currently using capistrano to deploy, there are a few different users on my team, we all use our own user accounts to deloy, shared deploy users are a no go in a PCI environment, this is fine for everything apart from this one case, where a gem that has executables is installed directly from git source.

Bundle install was run by user1 first, installing everything successfully.
user2 now cannot bundle install because what appears to be a big related to gems installed from git source that have executables.

The problem appears to lie in Bundler::Source::Path::Installer#generate_bin on 404
Where the source installer calls super this in turn calls Gem::Installer#generate_bin (rubygems core)

Gem::Installer#generate_bin chmods the executable (line 302 /usr/lib/ruby/1.9.1/rubygems/installer.rb:302)
This is where it breaks, only a file's owner can chmod, so because user1 ran bundle install first, user2 cannot because he/she cannot chmod the executable.

Gems installed directly from rubygems.org work with no issues regardless of the user.
Could this be solved with Bundler.requires_sudo? instead of calling super in Bundler::Source::Path::Installer#generate_bin maybe have some custom implementation?

Looking forward to your input.


Reply to this email directly or view it on GitHub:
#1977

raldred commented Jun 14, 2012

Indeed. However that is buried in rubygems core.
On Jun 13, 2012 10:13 PM, "André Arko" <
reply@reply.github.com>
wrote:

Maybe this can be fixed by just not trying to chmod if the file is owned
by another user?

On Jun 12, 2012, at 4:05 AM, Rob Aldredreply@reply.github.com wrote:

I've spent a while tracking this down. I'm going to run through the
whole process.

I'm currently using capistrano to deploy, there are a few different
users on my team, we all use our own user accounts to deloy, shared deploy
users are a no go in a PCI environment, this is fine for everything apart
from this one case, where a gem that has executables is installed directly
from git source.

Bundle install was run by user1 first, installing everything
successfully.
user2 now cannot bundle install because what appears to be a big
related to gems installed from git source that have executables.

The problem appears to lie in
Bundler::Source::Path::Installer#generate_bin on 404
Where the source installer calls super this in turn calls
Gem::Installer#generate_bin (rubygems core)

Gem::Installer#generate_bin chmods the executable (line 302
/usr/lib/ruby/1.9.1/rubygems/installer.rb:302)
This is where it breaks, only a file's owner can chmod, so because
user1 ran bundle install first, user2 cannot because he/she cannot
chmod the executable.

Gems installed directly from rubygems.org work with no issues
regardless of the user.
Could this be solved with Bundler.requires_sudo? instead of calling
super in Bundler::Source::Path::Installer#generate_bin maybe have some
custom implementation?

Looking forward to your input.


Reply to this email directly or view it on GitHub:
#1977


Reply to this email directly or view it on GitHub:
#1977 (comment)

Same issue over here with a gem installed from a different user.

Contributor

Bundler should be able to assume that it is able to write to its install directory. I can think of all sorts of horrible ways things could break otherwise.

And why are you deploying over the top of another deploy anyway? Deploy to a new directory and symlink.

@xaviershay xaviershay closed this Aug 12, 2013

Bundler should be able to assume that it is able to write to its install directory. I can think of all sorts of horrible ways things could break otherwise.

And why are you deploying over the top of another deploy anyway? Deploy to a new directory and symlink.

@xaviershay Could you explain further please? Are you suggesting not to have a shared bundler directory?

Owner

Wait... if the code that fails is buried inside rubygems, why does this fail only with gems from git repos? I don't get it.

Contributor

(speaking as an outsider with no knowledge of this implementation)

You need to pick your invariant:

  1. Files in vendor/cache should be immutable once written to, in which case current bundle behaviour is wrong. (I have no idea is this actually feasible.)
  2. Bundler needs to be able to change files in vendor/cache.

Trying to write code for some invariant in the middle ("some files are editable") is asking for trouble. Conditionals edge cases, bugs like this one.

Given that option 1 doesn't work at the moment, it's best to assume that option 2 is the right one (maybe @indirect can clarify). In that case, you either need to set up your group permissions so each user can write over files created by another user, or do shared nothing deploys.

In my first comment, I forgot that people use a shared bundler cache as a deployment optimisation, so sorry for the bluntness. I feel this is a bad practice in theory, but understand there aren't really readily accessible alternatives so understand its use. The approach I prefer is to have a deployable "artifact", which contains everything necessary for the deployment. This should be computed ahead of time so that release is "download artifact and untar" and activate is "switch symlink".

drewish commented Oct 9, 2013

I'm also curious why this got closed. We keep rediscovering it ever four months or so when we try adding a gem from a forked repo.

drewish commented Oct 9, 2013

@xaviershay why'd you close this issue? you'd marked #2078 as a duplicate of this and closed it too.

Ran into this today as well. We are not deploying on top of another deploy, it is just the shared bundle cache that is causing troubles.

timocp commented Dec 18, 2013

Have just been hit by this when switching a gem to git sources.

Re: "Bundler should be able to assume that it is able to write to its install directory. I can think of all sorts of horrible ways things could break otherwise."

In my case, everything is group writable so this assertion is true. The error occurs because it tries to chmod a file in the bin directory of a gem installed in a previous deployment by a different user. I haven't looked enough into this to understand why it is git sources only.

Owner

So you have a setup where the file is group writable, but not group chmod +xable? How is that possible?

On a more abstract level, if you feel this is a bug in Bundler, please open a new ticket with the information from ISSUES and reproduction steps. If you feel that this is a feature that should be added to Bundler, please open a ticket on the bundler-features repo so that it can be discussed there.

Thanks.

Yes, I have strong feelings this is a bug.

The steps are easy for me to reproduce in a Rails app.

  • Add a gem with executables to Gemfile by git url reference.
  • Deploy the app using capistrano.
  • Let my coworker deploy the same app.

Result: Errno::EPERM: Operation not permitted.

Owner
indirect commented Jan 3, 2014

@mlangenberg that's not enough for me to reproduce. I don't have a deploy server with multiple users.

I'd like to support installing git gems the same way regular gems are installed, but I don't understand the mechanism by which Rubygems raises an exception for git gems but not for regular gems.

@indirect indirect reopened this Jan 3, 2014
Owner
indirect commented Jan 9, 2014

Bundler now sets permissions on the binstubs it creates for git binaries to 0777 & ~File.umask. Please open a new ticket if you're seeing issues after trying either the master branch or one of the (out soon) 1.6 releases or prereleases.

@indirect indirect closed this Jan 9, 2014

Thx, I'll let you know if I still see the issue.

edestecd commented Jun 4, 2014

Here is a workaround for capistrano...

This cleans out the directories that have the git source/executables and forces a new install of only
the git sources as the currently deploying user.

Luckily bundler keeps a cache around for the git sources, so it does not have to refetch them from the remote repo. It is still reasonably fast and better than removing/reinstalling all gems on every deploy...

before "bundle:install", "bundle:wipe_git_sources"
namespace :bundle do
  desc <<-DESC
    Hack that forces us to reinstall all gems from git on each deploy
    https://github.com/bundler/bundler/issues/1977
  DESC
  task :wipe_git_sources, :except => { :no_release => true } do
    run "rm -rf #{File.join(shared_path, 'bundle', 'ruby', '*', 'bundler', 'gems', '*')}"
  end
end
johnf commented Aug 30, 2014

A pull request I just made against rubygems should help with this issue.

rubygems/rubygems#1002

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment