Skip to content
This repository

Wiki page explaining how to use rbenv in production .. #101

Closed
ghost opened this Issue September 23, 2011 · 57 comments
Deleted user

The content you are editing has changed. Reload the page and try again.

Perhaps using these details .. #98 (comment)

Sending Request…

Attach images by dragging & dropping or selecting them. Octocat-spinner-32 Uploading your images… Unfortunately, we don't support that file type. Try again with a PNG, GIF, or JPG. Yowza, that's a big file. Try again with an image file smaller than 10MB. This browser doesn't support image attachments. We recommend updating to the latest Internet Explorer, Google Chrome, or Firefox. Something went really wrong, and we can't process that image. Try again.

Sam Stephenson
Owner

We're using rbenv in production at 37signals. There's not much to it… what specifically would you like to know?

All apps are deployed as the app user; RBENV_ROOT is ~app/.rbenv. The apps run under Unicorn.

/cc @jeremy

Jeremy Kemper
Collaborator

We have packages built for each version of Ruby we deploy. They're installed (as root) to paths in /opt and symlinked to into ~app/.rbenv/versions.

Each app is isolated through disciplined use of RubyGems/Bundler-style binstubs. We start with the stubs Bundler generates (bundle install --binstubs) and change the shebang from #!/usr/bin/env ruby to #!/usr/bin/env ruby-local-exec. Update your Capistrano recipes, Chef cookbooks, Bluepill config, etc to use these binstubs throughout.

This draws a solid line between your app and your ops, meeting at just a handful of binstubs.

Now executing /u/apps/basecamp/current/bin/unicorn uses the Ruby in /u/apps/basecamp/current/.rbenv-version and the Unicorn bundled in /u/apps/basecamp/current/Gemfile.lock. Your app is self-contained. Your ops tools no longer know or care whether you use Bundler or whether you're on 1.8.7 or 1.9.3. Achievement unlocked.

Now, say Ruby 1.9.3-rc1 just came out and you want to upgrade your app. Push the new package to your servers (using Chef or whatever), change .rbenv-version from 1.9.3-preview1 to 1.9.3-rc1, and cap deploy. That's it.

Didn't work out? cap deploy:rollback and you're back at 1.9.3-preview1. Ahh.

Gosha Arinich

@jeremy Cool description. Thanks a lot!

Maybe sstephenson/rbenv should have wiki, @sstephenson?

James Miller

@jeremy Does bundler support changing the shebang in the binstubs out of the box? I see bundler respects RbConfig::CONFIG['ruby_install_name'] -- do you change that?

Jeremy Kemper
Collaborator

@bensie we edit the stubs by hand now. No config tricks. We've been meaning to get a --shebang option going for bundle install --binstubs, though.

(It is kinda nice having a slim little ./bin: just bundle, cap, rake, and unicorn rather than the boatload of never-used executable cruft from all our gem dependencies.)

Deleted user

The content you are editing has changed. Reload the page and try again.

@jeremy That's interesting. Thanks.

Sending Request…

Attach images by dragging & dropping or selecting them. Octocat-spinner-32 Uploading your images… Unfortunately, we don't support that file type. Try again with a PNG, GIF, or JPG. Yowza, that's a big file. Try again with an image file smaller than 10MB. This browser doesn't support image attachments. We recommend updating to the latest Internet Explorer, Google Chrome, or Firefox. Something went really wrong, and we can't process that image. Try again.

James Miller

I opened a pull request to get the --shebang option added to bundle install --binstubs - bundler/bundler#1467

Jeremy Kemper
Collaborator

@bensie wow, thank you!

northbear

Cool! i've just been going to ask it.
Well... How do you see it for multiuser rails/rack-hosting? It should be a separate accounts every users and/or apps.
I should install rbenv scripts system-wide (in /usr/lib/rbenv, by example), and setup proper skel files (under Linux of course) with paths to rbenv scripts. And further things i should done the same way as you do. Do i lost anything?

Jeremy Kemper
Collaborator

@northbear we don't do multiuser hosting, but that's sounds like a good approach. Let us know how it works for you!

Terence Lee

this is in the newly released 1.1.rc. Can someone test? Thanks!

Guillermo Iguaran
Collaborator

Just tested bundle install --binstubs --shebang ruby-local-exec, it's working fine

James Miller

@hone Tested here and looking good as well, thanks for the quick merge and release!

George Ornbo

@hone also tested ok for me

ara.t.howard

i am using rbenv and liking it a lot. however, on a production machine with 2, 3, even 4 rubyies i'm having a hard time seeing how it gives any advantage over ensuring each unicorn, passenger standalone, whatever is exec'd with the proper $PATH setting... can you explain?

Jeremy Kemper
Collaborator

@ahoward you're looking pretty good if your tooling is all setting PATH correctly. The two advantages I see to app-local binstubs are that 1) they're Bundler-agnostic, so your deploy tools don't have to care and 2) that it's simple to deploy a Ruby version change by just deploying the app, without having to change the PATH in the ops tooling (or the app having to know any ops internals, like what to prepend to PATH).

Francesc Esplugas

And what about using rbenv-vars to set the current rubies instead of app-local binstubs?

Jeremy Kemper
Collaborator

Could you expand on that, @fesplugas? I don't follow.

Lee Jarvis

jeremy, how do you guys handle installing gems per user when you've install ruby versions into /opt/ruby?

James Miller

@injekt You want to bundle install --deployment so your app's gems are installed to vendor/bundle (relative to your app) rather than installing into a per-user or the global gemset.

Jeremy Kemper
Collaborator

@injekt, exactly what @bensie said: we don't! Bundler is great for production.

In Capistrano-speak:

set(:bundle_cmd) { "#{release_path}/bin/bundle" }

after 'deploy:update_code', 'bundle:install'

namespace :bundle do
  task :install do
    run "#{bundle_cmd} install --gemfile #{release_path}/Gemfile --path #{shared_path}/bundle --deployment"
  end
end
ara.t.howard

we're keeping our passenger standalones alive with something like this:

https://gist.github.com/1274473

but i'd love un-hard wire the rbenv path... still it's a string specifying the version like anything else...

Erik Dahlstrand

Perhaps outside the scope for rbenv but I think beginners (including me) would greatly appreciate a complete deployment stack example. It takes some work to get all bits and pieces together...

E.g. Ubuntu + rbenv + Nginx + Passenger/Unicorn and how to deploy a Rails app to the stack using Capistrano + Bundler.

Sam Stephenson
Owner

@ahoward I think you can just replace the shebang with #!/usr/bin/env ruby-local-exec and remove the ENV['PATH'] line, no? (Assuming you have an .rbenv-version file in the script's directory, or a parent directory.)

Lee Jarvis

@jeremy apologies for the possibly stupid question, but in your previous comment you say your ./bin is nice and small and contains rake, bundle, cap, etc.. but hitting bundle install --binstubs doesn't generate any executables for bundle. So of course the cap task you mention:

set(:bundle_cmd) { "#{release_path}/bin/bundle" }

after 'deploy:update_code', 'bundle:install'

namespace :bundle do
  task :install do
    run "#{bundle_cmd} install --gemfile #{release_path}/Gemfile --path #{shared_path}/bundle --deployment"
  end
end

fails because #{release_path}/bin/bundle doesn't exist. I'm maybe missing something small here, but I'm bashing my head against the wall trying to perfect this deployment method. Any tips?

Jeremy Kemper
Collaborator

@injekt, let's think about the job that bin/bundle must perform. It needs to
1. invoke the correct Ruby, using ruby-local-exec as a shebang,
2. set BUNDLE_GEMFILE, so bundler is aware of the app's Gemfile, and
3. load RubyGems and invoke the bundle command.

Hmm... that's exactly what the generated binstubs do for other apps' executables. Are you thinking what I'm thinking? Yeah!

Just copy another generated binstub, change Gem.binpath('othergem', 'othercommand') to Gem.binpath('bundler', 'bundle'), and boom, you're good to go.

Cody Krieger

Notes on use with Capistrano/other tools would be nice as well. I found @shapeshed's article while Googling (http://shapeshed.com/journal/using-rbenv-to-manage-rubies/), which does a nice job of explaining everything.

/cc @erdah

Caleb Spare

@jeremy, you wrote

We have packages built for each version of Ruby we deploy. They're installed (as root) to paths in /opt and symlinked to into ~app/.rbenv/versions.

How, exactly, does that work? If I use ruby-build to install a ruby somewhere in /opt/, then there are all kinds of hard-coded paths to /opt/<blah> in my compiled ruby (both in various scripts, like gem, and in the compiled ruby executable). This seems like a manifestation of Issue 42 in ruby-build.

James Miller

Shouldn't be anything wrong with those hardcoded paths as long as you don't move the folder containing a compiled ruby out of /opt -- those paths will still be valid.

I think the issue in ruby-build surrounds the fact that if you build a binary and try to package/redistribute, it will only work if placed in the exact same path as it was originally installed.

Caleb Spare

@bensie, the binary packaging thing is what I was getting at. If you don't move it out of /opt/, which presumably is readonly for your app user, then you can't install gems without superuser permissions. (Although I guess you could change GEM_HOME to fix that?) And as you said, we can't cp the ruby to ~/.rbenv/versions/ because the paths will be wrong.

James Miller

The whole point here is that you don't install gems to Ruby gemsets. You use Bundler with the --deployment flag to install gems within the application (store them in vendor/bundle). So you should just install that Ruby and install bundler to its gemset. After that, all other gems get installed to app folders.

Caleb Spare

@bensie, oh sure, that makes sense. Thanks!

Sam Stephenson
Owner

Would someone like to transfer all the knowledge in this ticket to the wiki page?

Jonathan Rochkind

Hmm, with support for $RBENV_ROOT shell env variable in recent versions of rbenv can a "global" rbenv install be even simpler?

set RBENV_ROOT to, say, /opt/rbenv when installing rbenv or building rubies with rbenv build.

Make sure any user that will be using ruby/rbenv has RBENV_ROOT set to /opt/rbenv in their relevant profile/bashrc/etc files.

Done. No need for symlinks into ~/.rbenv.

I'll try this when I have a chance, but does it seem a worthwhile approach?

Fletcher Nichol

@jrochkind I'm doing something like this in a chef cookbook: https://github.com/fnichol/chef-rbenv/blob/master/templates/default/rbenv.sh.erb

That template gets dumped into /etc/profile.d/rbenv.sh which on many systems gets sourced for users on login.

Jonathan Rochkind
da01

When deploying:
Won't you have to create an unprivileged user for each app running on an app server cluster (e.g. Thin, Unicorn etc, behind NGINX or Apache)?

sudo thin -C config.yml start => Run thin first as root user, then drop down to user specified in config.yml

Or is each app cluster being run using the same "deploy user"?

James Miller

@jrochkind We run cron with /bin/bash -l -c 'cd /path/to/app && ./bin/rake task' - this ensures that the rbenv shims are loaded and the environment is setup correctly.

@da01 We run multiple applications running as the same deploy user with unicorn behind Nginx.

Hedgehog

@bensie, in that case are the shims really under /usr/local/rbenv/shims? looking at the rbenv and init sources it seems they are still expected to be under $HOME

James Miller

@hedgehog We add this to /etc/profile to ensure it looks in the right place:

export RBENV_ROOT=/usr/local/rbenv
export PATH=$RBENV_ROOT/shims:$RBENV_ROOT/bin:$PATH
Hedgehog

It also seems that here:

https://github.com/sstephenson/rbenv/blob/master/libexec/rbenv-init#L48

There needs to be some allowance for a system wide profile.

James Miller

The only thing that's required to make rbenv work is prepending the shims dir to your path. We don't use the init script.

Hedgehog

Essentially on Ubuntu you run into the issue of dash/bash, mentioned in another issue I can't recall right now.

I've raised a separate issue to discuss the choices and track what is involved: issue #191

In production settings dash is likely to be invoked more frequently in the days ahead, for the reasons cited in the move to dash - speed.

Appreciate any thoughts in issue #191.

Hedgehog

@sstephenson, I just added a bootstrap Ruby+Chef method to the wiki page.

IMO it is likely this is the best practice route in production settings, and maybe something like this should be moved to the rbenv README.md?

HTH?

Jonathan Rochkind

I just got the 'shared rbenv install' method working, apparently well so far, for a production machine.

I've significantly reorganized and expanded the Shared install wiki page, including some notes on production, and notes on pain points with permissions.

I've added a link to that wiki at the top of the Using rbenv in production wiki page.

It would be great if someone else who knows what they're doing could review, make sure it's accurate. Please let me know of anything that is wrong or could be improved (or just change it yourself in the wiki)

Josh Goebel
set(:bundle_cmd) { "#{release_path}/bin/bundle" }

How is this supposed to work unless bundle is already installed as a system gem? And if it's not where is the stub expected to load the actual bundler source from? And if it is, what's the point of the binstub for bundler?

James Miller

Bundler needs to be installed as a system gem, but that's often all you need. Bundler itself doesn't get loaded from a stub.

Josh Goebel

If that's true there isn't any point in setting bundle_cmd to a stub, is there?

James Miller

I can't think of any reason you would do that. You need bundler to generate the binstubs.

In our Chef recipes we install all Rubies for that box and install bundler in each Ruby.

Fletcher Nichol

@yyyc514 The reason for the bundle stub is to delegate to ruby-local-exec in the shebang which will use your .rbenv-version file. A bundle --binstubs --shebang ruby-local-exec will set up the correct shebang line in every wrapper script except for the bundle command itself. Without this bundle will be executed in whatever ruby binary it finds in its PATH at the time.

See @jeremy's comment above for an explanation.

Here's a sample bundle binstub:
https://github.com/fnichol/capistrano-fanfare/blob/master/lib/capistrano/fanfare/bundler.rb#L29-44

Jonathan Rochkind

Nope, I think the bundle_cmd stub may in fact be entirely unneccesary.

For my setup, used in production, with a shared rbenv install, documented at https://github.com/sstephenson/rbenv/wiki/shared-install-of-rbenv (I wrote that one), I don't think the bundler shebang is actually helpful --

For that particular setup, even with the bundler shebang, I seem to still need to have the PATH and RBENV_ROOT in the environment for things to work. And if I have those in the environment, the bundler shebang binstub is superfluous.

This could be because of my custom RBENV_ROOT, perhaps without that bundler shebang would work. Or it could be some other idiosyncracy of my setup, and maybe it could be made to work.

James Miller

Sounds like the shebang isn't necessary in your environment because you're only using a single ruby version (likely whatever you have set as global).

The shebang pointing to ruby-local-exec ensures that the correct ruby version is run, which it gets from RBENV_VERSION or a .rbenv-version file.

Jonathan Rochkind
Jan

@jeremy so are you still putting the bin/* directory under version control in the app? or are you creating them now on the production server with the --binstubs --shebang options?

Jeremy Kemper
Collaborator
jeremy commented July 08, 2012

@habermann24 yep! We like having just a handful of stubs rather than every gem's executables. We pare down the stubs, too. For example, bin/unicorn is

#!/usr/bin/env ruby-local-exec
require_relative '../config/boot'
load Gem.bin_path('unicorn', 'unicorn')
Jan

@jeremy thank you so much for that response! this approach seems very well thought through. just one more question. when running things as a cron job, as root or by some other means (runit service, whatever...), you would always need ruby-local-exec in your PATH, how do you ensure that? symlink /usr/local/bin/ruby-local-exec to ~app/.rbenv/bin/ruby-local-exec? I ran into some trouble there that's why i ask. Hope this hasn't been answered already in these comments!

as sstephenson said:
"All apps are deployed as the app user; RBENV_ROOT is ~app/.rbenv"

Mislav Marohnić
Collaborator

I've done a cleanup of the wiki, and with that I've removed the old versions of "using rbenv in production" and "shared install of rbenv". These guides were overly complex, suggested too many different ways of doing things, relied on dubious 3rd-party installation scripts, and almost none of the contents was in line with the process that @sstephenson and @jeremy outlined here.

The new "deployment" guide is here. Notice that it's very slim and the practices suggested in it are practically the same to what you should be doing in development, too.

With the upcoming 0.4 release (just around the corner), ruby-local-exec is no longer needed and will probably be deprecated. Your binstubs can just have ruby in the shebang. Therefore I've not included instructions for configuring shebangs for ruby-local-exec.

Basically the whole gist of the deployment guide is:

  1. Ensure rbenv is always in your PATH (take care of shell, cron, runit, whatever)
  2. Use generated binstubs to interact with the deployed app.
Mislav Marohnić mislav closed this December 31, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.