Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Stager component skeleton

This includes a basic gem skeleton that contains all of the staging code from the CC.
Apart from some minor modifications to get this working without Rails, the code is unchanged.

Test plan:

- Supplied unit tests pass

Change-Id: I65d55c686bed1f8d3301c91cd74ed857f3fd3aba
  • Loading branch information...
commit a59fd12acf0284f3072cfecdf04f86975579e8d5 0 parents
mpage authored
Showing with 4,162 additions and 0 deletions.
  1. +13 −0 Gemfile
  2. +52 −0 Gemfile.lock
  3. +52 −0 Rakefile
  4. +2 −0  lib/vcap/stager.rb
  5. +11 −0 lib/vcap/stager/plugin.rb
  6. +504 −0 lib/vcap/stager/plugin/common.rb
  7. +44 −0 lib/vcap/stager/plugin/gem_cache.rb
  8. +85 −0 lib/vcap/stager/plugin/gemfile_support.rb
  9. +282 −0 lib/vcap/stager/plugin/gemfile_task.rb
  10. +89 −0 lib/vcap/stager/plugin/grails/plugin.rb
  11. +7 −0 lib/vcap/stager/plugin/grails/stage
  12. BIN  lib/vcap/stager/plugin/java_common/resources/auto-reconfiguration-0.6.0-BUILD-SNAPSHOT.jar
  13. +16 −0 lib/vcap/stager/plugin/java_common/resources/autostaging_template_grails.xml
  14. +16 −0 lib/vcap/stager/plugin/java_common/resources/autostaging_template_spring.xml
  15. +2 −0  lib/vcap/stager/plugin/java_common/resources/droplet.yaml
  16. +60 −0 lib/vcap/stager/plugin/java_common/resources/generate_server_xml
  17. +15 −0 lib/vcap/stager/plugin/java_common/resources/set_environment
  18. BIN  lib/vcap/stager/plugin/java_common/resources/tomcat.zip
  19. +191 −0 lib/vcap/stager/plugin/java_common/tomcat.rb
  20. +123 −0 lib/vcap/stager/plugin/lift/plugin.rb
  21. +7 −0 lib/vcap/stager/plugin/lift/stage
  22. +22 −0 lib/vcap/stager/plugin/manifests/grails.yml
  23. +20 −0 lib/vcap/stager/plugin/manifests/lift.yml
  24. +12 −0 lib/vcap/stager/plugin/manifests/node.yml
  25. +12 −0 lib/vcap/stager/plugin/manifests/otp_rebar.yml
  26. +3 −0  lib/vcap/stager/plugin/manifests/platform.yml
  27. +35 −0 lib/vcap/stager/plugin/manifests/rails3.yml
  28. +37 −0 lib/vcap/stager/plugin/manifests/sinatra.yml
  29. +22 −0 lib/vcap/stager/plugin/manifests/spring.yml
  30. +50 −0 lib/vcap/stager/plugin/node/plugin.rb
  31. +5 −0 lib/vcap/stager/plugin/node/stage
  32. +120 −0 lib/vcap/stager/plugin/otp_rebar/plugin.rb
  33. +60 −0 lib/vcap/stager/plugin/otp_rebar/runtime_info.yml
  34. +5 −0 lib/vcap/stager/plugin/otp_rebar/stage
  35. +78 −0 lib/vcap/stager/plugin/rails3/database_support.rb
  36. +93 −0 lib/vcap/stager/plugin/rails3/plugin.rb
  37. +8 −0 lib/vcap/stager/plugin/rails3/stage
  38. +61 −0 lib/vcap/stager/plugin/sinatra/plugin.rb
  39. +5 −0 lib/vcap/stager/plugin/sinatra/stage
  40. +70 −0 lib/vcap/stager/plugin/spring/plugin.rb
  41. +7 −0 lib/vcap/stager/plugin/spring/stage
  42. +5 −0 lib/vcap/stager/version.rb
  43. +17 −0 spec/Rakefile
  44. BIN  spec/fixtures/apps/grails_default_appcontext_no_context_config/source.war
  45. BIN  spec/fixtures/apps/grails_skip_autoconfig/source.war
  46. BIN  spec/fixtures/apps/lift_no_lift_filter/source.war
  47. BIN  spec/fixtures/apps/lift_no_web_config/source.war
  48. BIN  spec/fixtures/apps/lift_simple/source.war
  49. BIN  spec/fixtures/apps/lift_simple_servlet/source.war
  50. +27 −0 spec/fixtures/apps/rails3_gitgems/source/Gemfile
  51. +184 −0 spec/fixtures/apps/rails3_gitgems/source/Gemfile.lock
  52. +7 −0 spec/fixtures/apps/rails3_gitgems/source/Rakefile
  53. +2 −0  spec/fixtures/apps/rails3_gitgems/source/app/controllers/application_controller.rb
  54. +5 −0 spec/fixtures/apps/rails3_gitgems/source/app/controllers/welcome_controller.rb
  55. +2 −0  spec/fixtures/apps/rails3_gitgems/source/app/helpers/application_helper.rb
  56. +14 −0 spec/fixtures/apps/rails3_gitgems/source/app/views/layouts/application.html.erb
  57. +4 −0 spec/fixtures/apps/rails3_gitgems/source/config.ru
  58. +10 −0 spec/fixtures/apps/rails3_gitgems/source/config/application.rb
  59. +6 −0 spec/fixtures/apps/rails3_gitgems/source/config/boot.rb
  60. +5 −0 spec/fixtures/apps/rails3_gitgems/source/config/environment.rb
  61. +11 −0 spec/fixtures/apps/rails3_gitgems/source/config/environments/development.rb
  62. +9 −0 spec/fixtures/apps/rails3_gitgems/source/config/environments/production.rb
  63. +10 −0 spec/fixtures/apps/rails3_gitgems/source/config/environments/test.rb
  64. +7 −0 spec/fixtures/apps/rails3_gitgems/source/config/initializers/backtrace_silencers.rb
  65. +10 −0 spec/fixtures/apps/rails3_gitgems/source/config/initializers/inflections.rb
  66. +5 −0 spec/fixtures/apps/rails3_gitgems/source/config/initializers/mime_types.rb
  67. +7 −0 spec/fixtures/apps/rails3_gitgems/source/config/initializers/secret_token.rb
  68. +8 −0 spec/fixtures/apps/rails3_gitgems/source/config/initializers/session_store.rb
  69. +5 −0 spec/fixtures/apps/rails3_gitgems/source/config/locales/en.yml
  70. +3 −0  spec/fixtures/apps/rails3_gitgems/source/config/routes.rb
  71. 0  spec/fixtures/apps/rails3_gitgems/source/public/favicon.ico
  72. +6 −0 spec/fixtures/apps/rails3_gitgems/source/script/rails
  73. +4 −0 spec/fixtures/apps/rails3_no_assets/source/Gemfile
  74. +78 −0 spec/fixtures/apps/rails3_no_assets/source/Gemfile.lock
  75. +7 −0 spec/fixtures/apps/rails3_no_assets/source/Rakefile
  76. +2 −0  spec/fixtures/apps/rails3_no_assets/source/app/controllers/application_controller.rb
  77. +5 −0 spec/fixtures/apps/rails3_no_assets/source/app/controllers/welcome_controller.rb
  78. +2 −0  spec/fixtures/apps/rails3_no_assets/source/app/helpers/application_helper.rb
  79. +14 −0 spec/fixtures/apps/rails3_no_assets/source/app/views/layouts/application.html.erb
  80. +4 −0 spec/fixtures/apps/rails3_no_assets/source/config.ru
  81. +10 −0 spec/fixtures/apps/rails3_no_assets/source/config/application.rb
  82. +6 −0 spec/fixtures/apps/rails3_no_assets/source/config/boot.rb
  83. +5 −0 spec/fixtures/apps/rails3_no_assets/source/config/environment.rb
  84. +11 −0 spec/fixtures/apps/rails3_no_assets/source/config/environments/development.rb
  85. +10 −0 spec/fixtures/apps/rails3_no_assets/source/config/environments/production.rb
  86. +10 −0 spec/fixtures/apps/rails3_no_assets/source/config/environments/test.rb
  87. +7 −0 spec/fixtures/apps/rails3_no_assets/source/config/initializers/backtrace_silencers.rb
  88. +10 −0 spec/fixtures/apps/rails3_no_assets/source/config/initializers/inflections.rb
  89. +5 −0 spec/fixtures/apps/rails3_no_assets/source/config/initializers/mime_types.rb
  90. +7 −0 spec/fixtures/apps/rails3_no_assets/source/config/initializers/secret_token.rb
  91. +8 −0 spec/fixtures/apps/rails3_no_assets/source/config/initializers/session_store.rb
  92. +5 −0 spec/fixtures/apps/rails3_no_assets/source/config/locales/en.yml
  93. +3 −0  spec/fixtures/apps/rails3_no_assets/source/config/routes.rb
  94. 0  spec/fixtures/apps/rails3_no_assets/source/public/favicon.ico
  95. +6 −0 spec/fixtures/apps/rails3_no_assets/source/script/rails
  96. +3 −0  spec/fixtures/apps/rails3_nodb/source/Gemfile
  97. +71 −0 spec/fixtures/apps/rails3_nodb/source/Gemfile.lock
  98. +7 −0 spec/fixtures/apps/rails3_nodb/source/Rakefile
  99. +2 −0  spec/fixtures/apps/rails3_nodb/source/app/controllers/application_controller.rb
  100. +5 −0 spec/fixtures/apps/rails3_nodb/source/app/controllers/welcome_controller.rb
  101. +2 −0  spec/fixtures/apps/rails3_nodb/source/app/helpers/application_helper.rb
  102. +14 −0 spec/fixtures/apps/rails3_nodb/source/app/views/layouts/application.html.erb
  103. +4 −0 spec/fixtures/apps/rails3_nodb/source/config.ru
  104. +10 −0 spec/fixtures/apps/rails3_nodb/source/config/application.rb
  105. +6 −0 spec/fixtures/apps/rails3_nodb/source/config/boot.rb
  106. +5 −0 spec/fixtures/apps/rails3_nodb/source/config/environment.rb
  107. +11 −0 spec/fixtures/apps/rails3_nodb/source/config/environments/development.rb
  108. +9 −0 spec/fixtures/apps/rails3_nodb/source/config/environments/production.rb
  109. +10 −0 spec/fixtures/apps/rails3_nodb/source/config/environments/test.rb
  110. +7 −0 spec/fixtures/apps/rails3_nodb/source/config/initializers/backtrace_silencers.rb
  111. +10 −0 spec/fixtures/apps/rails3_nodb/source/config/initializers/inflections.rb
  112. +5 −0 spec/fixtures/apps/rails3_nodb/source/config/initializers/mime_types.rb
  113. +7 −0 spec/fixtures/apps/rails3_nodb/source/config/initializers/secret_token.rb
  114. +8 −0 spec/fixtures/apps/rails3_nodb/source/config/initializers/session_store.rb
  115. +5 −0 spec/fixtures/apps/rails3_nodb/source/config/locales/en.yml
  116. +3 −0  spec/fixtures/apps/rails3_nodb/source/config/routes.rb
  117. 0  spec/fixtures/apps/rails3_nodb/source/public/favicon.ico
  118. +6 −0 spec/fixtures/apps/rails3_nodb/source/script/rails
  119. +5 −0 spec/fixtures/apps/sinatra_gemfile/source/Gemfile
  120. +25 −0 spec/fixtures/apps/sinatra_gemfile/source/Gemfile.lock
  121. +39 −0 spec/fixtures/apps/sinatra_gemfile/source/Rakefile
  122. +12 −0 spec/fixtures/apps/sinatra_gemfile/source/app.rb
  123. +12 −0 spec/fixtures/apps/sinatra_trivial/source/app.rb
  124. BIN  spec/fixtures/apps/spring_context_config_foo/source.war
  125. BIN  spec/fixtures/apps/spring_default_appcontext_context_param_no_context_config/source.war
  126. BIN  spec/fixtures/apps/spring_default_appcontext_no_context_config/source.war
  127. BIN  spec/fixtures/apps/spring_default_servletcontext_init_param_no_context_config/source.war
  128. BIN  spec/fixtures/apps/spring_default_servletcontext_no_init_param/source.war
  129. BIN  spec/fixtures/apps/spring_guestbook/source.war
  130. BIN  spec/fixtures/apps/spring_no_context_config/source.war
  131. BIN  spec/fixtures/apps/spring_no_web_config/source.war
  132. BIN  spec/fixtures/apps/spring_servlet_context_config_foo/source.war
  133. BIN  spec/fixtures/apps/spring_servlet_no_init_param/source.war
  134. +15 −0 spec/functional/disabled_frameworks_spec.rb
  135. +78 −0 spec/functional/grails_spec.rb
  136. +128 −0 spec/functional/lift_spec.rb
  137. +124 −0 spec/functional/rails3_spec.rb
  138. +71 −0 spec/functional/sinatra_spec.rb
  139. +415 −0 spec/functional/spring_spec.rb
  140. +41 −0 spec/spec_helper.rb
  141. +33 −0 spec/support/custom_matchers.rb
  142. +70 −0 spec/support/staging_spec_helpers.rb
13 Gemfile
@@ -0,0 +1,13 @@
+source :rubygems
+
+gem 'i18n' # Active support apparently requires this but it isn't in the Gemfile??
+gem 'active_support'
+gem 'nokogiri', '>= 1.4.4'
+gem 'rake'
+gem 'yajl-ruby', '>= 0.7.9'
+
+gem 'vcap_common'
+
+group :test do
+ gem 'rspec'
+end
52 Gemfile.lock
@@ -0,0 +1,52 @@
+GEM
+ remote: http://rubygems.org/
+ specs:
+ active_support (3.0.0)
+ activesupport (= 3.0.0)
+ activesupport (3.0.0)
+ daemons (1.1.4)
+ diff-lcs (1.1.2)
+ eventmachine (0.12.10)
+ i18n (0.5.0)
+ json_pure (1.5.3)
+ little-plugger (1.1.2)
+ logging (1.5.2)
+ little-plugger (>= 1.1.2)
+ nats (0.4.10)
+ daemons (>= 1.1.0)
+ eventmachine (>= 0.12.10)
+ json_pure (>= 1.5.1)
+ nokogiri (1.5.0)
+ rack (1.3.1)
+ rake (0.9.2)
+ rspec (2.6.0)
+ rspec-core (~> 2.6.0)
+ rspec-expectations (~> 2.6.0)
+ rspec-mocks (~> 2.6.0)
+ rspec-core (2.6.4)
+ rspec-expectations (2.6.0)
+ diff-lcs (~> 1.1.2)
+ rspec-mocks (2.6.0)
+ thin (1.2.11)
+ daemons (>= 1.0.9)
+ eventmachine (>= 0.12.6)
+ rack (>= 1.0.0)
+ vcap_common (0.99)
+ eventmachine (~> 0.12.10)
+ logging (>= 1.5.0)
+ nats
+ thin
+ yajl-ruby
+ yajl-ruby (0.8.2)
+
+PLATFORMS
+ ruby
+
+DEPENDENCIES
+ active_support
+ i18n
+ nokogiri (>= 1.4.4)
+ rake
+ rspec
+ vcap_common
+ yajl-ruby (>= 0.7.9)
52 Rakefile
@@ -0,0 +1,52 @@
+require 'rubygems'
+require 'rake'
+require 'rake/gempackagetask'
+
+$LOAD_PATH.unshift(File.join(File.dirname(__FILE__), 'lib'))
+require 'vcap/stager/version'
+
+GEM_NAME = 'vcap_stager'
+GEM_VERSION = VCAP::Stager::VERSION
+
+gemspec = Gem::Specification.new do |s|
+ s.name = GEM_NAME
+ s.version = GEM_VERSION
+ s.platform = Gem::Platform::RUBY
+ s.summary = 'Component responsible for staging apps'
+ s.description = 'Takes an app package, environment, and services' \
+ + ' and produces a droplet that is executable by the DEA'
+ s.authors = ['Matt Page']
+ s.email = 'mpage@vmware.com'
+ s.homepage = 'http://www.cloudfoundry.com'
+ s.executables = []
+ s.bindir = 'bin'
+ s.require_path = 'lib'
+ s.files = %w(Rakefile) + Dir.glob("{lib,spec,vendor}/**/*")
+end
+
+Rake::GemPackageTask.new(gemspec) do |pkg|
+ pkg.gem_spec = gemspec
+end
+
+task :install => [:package] do
+ sh "gem install --no-ri --no-rdoc pkg/#{GEM_NAME}-#{GEM_VERSION}"
+end
+
+task :spec => ['bundler:install:test'] do
+ desc 'Run tests'
+ sh('cd spec && rake spec')
+end
+
+namespace 'bundler' do
+ task 'install' do
+ sh('bundle install')
+ end
+
+ environments = %w(test development production)
+ environments.each do |env|
+ desc "Install gems for #{env}"
+ task "install:#{env}" do
+ sh("bundle install --local --without #{(environments - [env]).join(' ')}")
+ end
+ end
+end
2  lib/vcap/stager.rb
@@ -0,0 +1,2 @@
+require 'vcap/stager/plugin'
+require 'vcap/stager/version'
11 lib/vcap/stager/plugin.rb
@@ -0,0 +1,11 @@
+require 'vcap/stager/plugin/gem_cache'
+require 'vcap/stager/plugin/gemfile_support'
+require 'vcap/stager/plugin/gemfile_task'
+require 'vcap/stager/plugin/common'
+require 'vcap/stager/plugin/grails/plugin'
+require 'vcap/stager/plugin/lift/plugin'
+require 'vcap/stager/plugin/node/plugin'
+require 'vcap/stager/plugin/otp_rebar/plugin'
+require 'vcap/stager/plugin/rails3/plugin'
+require 'vcap/stager/plugin/sinatra/plugin'
+require 'vcap/stager/plugin/spring/plugin'
504 lib/vcap/stager/plugin/common.rb
@@ -0,0 +1,504 @@
+require 'rubygems'
+
+# XXX - Why are we doing this?
+gemfile = File.expand_path('../../../../../Gemfile', __FILE__)
+if defined?(Bundler)
+ if File.realpath(gemfile) != File.realpath(ENV['BUNDLE_GEMFILE'])
+ puts "Incorrect BUNDLE_GEMFILE at staging startup: #{ENV['BUNDLE_GEMFILE']}"
+ exit 1
+ end
+else
+ ENV['BUNDLE_GEMFILE'] = gemfile
+ require 'bundler/setup'
+end
+
+require 'yaml'
+require 'yajl'
+require 'erb'
+require 'rbconfig'
+require 'active_support/core_ext'
+
+require 'tmpdir' # TODO - Replace this with something less absurd.
+# WARNING WARNING WARNING - Only create temp directories when running as a separate process.
+# The Ruby stdlib tmpdir implementation is beyond scary in long-running processes.
+# You Have Been Warned.
+
+
+require 'vcap/stager/plugin/gemfile_support'
+require 'vcap/stager/plugin/gemfile_task'
+require 'vcap/stager/plugin/gem_cache'
+
+# TODO - Separate the common staging helper methods from the 'StagingPlugin' base class, for more clarity.
+# Staging plugins (at least the ones written in Ruby) are expected to subclass this. See ruby/sinatra for a simple example.
+class StagingPlugin
+ attr_accessor :source_directory, :destination_directory, :environment_json
+
+ def self.staging_root
+ File.expand_path('..', __FILE__)
+ end
+
+ def self.manifest_root
+ ENV['STAGING_CONFIG_DIR'] || File.join(staging_root, 'manifests')
+ end
+
+ # This is a digestable version for the outside world
+ def self.manifests_info
+ @@manifests_info ||= {}
+ end
+
+ def self.manifests
+ @@manifests ||= {}
+ end
+
+ def self.platform_config
+ path = File.join(manifest_root, 'platform.yml')
+ YAML.load_file(path)
+ end
+
+ def self.validate_configuration!
+ config = platform_config
+ staging_cache_dir = config['cache']
+ begin
+ # NOTE - Make others as needed for other kinds of package managers.
+ FileUtils.mkdir_p File.join(staging_cache_dir, 'gems')
+ # TODO - Validate java runtimes as well.
+ check_ruby_runtimes
+ rescue => ex
+ puts "Staging environment validation failed: #{ex}"
+ exit 1
+ end
+ end
+
+ def self.get_ruby_version(exe)
+ get_ver = %{-e "print RUBY_VERSION,'p',RUBY_PATCHLEVEL"}
+ `env -i PATH=#{ENV['PATH']} #{exe} #{get_ver}`
+ end
+
+ # Checks the existence and version of the Ruby runtimes specified
+ # by the sinatra and rails staging manifests.
+ def self.check_ruby_runtimes
+ %w[sinatra rails3].each do |framework|
+ manifests[framework]['runtimes'].each do |hash|
+ hash.each do |name, properties|
+ exe, ver = properties['executable'], properties['version']
+ ver_pattern = Regexp.new(Regexp.quote(ver))
+ output = get_ruby_version(exe)
+ if $? == 0
+ unless output.strip =~ ver_pattern
+ raise "#{framework} runtime #{name} version was #{output.strip}, expected to match #{ver}*"
+ end
+ else
+ raise "#{framework} staging manifest has a bad runtime: #{name} (#{output.strip})"
+ end
+ end
+ end
+ end
+ end
+
+ # Generate a client side consumeable version of the manifest info
+ def self.generate_manifests_info
+ manifests.each_pair do |name, manifest|
+
+ runtimes = []
+ appservers = []
+
+ manifest['runtimes'].each do |runtime|
+ runtime.each_pair do |runtime_name, runtime_info|
+ runtimes << {
+ :name => runtime_name,
+ :version => runtime_info['version'],
+ :description => runtime_info['description'] }
+ end
+ end
+
+ if manifest['app_servers']
+ manifest['app_servers'].each do |appserver|
+ appserver.each_pair do |appserver_name, appserver_info|
+ appservers << {
+ :name => appserver_name,
+ # :version => appserver_info['version'],
+ :description => appserver_info['description'] }
+ end
+ end
+ end
+
+ m = {
+ :name => manifest['name'],
+ :runtimes => runtimes,
+ :appservers => appservers,
+ :detection => manifest['detection']
+ }
+ manifests_info[name] = m
+
+ end
+ end
+
+ def self.load_all_manifests
+ pattern = File.join(manifest_root, '*.yml')
+ Dir[pattern].each do |yaml_file|
+ next if File.basename(yaml_file) == 'platform.yml'
+ load_manifest(yaml_file)
+ end
+ generate_manifests_info
+ end
+
+ def self.load_plugin_for(framework)
+ framework = framework.to_s
+ plugin_path = File.join(staging_root, framework, 'plugin.rb')
+ require plugin_path
+ # This loads the default manifest; if a plugin gets passed an alternate
+ # manifest directory, and it finds a framework.yml file, it will replace this.
+ manifest_path = File.join(manifest_root, "#{framework}.yml")
+ load_manifest(manifest_path)
+ Object.const_get("#{framework.camelize}Plugin")
+ end
+
+ def self.load_manifest(path)
+ framework = File.basename(path, '.yml')
+ m = YAML.load_file(path)
+ unless m['disabled']
+ manifests[framework] = m
+ else
+ manifests.delete(framework)
+ end
+ rescue
+ puts "Failed to load staging manifest for #{framework} from #{path.inspect}"
+ exit 1
+ end
+
+ # This returns the staging framework names that claim to recognize the app
+ # found in the given +dir+. Order is not specified, and the caller must decide what
+ # it plans to do if multiple frameworks can be found in the given directory.
+ def self.matching_frameworks_for(dir)
+ matched = []
+ manifests.each do |name, staging_manifest|
+ rules = staging_manifest['detection']
+ if rules.all? { |rule| rule_matches_directory?(rule, dir) }
+ matched.push(name)
+ end
+ end
+ matched
+ end
+
+ def self.default_runtime_for(framework)
+ manifest = manifests[framework]
+ return nil unless manifest && manifest['runtimes']
+ manifest['runtimes'].each do |rt|
+ rt.each do |name, rt_info|
+ return name if rt_info['default']
+ end
+ end
+ end
+
+ # Exits the process with a nonzero status if ARGV does not contain valid
+ # staging args. If you call this in-process in an app server you deserve your fate.
+ def self.validate_arguments!
+ source, dest, env, manifest_dir, uid, gid = *ARGV
+ argfail! unless source && dest && env
+ argfail! unless File.directory?(File.expand_path(source))
+ argfail! unless File.directory?(File.expand_path(dest))
+ argfail! unless String === env
+ if manifest_dir
+ argfail! unless File.directory?(File.expand_path(manifest_dir))
+ end
+ end
+
+ def self.argfail!
+ puts "Invalid arguments for staging: #{ARGV.inspect}"
+ exit 1
+ end
+
+ def self.rule_matches_directory?(rule, dir)
+ dir = File.expand_path(dir)
+ results = rule.map do |glob, what|
+ full_glob = File.join(dir, glob)
+ case what
+ when String
+ pattern = Regexp.new(what)
+ scan_files_for_regexp(dir, full_glob, pattern).any?
+ when true
+ scan_files(dir, full_glob).any?
+ else
+ scan_files(dir, full_glob).empty?
+ end
+ end
+ results.all?
+ end
+
+ def self.scan_files(base_dir, glob)
+ found = []
+ base_dir << '/' unless base_dir.ends_with?('/')
+ Dir[glob].each do |full_path|
+ matched = block_given? ? yield(full_path) : true
+ if matched
+ relative_path = full_path.dup
+ relative_path[base_dir] = ''
+ found.push(relative_path)
+ end
+ end
+ found
+ end
+
+ def self.scan_files_for_regexp(base_dir, glob, pattern)
+ scan_files(base_dir, glob) do |path|
+ matched = false
+ File.open(path, 'rb') do |f|
+ matched = true if f.read.match(pattern)
+ end
+ matched
+ end
+ end
+
+ # If you re-implement this in a subclass:
+ # A) Do not change the method signature
+ # B) Make sure you call 'super'
+ #
+ # a good subclass impl would look like:
+ # def initialize(source, dest, env = nil, manifest_dir = nil)
+ # super
+ # whatever_you_have_planned
+ # end
+ def initialize(source_directory, destination_directory, environment_json = nil, manifest_dir = nil, uid=nil, gid=nil)
+ @source_directory = File.expand_path(source_directory)
+ @destination_directory = File.expand_path(destination_directory)
+ @environment_json = environment_json || '{}'
+ @manifest_dir = nil
+ if manifest_dir
+ @manifest_dir = ENV['STAGING_CONFIG_DIR'] = File.expand_path(manifest_dir)
+ end
+
+ # Drop privs before staging
+ # res == real, effective, saved
+ @staging_gid = gid.to_i if gid
+ @staging_uid = uid.to_i if uid
+ end
+
+ def framework
+ raise NotImplementedError, "subclasses must implement a 'framework' method that returns a string"
+ end
+
+ def stage_application
+ raise NotImplementedError, "subclasses must implement a 'stage_application' method"
+ end
+
+ def environment
+ @environment ||= Yajl::Parser.parse(environment_json, :symbolize_keys => true)
+ end
+
+ def staging_command
+ runtime['staging']
+ end
+
+ def start_command
+ app_server['executable']
+ end
+
+ def local_runtime
+ '%VCAP_LOCAL_RUNTIME%'
+ end
+
+ def application_memory
+ if environment[:resources] && environment[:resources][:memory]
+ environment[:resources][:memory]
+ else
+ 512 #MB
+ end
+ end
+
+ def manifest
+ @manifest ||= begin
+ if @manifest_dir
+ path = File.join(@manifest_dir, "#{framework}.yml")
+ if File.exists?(path)
+ StagingPlugin.load_manifest(path)
+ else
+ StagingPlugin.manifests[framework]
+ end
+ else
+ StagingPlugin.manifests[framework]
+ end
+ end
+ end
+
+ # The specified :runtime, or the default.
+ def runtime
+ find_in_manifest(:runtimes, :runtime, 'a runtime')
+ end
+
+ # The specified :server, or the default.
+ def app_server
+ find_in_manifest(:app_servers, :server, 'an app server')
+ end
+
+ # Looks in the specified +environment+ key. If it is set, looks
+ # for a matching entry in the staging manifest and returns it.
+ # If not found in the environment, the default is returned.
+ # The process will exit if an unknown entry is given in the environment.
+ def find_in_manifest(manifest_key, environment_key, what)
+ choices = manifest[manifest_key.to_s]
+ if entry_name = environment[environment_key.to_sym]
+ choices.each do |hash|
+ hash.each do |name, attrs|
+ return attrs if name.to_s == entry_name
+ end
+ end
+ puts "Unable to find #{what} matching #{entry_name.inspect} in #{choices.inspect}"
+ exit 1
+ else
+ select_default_from choices
+ end
+ end
+
+ # Environment variables specified on the app supersede those
+ # set in the staging manifest for the runtime. Theoretically this
+ # would allow a user to run their Rails app in development mode, etc.
+ def environment_hash
+ @env_variables ||= build_environment_hash
+ end
+
+ # Given a list of 'runtimes' or 'app_servers', pick out the
+ # one that was marked as default. If none are so marked,
+ # the first option listed is returned.
+ def select_default_from(declarations)
+ listed = Array(declarations)
+ chosen = nil
+ listed.each do |hash|
+ hash.each do |name, properties|
+ if properties['default']
+ chosen = properties
+ else
+ chosen ||= properties
+ end
+ end
+ end
+ chosen
+ end
+
+ # Overridden in subclasses when the framework needs to start from a different directory.
+ def change_directory_for_start
+ "cd app"
+ end
+
+ def generate_startup_script(env_vars = {})
+ after_env_before_script = block_given? ? yield : "\n"
+ template = <<-SCRIPT
+#!/bin/bash
+<%= environment_statements_for(env_vars) %>
+<%= after_env_before_script %>
+<%= change_directory_for_start %>
+<%= start_command %> > ../logs/stdout.log 2> ../logs/stderr.log &
+STARTED=$!
+echo "$STARTED" >> ../run.pid
+echo "#!/bin/bash" >> ../stop
+echo "kill -9 $STARTED" >> ../stop
+echo "kill -9 $PPID" >> ../stop
+chmod 755 ../stop
+wait $STARTED
+ SCRIPT
+ # TODO - ERB is pretty irritating when it comes to blank lines, such as when 'after_env_before_script' is nil.
+ # There is probably a better way that doesn't involve making the above Heredoc horrible.
+ ERB.new(template).result(binding).lines.reject {|l| l =~ /^\s*$/}.join
+ end
+
+ # Generates newline-separated exports for the specified environment variables.
+ # If the value of one of the keys is false or nil, it will be an 'unset' instead of an 'export'
+ def environment_statements_for(vars)
+ lines = []
+ vars.each do |name, value|
+ if value
+ lines << "export #{name}=\"#{value}\""
+ else
+ lines << "unset #{name}"
+ end
+ end
+ lines.sort.join("\n")
+ end
+
+ def create_app_directories
+ FileUtils.mkdir_p File.join(destination_directory, 'app')
+ FileUtils.mkdir_p File.join(destination_directory, 'logs')
+ end
+
+ def create_startup_script
+ path = File.join(destination_directory, 'startup')
+ File.open(path, 'wb') do |f|
+ f.puts startup_script
+ end
+ FileUtils.chmod(0500, path)
+ end
+
+ def copy_source_files(dest = nil)
+ dest ||= File.join(destination_directory, 'app')
+ system "cp -a #{File.join(source_directory, "*")} #{dest}"
+ end
+
+ def detection_rules
+ manifest['detection']
+ end
+
+ def bound_services
+ environment[:services] || []
+ end
+
+ # Returns all the application files that match detection patterns.
+ # This excludes files that are checked for existence/non-existence.
+ # Returned pathnames are relative to the app directory:
+ # e.g. [sinatra_app.rb, lib/somefile.rb]
+ def app_files_matching_patterns
+ matching = []
+ app_dir = File.join(destination_directory, 'app')
+ detection_rules.each do |rule|
+ rule.each do |glob, pattern|
+ next unless String === pattern
+ full_glob = File.join(app_dir, glob)
+ files = StagingPlugin.scan_files_for_regexp(app_dir, full_glob, pattern)
+ matching.concat(files)
+ end
+ end
+ matching
+ end
+
+ # Full path to the Ruby we are running under.
+ def current_ruby
+ File.join(Config::CONFIG['bindir'], Config::CONFIG['ruby_install_name'])
+ end
+
+ # Returns a set of environment clauses, only allowing the names specified.
+ def minimal_env(*allowed)
+ env = ''
+ allowed.each do |var|
+ next unless ENV.key?(var)
+ env << "#{var}=#{ENV[var]} "
+ end
+ env.strip
+ end
+
+ # Constructs a hash containing the variables associated
+ # with the app's runtime.
+ def build_environment_hash
+ ret = {}
+ (runtime['environment'] || {}).each do |key,val|
+ ret[key.to_s.upcase] = val
+ end
+ ret
+ end
+
+ # If the manifest specifies a workable ruby, returns that.
+ # Otherwise, returns the path to the ruby we were started with.
+ def ruby
+ @ruby ||= \
+ begin
+ rb = runtime['executable']
+ pattern = Regexp.new(Regexp.quote(runtime['version']))
+ output = StagingPlugin.get_ruby_version(rb)
+ if $? == 0 && output.strip =~ pattern
+ rb
+ elsif "#{RUBY_VERSION}p#{RUBY_PATCHLEVEL}" =~ pattern
+ current_ruby
+ else
+ puts "No suitable runtime found. Needs version matching #{runtime['version']}"
+ exit 1
+ end
+ end
+ end
+end
44 lib/vcap/stager/plugin/gem_cache.rb
@@ -0,0 +1,44 @@
+require "digest/sha1"
+require "fileutils"
+require "tempfile"
+require "tmpdir"
+
+class GemCache
+
+ def initialize(directory)
+ @directory = directory
+ end
+
+ def put(gemfile_path, installed_gem_path)
+ return unless gemfile_path && File.exists?(gemfile_path)
+ return unless installed_gem_path && File.exists?(installed_gem_path)
+
+ dst_dir = cached_obj_dir(gemfile_path)
+
+ spec_dir = File.join(dst_dir, "specifications")
+ FileUtils.mkdir_p(spec_dir)
+
+ `cp -n #{installed_gem_path}/specifications/*.gemspec #{spec_dir}`
+ # Someone else is copying gem?
+ return installed_gem_path if $?.exitstatus != 0
+
+ `cp -a #{installed_gem_path}/* #{dst_dir} && touch #{dst_dir}/.done`
+ return installed_gem_path if $?.exitstatus != 0
+ dst_dir
+ end
+
+ def get(path)
+ return nil unless path && File.exists?(path)
+ dir = cached_obj_dir(path)
+ return nil if !File.exists?(File.join(dir, ".done"))
+ File.directory?(dir) ? dir : nil
+ end
+
+ private
+
+ def cached_obj_dir(path)
+ sha1 = Digest::SHA1.file(path).hexdigest
+ "%s/%s/%s/%s" % [ @directory, sha1[0..1], sha1[2..3], sha1[4..-1] ]
+ end
+
+end
85 lib/vcap/stager/plugin/gemfile_support.rb
@@ -0,0 +1,85 @@
+require 'vcap/stager/plugin/gemfile_task'
+require 'vcap/stager/plugin/common'
+
+module GemfileSupport
+
+ # OK, so this is our workhorse.
+ # 1. If file has no Gemfile.lock we never attempt to outsmart it, just stage it as is.
+ # 2. If app has been a subject to 'bundle install --local --deployment' we ignore it as
+ # user seems to be confident it just work in the environment he pushes into.
+ # 3. If app has been 'bundle package'd we attempt to compile and cache its gems so we can
+ # bypass compilation on the next staging (going to step 4 for missing gems).
+ # 4. If app just has Gemfile.lock, we fetch gems from Rubygems and cache them locally, then
+ # compile them and cache compilation results (using the same cache as in step 3).
+ # 5. Finally we just copy all these files back to a well-known location the app honoring
+ # Rubygems path structure.
+ # NB: ideally this should be refactored into a set of saner helper classes, as it's really
+ # hard to follow who calls what and where.
+ def compile_gems
+ @rack = true
+ @thin = true
+
+ return unless uses_bundler?
+ return if packaged_with_bundler_in_deployment_mode?
+
+ safe_env = [ "HTTP_PROXY", "HTTPS_PROXY", "NO_PROXY", "C_INCLUDE_PATH", "LIBRARY_PATH" ].map { |e| "#{e}='#{ENV[e]}'" }.join(" ")
+ path = [ "/bin", "/usr/bin", "/sbin", "/usr/sbin"]
+ path.unshift(File.dirname(ruby)) if ruby[0] == '/'
+
+ safe_env << " PATH='%s'" % [ path.uniq.join(":") ]
+ base_dir = StagingPlugin.platform_config["cache"]
+
+ app_dir = File.join(destination_directory, 'app')
+ ruby_cmd = "env -i #{safe_env} #{ruby}"
+
+ task = GemfileTask.new(app_dir, library_version, ruby_cmd, base_dir, @staging_uid, @staging_gid)
+
+ task.install
+ task.install_bundler
+ task.remove_gems_cached_in_app
+
+ @rack = task.bundles_rack?
+ @thin = task.bundles_thin?
+
+ write_bundle_config
+ end
+
+ def library_version
+ environment[:runtime] == "ruby19" ? "1.9.1" : "1.8"
+ end
+
+ # Can we expect to run this app on Rack?
+ def rack?
+ @rack
+ end
+
+ # Can we expect to run this app on Thin?
+ def thin?
+ @thin
+ end
+
+ def uses_bundler?
+ File.exists?(File.join(source_directory, 'Gemfile.lock'))
+ end
+
+ def packaged_with_bundler_in_deployment_mode?
+ File.directory?(File.join(source_directory, 'vendor', 'bundle', library_version))
+ end
+
+ # This sets a relative path to the bundle directory, so nothing is confused
+ # after the app is unpacked on a DEA.
+ def write_bundle_config
+ config = <<-CONFIG
+---
+BUNDLE_PATH: rubygems
+BUNDLE_DISABLE_SHARED_GEMS: "1"
+BUNDLE_WITHOUT: test
+ CONFIG
+ dot_bundle = File.join(destination_directory, 'app', '.bundle')
+ FileUtils.mkdir_p(dot_bundle)
+ File.open(File.join(dot_bundle, 'config'), 'wb') do |config_file|
+ config_file.print(config)
+ end
+ end
+end
+
282 lib/vcap/stager/plugin/gemfile_task.rb
@@ -0,0 +1,282 @@
+require "logger"
+require "fileutils"
+
+require 'vcap/stager/plugin/gem_cache'
+
+class GemfileTask
+
+ def initialize(app_dir, library_version, ruby_cmd, base_dir, uid=nil, gid=nil)
+ @app_dir = File.expand_path(app_dir)
+ @library_version = library_version
+ @cache_base_dir = File.join(base_dir, @library_version)
+
+ @ruby_cmd = ruby_cmd
+ @uid = uid
+ @gid = gid
+
+ log_file = File.expand_path(File.join(@app_dir, '..', 'logs', 'staging.log'))
+ FileUtils.mkdir_p(File.dirname(log_file))
+
+ @logger = Logger.new(log_file)
+ @logger.level = ENV["DEBUG"] ? Logger::DEBUG : Logger::INFO
+ @logger.formatter = lambda { |sev, time, pname, msg| "#{msg}\n" }
+
+ @cache = GemCache.new(File.join(@cache_base_dir, "gem_cache"))
+ end
+
+ def lockfile_path
+ File.join(@app_dir, 'Gemfile.lock')
+ end
+
+ def lockfile
+ File.read(lockfile_path)
+ end
+
+ # Returns an array of [gemname, version] pairs.
+ def dependencies
+ return @dependencies unless @dependencies.nil?
+ @dependencies = [ ]
+ lockfile.each_line do |line|
+ if line =~ /^\s{4}([-\w_.0-9]+)\s*\((.*)\)/
+ @dependencies << [$1, $2]
+ end
+ end
+ @dependencies
+ end
+
+ # TODO - Inject EM.system-compatible control here.
+ def install
+ install_gems(dependencies)
+ end
+
+ def remove_gems_cached_in_app
+ FileUtils.rm_rf(File.join(installation_directory, "cache"))
+ end
+
+ def install_bundler
+ install_gems([ ['bundler', '1.0.10'] ])
+ end
+
+ # The application includes some version of Thin in its bundle.
+ def bundles_thin?
+ dependencies.assoc('thin')
+ end
+
+ # The application includes some version of Rack in its bundle.
+ def bundles_rack?
+ dependencies.assoc('rack')
+ end
+
+ private
+
+ # Each dependency is a gem [name, version] pair;
+ # e.g. ['thin', '1.2.10']
+ def install_gems(gems)
+ missing = [ ]
+
+ blessed_gems_dir = File.join(@cache_base_dir, "blessed_gems")
+ FileUtils.mkdir_p(blessed_gems_dir)
+
+ gems.each do |(name, version)|
+ gem_filename = "%s-%s.gem" % [ name, version ]
+
+ user_gem_path = File.join(@app_dir, "vendor", "cache", gem_filename)
+ blessed_gem_path = File.join(blessed_gems_dir, gem_filename)
+
+ if File.exists?(user_gem_path)
+ installed_gem_path = @cache.get(user_gem_path)
+ unless installed_gem_path
+ @logger.debug "Installing user gem: #{user_gem_path}"
+ tmp_gem_dir = install_gem(user_gem_path)
+ installed_gem_path = @cache.put(user_gem_path, tmp_gem_dir)
+ end
+ @logger.info "Adding #{gem_filename} to app..."
+ copy_gem_to_app(installed_gem_path)
+
+ elsif File.exists?(blessed_gem_path)
+ installed_gem_path = @cache.get(blessed_gem_path)
+ unless installed_gem_path
+ @logger.debug "Installing blessed gem: #{blessed_gem_path}"
+ tmp_gem_dir = install_gem(blessed_gem_path)
+ installed_gem_path = @cache.put(blessed_gem_path, tmp_gem_dir)
+ end
+ @logger.info "Adding #{gem_filename} to app..."
+ copy_gem_to_app(installed_gem_path)
+
+ else
+ missing << [ name, version ]
+ end
+ end
+
+ Dir.mktmpdir do |tmp_dir|
+ fetch_gems(missing, tmp_dir)
+
+ missing.each do |(name, version)|
+ gem_filename = "%s-%s.gem" % [ name, version ]
+ gem_path = File.join(tmp_dir, gem_filename)
+
+ @logger.debug "Installing downloaded gem: #{gem_path}"
+ tmp_gem_dir = install_gem(gem_path)
+ installed_gem_path = @cache.put(gem_path, tmp_gem_dir)
+ output = `cp -n #{gem_path} #{blessed_gems_dir} 2>&1`
+ if $?.exitstatus != 0
+ @logger.debug "Failed adding #{gem_path} to #{blessed_gems_dir}: #{output}"
+ end
+ @logger.info "Adding #{gem_filename} to app..."
+
+ copy_gem_to_app(installed_gem_path)
+ end
+ end
+ end
+
+ def copy_gem_to_app(src)
+ return unless src && File.exists?(src)
+ FileUtils.mkdir_p(installation_directory)
+ `cp -a #{src}/* #{installation_directory}`
+ end
+
+ def installation_directory
+ File.join(@app_dir, 'rubygems', 'ruby', @library_version)
+ end
+
+ def fetch_gems(gems, directory)
+ return if gems.empty?
+ urls = gems.map { |(name, version)| rubygems_url_for(name, version) }.join(" ")
+ cmd = "wget --quiet --retry-connrefused --connect-timeout=5 --no-check-certificate #{urls}"
+
+ Dir.chdir(directory) do
+ system(cmd)
+ end
+ end
+
+ def rubygems_url_for(name, version)
+ "http://production.s3.rubygems.org/gems/#{name}-#{version}.gem"
+ end
+
+ # Stage the gemfile in a temporary directory that is readable by a secure user
+ # We may be able to get away with mv here instead of a cp
+ def stage_gemfile_for_install(src, tmp_dir)
+ output = `cp #{src} #{tmp_dir} 2>&1`
+ if $?.exitstatus != 0
+ @logger.debug "Failed copying #{src} to #{tmp_dir}: #{output}"
+ return nil
+ end
+
+ staged_gemfile = File.join(tmp_dir, File.basename(src))
+
+ output = `chmod -R 0744 #{staged_gemfile} 2>&1`
+ if $?.exitstatus != 0
+ @logger.debug "Failed chmodding #{tmp_dir}: #{output}"
+ nil
+ else
+ staged_gemfile
+ end
+ end
+
+ # Perform a gem install from src_dir into a temporary directory
+ def install_gem(gemfile_path)
+ # Create tempdir that will house everything
+ tmp_dir = Dir.mktmpdir
+ at_exit do
+ user = `whoami`.chomp
+ `sudo /bin/chown -R #{user} #{tmp_dir}` if @uid
+ FileUtils.rm_rf(tmp_dir)
+ end
+
+ # Copy gemfile into tempdir, make sure secure user can read it
+ staged_gemfile = stage_gemfile_for_install(gemfile_path, tmp_dir)
+ unless staged_gemfile
+ @logger.debug "Failed copying gemfile to staging dir for install"
+ return nil
+ end
+
+ # Create a temp dir that the user can write into (gem install into)
+ gem_install_dir = File.join(tmp_dir, 'gem_install_dir')
+ begin
+ Dir.mkdir(gem_install_dir)
+ rescue => e
+ @logger.error "Failed creating gem install dir: #{e}"
+ return nil
+ end
+
+ if @uid
+ chmod_output = `/bin/chmod 0755 #{gem_install_dir} 2>&1`
+ if $?.exitstatus != 0
+ @logger.error "Failed chmodding install dir: #{chmod_output}"
+ return nil
+ end
+
+ chown_output = `sudo /bin/chown -R #{@uid} #{tmp_dir} 2>&1`
+ if $?.exitstatus != 0
+ @logger.debug "Failed chowning install dir: #{chown_output}"
+ return nil
+ end
+ end
+
+ @logger.debug("Doing a gem install from #{staged_gemfile} into #{gem_install_dir} as user #{@uid || 'cc'}")
+ staging_cmd = "#{@ruby_cmd} -S gem install #{staged_gemfile} --local --no-rdoc --no-ri -E -w -f --ignore-dependencies --install-dir #{gem_install_dir}"
+ staging_cmd = "cd / && sudo -u '##{@uid}' #{staging_cmd}" if @uid
+
+ # Finally, do the install
+ pid = fork
+ if pid
+ # Parent, wait for staging to complete
+ Process.waitpid(pid)
+ child_status = $?
+
+ # Kill any stray processes that the gem compilation may have created
+ if @uid
+ `sudo -u '##{@uid}' pkill -9 -U #{@uid} 2>&1`
+ me = `whoami`.chomp
+ `sudo chown -R #{me} #{tmp_dir}`
+ @logger.debug "Failed chowning #{tmp_dir} to #{me}" if $?.exitstatus != 0
+ end
+
+ if child_status.exitstatus != 0
+ @logger.debug("Failed executing #{staging_cmd}")
+ nil
+ else
+ @logger.debug("Success!")
+ gem_install_dir
+ end
+ else
+ close_fds
+ exec(staging_cmd)
+ end
+ end
+
+ def close_fds
+ 3.upto(get_max_open_fd) do |fd|
+ begin
+ IO.for_fd(fd, "r").close
+ rescue
+ end
+ end
+ end
+
+ def get_max_open_fd
+ max = 0
+
+ dir = nil
+ if File.directory?("/proc/self/fd/") # Linux
+ dir = "/proc/self/fd/"
+ elsif File.directory?("/dev/fd/") # Mac
+ dir = "/dev/fd/"
+ end
+
+ if dir
+ Dir.foreach(dir) do |entry|
+ begin
+ pid = Integer(entry)
+ max = pid if pid > max
+ rescue
+ end
+ end
+ else
+ max = 65535
+ end
+
+ max
+ end
+
+end
89 lib/vcap/stager/plugin/grails/plugin.rb
@@ -0,0 +1,89 @@
+require 'nokogiri'
+
+require File.expand_path('../../java_common/tomcat', __FILE__)
+
+class GrailsPlugin < StagingPlugin
+ VMC_GRAILS_PLUGIN = "CloudFoundryGrailsPlugin"
+ def framework
+ 'grails'
+ end
+
+ def autostaging_template
+ File.join(File.dirname(__FILE__), '../java_common/resources', 'autostaging_template_grails.xml')
+ end
+
+ # Staging is skipped if the Grails configuration in ""WEB-INF/grails.xml" contains
+ # a reference to "VmcGrailsPlugin"
+ def skip_staging webapp_root
+ skip = false
+ grails_config_file = File.join(webapp_root, 'WEB-INF/grails.xml')
+ if File.exist? grails_config_file
+ skip = self.vmc_plugin_present grails_config_file
+ end
+ skip
+ end
+
+ def vmc_plugin_present grails_config_file
+ grails_config = Nokogiri::XML(open(grails_config_file))
+ prefix = grails_config.root.namespace ? "xmlns:" : ''
+ plugins = grails_config.xpath("//#{prefix}plugins/#{prefix}plugin[contains(normalize-space(), '#{VMC_GRAILS_PLUGIN}')]")
+ if (plugins == nil || plugins.empty?)
+ return false
+ end
+ true
+ end
+
+ def stage_application
+ Dir.chdir(destination_directory) do
+ create_app_directories
+ webapp_root = Tomcat.prepare(destination_directory)
+ copy_source_files(webapp_root)
+ Tomcat.configure_tomcat_application(destination_directory, webapp_root, self.autostaging_template, environment) unless self.skip_staging(webapp_root)
+ create_startup_script
+ end
+ end
+
+ def create_app_directories
+ FileUtils.mkdir_p File.join(destination_directory, 'logs')
+ end
+
+ # The Tomcat start script runs from the root of the staged application.
+ def change_directory_for_start
+ "cd tomcat"
+ end
+
+ # We redefine this here because Tomcat doesn't want to be passed the cmdline
+ # args that were given to the 'start' script.
+ def start_command
+ "./bin/catalina.sh run"
+ end
+
+ def configure_catalina_opts
+ # We want to set this to what the user requests, *not* set a minum bar
+ "-Xms#{application_memory}m -Xmx#{application_memory}m"
+ end
+
+ private
+ def startup_script
+ vars = environment_hash
+ vars['CATALINA_OPTS'] = configure_catalina_opts
+ generate_startup_script(vars) do
+ <<-GRAILS
+export CATALINA_OPTS="$CATALINA_OPTS `ruby resources/set_environment`"
+PORT=-1
+while getopts ":p:" opt; do
+ case $opt in
+ p)
+ PORT=$OPTARG
+ ;;
+ esac
+done
+if [ $PORT -lt 0 ] ; then
+ echo "Missing or invalid port (-p)"
+ exit 1
+fi
+ruby resources/generate_server_xml $PORT
+ GRAILS
+ end
+ end
+end
7 lib/vcap/stager/plugin/grails/stage
@@ -0,0 +1,7 @@
+#!/usr/bin/env ruby
+require File.expand_path('../../common', __FILE__)
+plugin_class = StagingPlugin.load_plugin_for('grails')
+plugin_class.validate_arguments!
+plugin_class.new(*ARGV).stage_application
+
+# vim: ts=2 sw=2 filetype=ruby
BIN  lib/vcap/stager/plugin/java_common/resources/auto-reconfiguration-0.6.0-BUILD-SNAPSHOT.jar
Binary file not shown
16 lib/vcap/stager/plugin/java_common/resources/autostaging_template_grails.xml
@@ -0,0 +1,16 @@
+<web-app>
+ <context-param>
+ <param-name>contextConfigLocation</param-name>
+ <param-value>classpath:META-INF/cloud/cloudfoundry-auto-reconfiguration-context.xml</param-value>
+ </context-param>
+
+ <servlet>
+ <servlet-name>Srping MVC Dispatcher Servlet</servlet-name>
+ <servlet-class>org.codehaus.groovy.grails.web.servlet.GrailsDispatcherServlet</servlet-class>
+ <init-param>
+ <param-name>contextConfigLocation</param-name>
+ <param-value>classpath:META-INF/cloud/cloudfoundry-auto-reconfiguration-context.xml</param-value>
+ </init-param>
+ <load-on-startup>1</load-on-startup>
+ </servlet>
+</web-app>
16 lib/vcap/stager/plugin/java_common/resources/autostaging_template_spring.xml
@@ -0,0 +1,16 @@
+<web-app>
+ <context-param>
+ <param-name>contextConfigLocation</param-name>
+ <param-value>classpath:META-INF/cloud/cloudfoundry-auto-reconfiguration-context.xml</param-value>
+ </context-param>
+
+ <servlet>
+ <servlet-name>Srping MVC Dispatcher Servlet</servlet-name>
+ <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
+ <init-param>
+ <param-name>contextConfigLocation</param-name>
+ <param-value>classpath:META-INF/cloud/cloudfoundry-auto-reconfiguration-context.xml</param-value>
+ </init-param>
+ <load-on-startup>1</load-on-startup>
+ </servlet>
+</web-app>
2  lib/vcap/stager/plugin/java_common/resources/droplet.yaml
@@ -0,0 +1,2 @@
+---
+state_file: tomcat.state
60 lib/vcap/stager/plugin/java_common/resources/generate_server_xml
@@ -0,0 +1,60 @@
+#!/usr/bin/env ruby
+
+# This script executes at app startup time because the
+# actual appserver port must be in the server.xml.
+
+tomcat_port = ARGV[0]
+exit 1 unless tomcat_port
+
+require 'erb'
+OUTPUT_PATH = "tomcat/conf/server.xml"
+
+TEMPLATE = <<-ERB
+<?xml version='1.0' encoding='utf-8'?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<!-- Note: A "Server" is not itself a "Container", so you may not
+ define subcomponents such as "Valves" at this level.
+ Documentation at /docs/config/server.html
+ -->
+<Server port="-1">
+
+ <Listener className="org.apache.catalina.core.JasperListener" />
+ <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
+ <Listener className="org.apache.catalina.mbeans.ServerLifecycleListener" />
+ <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
+
+ <Service name="Catalina">
+
+ <Connector port="<%= tomcat_port %>" protocol="HTTP/1.1"
+ connectionTimeout="20000" />
+
+ <Engine name="Catalina" defaultHost="localhost">
+
+ <Host name="localhost" appBase="webapps"
+ unpackWARs="true" autoDeploy="true"
+ xmlValidation="false" xmlNamespaceAware="false">
+ </Host>
+ </Engine>
+ </Service>
+</Server>
+ERB
+
+template = ERB.new(TEMPLATE)
+File.open(File.expand_path(OUTPUT_PATH), 'wb') do |file|
+ file.puts(template.result(binding))
+end
15 lib/vcap/stager/plugin/java_common/resources/set_environment
@@ -0,0 +1,15 @@
+#!/usr/bin/env ruby
+require 'uri'
+
+# TODO : replace this with a helper running in Tomcat that sets these up, since it's very hard (if not impossible)
+# to escape these (NO_PROXY regexp) in the command line.
+
+env = []
+{'HTTP_PROXY' => 'http', 'HTTPS_PROXY' => 'https'}.each do |env_var, protocol|
+ proxy = ENV[env_var]
+ if proxy
+ uri = URI.parse(proxy)
+ env << "-D#{protocol}.proxyHost=#{uri.host} -D#{protocol}.proxyPort=#{uri.port}"
+ end
+end
+puts env.join(' ')
BIN  lib/vcap/stager/plugin/java_common/resources/tomcat.zip
Binary file not shown
191 lib/vcap/stager/plugin/java_common/tomcat.rb
@@ -0,0 +1,191 @@
+require 'nokogiri'
+require 'fileutils'
+
+class Tomcat
+ AUTOSTAGING_JAR = 'auto-reconfiguration-0.6.0-BUILD-SNAPSHOT.jar'
+ DEFAULT_APP_CONTEXT = "/WEB-INF/applicationContext.xml"
+ DEFAULT_SERVLET_CONTEXT_SUFFIX = "-servlet.xml"
+
+ def self.resource_dir
+ File.join(File.dirname(__FILE__), 'resources')
+ end
+
+ def self.prepare(dir)
+ FileUtils.cp_r(resource_dir, dir)
+ output = %x[cd #{dir}; unzip -q resources/tomcat.zip]
+ raise "Could not unpack Tomcat: #{output}" unless $? == 0
+ webapp_path = File.join(dir, "tomcat", "webapps", "ROOT")
+ server_xml = File.join(dir, "tomcat", "conf", "server.xml")
+ FileUtils.rm_f(server_xml)
+ FileUtils.rm(File.join(dir, "resources", "tomcat.zip"))
+ FileUtils.mv(File.join(dir, "resources", "droplet.yaml"), dir)
+ FileUtils.mkdir_p(webapp_path)
+ webapp_path
+ end
+
+ def self.configure_tomcat_application(staging_dir, webapp_root, autostaging_template, environment)
+ configure_autostaging(webapp_root, autostaging_template)
+ end
+
+ def self.configure_autostaging(webapp_path, autostaging_template)
+ web_config_file = File.join(webapp_path, 'WEB-INF/web.xml')
+ autostaging_context = get_autostaging_context autostaging_template
+ if File.exist? web_config_file
+ modify_autostaging_context(autostaging_context, web_config_file, webapp_path)
+ else
+ raise "Spring / J2EE application staging failed: web.xml not found"
+ end
+ jar_dest = File.join(webapp_path, 'WEB-INF/lib')
+ copy_jar AUTOSTAGING_JAR, jar_dest
+ end
+
+ def self.modify_autostaging_context(autostaging_context, web_config_file, webapp_path)
+ web_config = Nokogiri::XML(open(web_config_file))
+ web_config = configure_autostaging_context_param autostaging_context, web_config, webapp_path
+ web_config = configure_autostaging_servlet autostaging_context, web_config, webapp_path
+ File.open(web_config_file, 'w') {|f| f.write(web_config.to_xml) }
+ end
+
+ # Look for the presence of the "context-param" element in the top level (global context) of WEB-INF/web.xml
+ # and for a "contextConfigLocation" node within that.
+ # If present, update it if necessary (i.e. it does have a valid location) to include the context reference
+ # (provided by autostaging_context) that will handle autostaging.
+ # If not present, check for the presence of a default app context at WEB-INF/applicationContext.xml. If a
+ # default app context is present, introduce a "contextConfigLocation" element and set its value to include
+ # both the default app context as well as the context reference for autostaging.
+ def self.configure_autostaging_context_param(autostaging_context, webapp_config, webapp_path)
+ autostaging_context_param_name_node = autostaging_context.xpath("//context-param/param-name").first
+ autostaging_context_param_name = autostaging_context_param_name_node.content.strip
+ autostaging_context_param_value_node = autostaging_context.xpath("//context-param/param-value").first
+ autostaging_context_param_value = autostaging_context_param_value_node.content
+
+ prefix = webapp_config.root.namespace ? "xmlns:" : ''
+ context_param_nodes = webapp_config.xpath("//#{prefix}context-param")
+ if (context_param_nodes != nil && context_param_nodes.length > 0)
+ context_param_node = webapp_config.xpath("//#{prefix}context-param[contains(normalize-space(#{prefix}param-name), normalize-space('#{autostaging_context_param_name}'))]").first
+ if (context_param_node != nil)
+ webapp_config = update_context_value context_param_node.parent, prefix, "context-param", webapp_config, autostaging_context_param_name, autostaging_context_param_value
+ else
+ default_application_context_file = get_default_application_context_file(webapp_path)
+ unless default_application_context_file == nil
+ context_param_node = context_param_nodes.first
+ webapp_config = configure_default_context webapp_path, webapp_config, autostaging_context_param_name_node, autostaging_context_param_value, context_param_node, DEFAULT_APP_CONTEXT
+ end
+ end
+ else
+ default_application_context_file = get_default_application_context_file(webapp_path)
+ unless default_application_context_file == nil
+ context_param_node = Nokogiri::XML::Node.new 'context-param', webapp_config
+ webapp_config.root.add_child context_param_node
+ webapp_config = configure_default_context webapp_path, webapp_config, autostaging_context_param_name_node, autostaging_context_param_value, context_param_node, DEFAULT_APP_CONTEXT
+ end
+ end
+ webapp_config
+ end
+
+ # Look for the presence of the "init-param" element in the DispatcherServlet element of WEB-INF/web.xml
+ # and for a "contextConfigLocation" node within that.
+ # If present, update it to include the context reference (provided by the autostaging_context) that
+ # will handle autostaging.
+ # If not present, check for the presence of a default servlet context at
+ # WEB-INF/<servlet-name>-applicationContext.xml. If a default app context is present,
+ # introduce a "contextConfigLocation" element and set its value to include
+ # both the default servlet context as well as the context reference for autostaging.
+ def self.configure_autostaging_servlet (autostaging_context, webapp_config, webapp_path)
+ autostaging_servlet_class = autostaging_context.xpath("//servlet-class").first.content.strip
+ autostaging_init_param_name_node = autostaging_context.xpath("//servlet/init-param/param-name").first
+ autostaging_init_param_name = autostaging_init_param_name_node.content.strip
+ autostaging_init_param_value_node = autostaging_context.xpath("//servlet/init-param/param-value").first
+ autostaging_init_param_value = autostaging_init_param_value_node.content
+
+ prefix = webapp_config.root.namespace ? "xmlns:" : ''
+ dispatcher_servlet_nodes = webapp_config.xpath("//#{prefix}servlet[contains(normalize-space(#{prefix}servlet-class), normalize-space('#{autostaging_servlet_class}'))]")
+ if (dispatcher_servlet_nodes != nil && !dispatcher_servlet_nodes.empty?)
+ dispatcher_servlet_nodes.each do |dispatcher_servlet_node|
+ dispatcher_servlet_name = dispatcher_servlet_node.xpath("#{prefix}servlet-name").first.content.strip
+ default_servlet_context = "/WEB-INF/#{dispatcher_servlet_name}#{DEFAULT_SERVLET_CONTEXT_SUFFIX}"
+ init_param_node = dispatcher_servlet_node.xpath("#{prefix}init-param").first
+ if init_param_node != nil
+ init_param_name_node = dispatcher_servlet_node.xpath("#{prefix}init-param[contains(normalize-space(#{prefix}param-name), normalize-space('#{autostaging_init_param_name}'))]").first
+ if init_param_name_node != nil
+ webapp_config = update_context_value dispatcher_servlet_node, prefix, "init-param", webapp_config, autostaging_init_param_name, autostaging_init_param_value
+ else
+ webapp_config = configure_init_param_node(autostaging_init_param_name_node, autostaging_init_param_value, autostaging_init_param_value_node, default_servlet_context, dispatcher_servlet_name, dispatcher_servlet_node, init_param_node, webapp_config, webapp_path)
+ end
+ else
+ init_param_node = Nokogiri::XML::Node.new 'init-param', webapp_config
+ webapp_config = configure_init_param_node(autostaging_init_param_name_node, autostaging_init_param_value, autostaging_init_param_value_node, default_servlet_context, dispatcher_servlet_name, dispatcher_servlet_node, init_param_node, webapp_config, webapp_path)
+ end
+ end
+ end
+ webapp_config
+ end
+
+ def self.configure_default_context webapp_path, webapp_config, autostaging_context_param_name_node, autostaging_context_param_value, parent, default_context
+ context_param_value = "#{default_context} #{autostaging_context_param_value}"
+ context_param_value_node = Nokogiri::XML::Node.new 'param-value', webapp_config
+ context_param_value_node.content = context_param_value
+
+ parent.add_child autostaging_context_param_name_node
+ parent.add_child context_param_value_node
+
+ webapp_config
+
+ end
+
+ def self.update_context_value parent, prefix, selector, webapp_config, autostaging_context_param_name, autostaging_context_param_value
+ node = parent.xpath("#{prefix}#{selector}[contains(normalize-space(#{prefix}param-name), normalize-space('#{autostaging_context_param_name}'))]").first
+ context_param_value_node = node.xpath("#{prefix}param-value")
+ context_param_value = context_param_value_node.first.content
+
+ unless context_param_value.split.include?(autostaging_context_param_value) || context_param_value == ''
+ node.xpath("#{prefix}param-value").first.unlink
+ context_param_value << " #{autostaging_context_param_value}"
+
+ context_param_value_node = Nokogiri::XML::Node.new 'param-value', webapp_config
+ context_param_value_node.content = context_param_value
+ node.add_child context_param_value_node
+ end
+ webapp_config
+
+ end
+
+ def self.get_default_application_context_file(webapp_path)
+ default_application_context = File.join(webapp_path, 'WEB-INF/applicationContext.xml')
+ if File.exist? default_application_context
+ return default_application_context
+ end
+ nil
+ end
+
+ def self.get_default_servlet_context_file(webapp_path, servlet_name)
+ default_servlet_context = File.join(webapp_path, "WEB-INF/#{servlet_name}#{DEFAULT_SERVLET_CONTEXT_SUFFIX}")
+ if File.exist? default_servlet_context
+ return default_servlet_context
+ end
+ nil
+ end
+
+ def self.configure_init_param_node(autostaging_init_param_name_node, autostaging_init_param_value, autostaging_init_param_value_node, default_servlet_context, dispatcher_servlet_name, dispatcher_servlet_node, init_param_node, webapp_config, webapp_path)
+ default_servlet_context_file = get_default_servlet_context_file(webapp_path, dispatcher_servlet_name)
+ dispatcher_servlet_node.add_child init_param_node
+ unless default_servlet_context_file == nil
+ webapp_config = configure_default_context webapp_path, webapp_config, autostaging_init_param_name_node, autostaging_init_param_value, init_param_node, default_servlet_context
+ else
+ init_param_node.add_child autostaging_init_param_name_node
+ init_param_node.add_child autostaging_init_param_value_node
+ end
+ webapp_config
+ end
+
+ def self.copy_jar jar, dest
+ jar_path = File.join(File.dirname(__FILE__), 'resources', jar)
+ FileUtils.mkdir_p dest
+ FileUtils.cp(jar_path, dest)
+ end
+
+ def self.get_autostaging_context autostaging_template
+ Nokogiri::XML(open(autostaging_template))
+ end
+
+end
123 lib/vcap/stager/plugin/lift/plugin.rb
@@ -0,0 +1,123 @@
+# Copyright (c) 2009-2011 VMware, Inc.
+# Author: A.B.Srinivasan - asrinivasan@vmware.com
+
+require File.expand_path('../../java_common/tomcat', __FILE__)
+
+class LiftPlugin < StagingPlugin
+
+ LIFT_FILTER_CLASS = 'net.liftweb.http.LiftFilter'
+ CF_LIFT_PROPERTIES_GENERATOR_CLASS =
+ 'org.cloudfoundry.reconfiguration.CloudLiftServicesPropertiesGenerator';
+
+ def framework
+ 'lift'
+ end
+
+ def autostaging_template
+ File.join(File.dirname(__FILE__), '../java_common/resources', 'autostaging_template_spring.xml')
+ end
+
+ def skip_staging webapp_root
+ false
+ end
+
+ def stage_application
+ Dir.chdir(destination_directory) do
+ create_app_directories
+ webapp_root = Tomcat.prepare(destination_directory)
+ copy_source_files(webapp_root)
+ configure_cf_lift_servlet_context_listener(webapp_root)
+ Tomcat.configure_tomcat_application(destination_directory, webapp_root, self.autostaging_template, environment) unless self.skip_staging(webapp_root)
+ create_startup_script
+ end
+ end
+
+ def create_app_directories
+ FileUtils.mkdir_p File.join(destination_directory, 'logs')
+ end
+
+ # The Tomcat start script runs from the root of the staged application.
+ def change_directory_for_start
+ "cd tomcat"
+ end
+
+ # We redefine this here because Tomcat doesn't want to be passed the cmdline
+ # args that were given to the 'start' script.
+ def start_command
+ "./bin/catalina.sh run"
+ end
+
+ def configure_catalina_opts
+ # We want to set this to what the user requests, *not* set a minum bar
+ "-Xms#{application_memory}m -Xmx#{application_memory}m"
+ end
+
+ private
+ def startup_script
+ vars = environment_hash
+ vars['CATALINA_OPTS'] = configure_catalina_opts
+ generate_startup_script(vars) do
+ <<-LIFT
+export CATALINA_OPTS="$CATALINA_OPTS `ruby resources/set_environment`"
+env > env.log
+PORT=-1
+while getopts ":p:" opt; do
+ case $opt in
+ p)
+ PORT=$OPTARG
+ ;;
+ esac
+done
+if [ $PORT -lt 0 ] ; then
+ echo "Missing or invalid port (-p)"
+ exit 1
+fi
+ruby resources/generate_server_xml $PORT
+ LIFT
+ end
+ end
+
+ # We introspect the web configuration ('WEB-INF/web.xml' file) and
+ # if we find a LiftFilter node, we add a ServletContextListener
+ # before any servlet and filter nodes.
+ # The added ServletContextListener is responsible for generating a properties
+ # file that is consulted by the Lift framework to determine the binding
+ # information of the services used by the application.
+ def configure_cf_lift_servlet_context_listener(webapp_path)
+ web_config_file = File.join(webapp_path, 'WEB-INF/web.xml')
+ if File.exist? web_config_file
+ web_config = Nokogiri::XML(open(web_config_file))
+ prefix = web_config.root.namespace ? "xmlns:" : ''
+ lift_filter = web_config.xpath("//web-app/filter[contains(
+ normalize-space(#{prefix}filter-class),
+ '#{LIFT_FILTER_CLASS}')]")
+ unless lift_filter == nil || lift_filter.empty?
+ servlet_node = web_config.xpath("//web-app/servlet")
+ if servlet_node == nil || servlet_node.empty?
+ target_node = lift_filter.first
+ else
+ target_node = servlet_node.first
+ end
+ servlet_context_listener = generate_cf_servlet_context_listener(web_config)
+ target_node.add_previous_sibling(servlet_context_listener)
+ File.open(web_config_file, 'w') {|f| f.write(web_config.to_xml) }
+ else
+ raise "Scala / Lift application staging failed: no LiftFilter class found in web.xml"
+ end
+ else
+ raise "Scala / Lift application staging failed: web.xml not found"
+ end
+ end
+
+ def generate_cf_servlet_context_listener(web_config)
+ cf_servlet_context_listener = Nokogiri::XML::Node.new('listener', web_config)
+
+ cf_servlet_context_listener_class = Nokogiri::XML::Node.new('listener-class', web_config)
+ cf_servlet_context_listener_class.content = CF_LIFT_PROPERTIES_GENERATOR_CLASS
+
+ cf_servlet_context_listener.add_child(cf_servlet_context_listener_class)
+
+ cf_servlet_context_listener
+ end
+
+end
7 lib/vcap/stager/plugin/lift/stage
@@ -0,0 +1,7 @@
+#!/usr/bin/env ruby
+require File.expand_path('../../common', __FILE__)
+plugin_class = StagingPlugin.load_plugin_for('lift')
+plugin_class.validate_arguments!
+plugin_class.new(*ARGV).stage_application
+
+# vim: ts=2 sw=2 filetype=ruby
22 lib/vcap/stager/plugin/manifests/grails.yml
@@ -0,0 +1,22 @@
+---
+name: "grails"
+runtimes:
+ - "java":
+ description: "Java 6"
+ version: "1.6"
+ executable: "java"
+ default: true
+app_servers:
+ - "tomcat":
+ description: "Tomcat"
+ executable: "false"
+ default: true
+detection:
+ - "*.war": true # TODO - this thinks everything with an XML file is Spring
+staged_services:
+ - "name": "mysql"
+ "version": "*"
+ - "name": "postgresql"
+ "version": "*"
+
+# vim: filetype=yaml
20 lib/vcap/stager/plugin/manifests/lift.yml
@@ -0,0 +1,20 @@
+---
+name: "lift"
+runtimes:
+ - "java":
+ description: "Java 6"
+ version: "1.6"
+ executable: "java"
+ default: true
+app_servers:
+ - "tomcat":
+ description: "Tomcat"
+ executable: "false"
+ default: true
+detection:
+ - "*.war": true
+staged_services:
+ - "name": "mysql"
+ "version": "*"
+
+# vim: filetype=yaml
12 lib/vcap/stager/plugin/manifests/node.yml
@@ -0,0 +1,12 @@
+---
+name: "node"
+runtimes:
+ - node:
+ version: '0.4.5'
+ description: 'Node.js'
+ executable: node
+ default: true
+app_servers:
+detection:
+ - "*.js": '.'
+staged_services:
12 lib/vcap/stager/plugin/manifests/otp_rebar.yml
@@ -0,0 +1,12 @@
+---
+name: "otp_rebar"
+runtimes:
+ - erlangR14B02:
+ version: 'R14B02'
+ description: 'Erlang R14B02'
+ executable: /var/vcap/runtimes/erlang-R14B02/bin/erl
+ default: true
+app_servers:
+detection:
+ - "releases/*/*.rel": '.'
+staged_services:
3  lib/vcap/stager/plugin/manifests/platform.yml
@@ -0,0 +1,3 @@
+---
+cache: /var/vcap.local/staging
+runtime: # if nil, determined at load time by default 'ruby'
35 lib/vcap/stager/plugin/manifests/rails3.yml
@@ -0,0 +1,35 @@
+---
+name: "rails3"
+runtimes:
+ - "ruby18":
+ version: "1.8.7" # FIXME change to 1.8.7-p334
+ description: "Ruby 1.8.7"
+ executable: "/usr/bin/ruby" # FIXME - match vcap_setup
+ default: true
+ environment:
+ rails_env: "production"
+ bundle_gemfile:
+ rack_env: "production"
+ - "ruby19":
+ version: "1.9.2p180"
+ description: "Ruby 1.9.2"
+ executable: "ruby"
+ environment:
+ rails_env: "production"
+ bundle_gemfile:
+ rack_env: "production"
+app_servers:
+ - "thin":
+ description: "Thin"
+ executable: false
+ default: true
+detection:
+ - "config/application.rb": true
+ - "config/environment.rb": true
+staged_services:
+ - "name": "mysql"
+ "version": "*"
+ - "name": "postgresql"
+ "version": "*"
+
+# vim: filetype=yaml
37 lib/vcap/stager/plugin/manifests/sinatra.yml
@@ -0,0 +1,37 @@
+---
+name: "sinatra"
+runtimes:
+ - "ruby18":
+ version: "1.8.7" # FIXME change to 1.8.7-p334
+ description: "Ruby 1.8.7"
+ executable: "/usr/bin/ruby"
+ default: true
+ environment:
+ rails_env: "production"
+ bundle_gemfile:
+ rack_env: "production"
+ - "ruby19":
+ version: "1.9.2p180"
+ description: "Ruby 1.9.2"
+ executable: "ruby"
+ environment:
+ rails_env: "production"
+ bundle_gemfile:
+ rack_env: "production"
+app_servers:
+ - "thin":
+ description: "Thin"
+ executable: false # determined during staging
+ default: true
+detection:
+ - "*.rb": "require 'sinatra'|require \"sinatra\"" # .rb files in the root dir containing a require?
+ - "config/environment.rb": false # and config/environment.rb must not exist
+staged_services:
+ - "name": "mysql"
+ "version": "*"
+ - "name": "postgresql"
+ "version": "*"
+ - "name": "redis"
+ "version": "2"
+
+# vim: filetype=yaml
22 lib/vcap/stager/plugin/manifests/spring.yml
@@ -0,0 +1,22 @@
+---
+name: "spring"
+runtimes:
+ - "java":
+ description: "Java 6"
+ version: "1.6"
+ executable: "java"
+ default: true
+app_servers:
+ - "tomcat":
+ description: "Tomcat"
+ executable: "false"
+ default: true
+detection:
+ - "*.war": true # TODO - this thinks everything with an XML file is Spring
+staged_services:
+ - "name": "mysql"
+ "version": "*"
+ - "name": "postgresql"
+ "version": "*"
+
+# vim: filetype=yaml
50 lib/vcap/stager/plugin/node/plugin.rb
@@ -0,0 +1,50 @@
+class NodePlugin < StagingPlugin
+ # TODO - Is there a way to avoid this without some kind of 'register' callback?
+ # e.g. StagingPlugin.register('sinatra', __FILE__)
+ def framework
+ 'node'
+ end
+
+ def stage_application
+ Dir.chdir(destination_directory) do
+ create_app_directories
+ copy_source_files
+ create_startup_script
+ end
+ end
+
+ # Let DEA fill in as needed..
+ def start_command
+ "%VCAP_LOCAL_RUNTIME% #{detect_main_file} $@"
+ end
+
+ private
+ def startup_script
+ vars = environment_hash
+ generate_startup_script(vars)
+ end
+
+ # TODO - I'm fairly sure this problem of 'no standard startup command' is
+ # going to be limited to Sinatra and Node.js. If not, it probably deserves
+ # a place in the sinatra.yml manifest.
+ def detect_main_file
+ file, js_files = nil, app_files_matching_patterns
+
+ if js_files.length == 1
+ file = js_files.first
+ else
+ # We need to make this smarter, and then allow client to choose or
+ # send us a hint.
+
+ ['server.js', 'app.js', 'index.js', 'main.js', 'application.js'].each do |fname|
+ file = fname if js_files.include? fname
+ end
+ end
+
+ # TODO - Currently staging exceptions are not handled well.
+ # Convert to using exit status and return value on a case-by-case basis.
+ raise "Unable to determine Node.js startup command" unless file
+ file
+ end
+end
+
5 lib/vcap/stager/plugin/node/stage
@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+require File.expand_path('../../common', __FILE__)
+plugin_class = StagingPlugin.load_plugin_for('node')
+plugin_class.validate_arguments!
+plugin_class.new(*ARGV).stage_application
120 lib/vcap/stager/plugin/otp_rebar/plugin.rb
@@ -0,0 +1,120 @@
+class OtpRebarPlugin < StagingPlugin
+ def runtime_info_for(runtime_name)
+ unless @runtime_info
+ @runtime_info = YAML::load_file(File.expand_path('../runtime_info.yml', __FILE__))
+ end
+
+ @runtime_info[runtime_name]
+ end
+
+ # TODO - Is there a way to avoid this without some kind of 'register' callback?
+ # e.g. StagingPlugin.register('sinatra', __FILE__)
+ def framework
+ 'otp_rebar'
+ end
+
+ def stage_application
+ Dir.chdir(destination_directory) do
+ create_app_directories
+ copy_source_files
+ create_startup_script
+ rewrite_libs
+ update_vm_args
+ end
+ end
+
+ def start_command
+ command_lines = []
+
+ # Users can create a vmc.args file with shell variables (like $VMC_APP_PORT). The contents of
+ # this will be appended to the vm.args file.
+ if File.exists? 'app/etc/vmc.args'
+ File.read('app/etc/vmc.args').lines.each do |l|
+ command_lines << "echo #{l.strip.inspect} >>etc/vm.args"
+ end
+ end
+
+ # Always generate a node name
+ command_lines << "echo \"-name erl$VMC_APP_PORT@`hostname`\" >>etc/vm.args"
+
+ # Finally, we can start the app
+ command_lines << "sh bin/#{detect_app_name} console"
+
+ command_lines.join("\n")
+ end
+
+ private
+ def startup_script
+ vars = environment_hash
+ generate_startup_script(vars)
+ end
+
+ # We can't always assume that the libraries being pointed to in the release are compatible with our
+ # platform. For instance, if the release is built on a Mac, then included shared libraries won't work
+ # on Linux. So we'll rewrite all of the libs that are builtin to be symlinks to the runtime.
+ def rewrite_libs
+ runtime_version = runtime['version']
+ runtime_info = runtime_info_for(runtime_version)
+ runtime_dir = "/var/vcap/runtimes/erlang-#{runtime_version}"
+
+ # Ensure that our runtime matches the one that the libraries were packaged for
+ start_erl_data = File.read('app/releases/start_erl.data')
+ expected_erts = start_erl_data.split(/ /).first
+ runtime_erts = runtime_info['erts_version']
+ unless expected_erts == runtime_erts
+ raise "Application was released with different Erlang version to runtime. Selected Runtime ERTS: #{runtime_erts}, Packaged ERTS: #{expected_erts}"
+ end
+
+ # Link in the system runtime
+ FileUtils.rm_rf "app/erts-#{runtime_erts}"
+ FileUtils.ln_s "#{runtime_dir}/lib/erlang/erts-#{runtime_erts}", "app/erts-#{runtime_erts}"
+
+ builtin = runtime_info['builtins']
+ Dir['app/lib/*'].each do |lib_name|
+ base_lib_name = File.basename(lib_name)
+ if builtin.include? base_lib_name
+ # Candidate for replacement
+ FileUtils.rm_rf lib_name
+ FileUtils.ln_s "#{runtime_dir}/lib/erlang/lib/#{base_lib_name}", lib_name
+ end
+ end
+ end
+
+ # We want to alter the VM so that it doesn't want input, and that it doesn't need the double INT to close.
+ def update_vm_args
+ existing_args = File.read('app/etc/vm.args')
+
+ # We need to remove any -name declarations, since that would prevent us running multiple instances
+ cleaned_args = existing_args.lines.map { |l| if l =~ /^-name .*/ then '' else l end }
+ File.open('app/etc/vm.args', 'w') do |f|
+ f.puts cleaned_args.join
+ f.puts
+ f.puts "+B"
+ f.puts "-noinput"
+ end
+ end
+
+ # Detect the name of the application by looking for a startup script matching the .rel files.
+ def detect_app_name
+ app_files = app_files_matching_patterns
+
+ # We may have multiple releases. Look for app names where we also have a script in bin/ to boot them
+ interesting_app_files = app_files.select do |app_file|
+ app_name = File.basename(app_file)[0..-5] # Remove the .rel suffix
+ File.exists? "app/bin/#{app_name}"
+ end
+
+ appname = if interesting_app_files.length == 1
+ File.basename(interesting_app_files.first)[0..-5] # Remove the .rel suffix
+ elsif interesting_app_files.length == 0
+ raise "No valid Erlang releases with start scripts found. Cannot start application."
+ else
+ raise "Multiple Erlang releases with different names found. Cannot start application. (Found: #{interesting_app_files.inspect})"
+ end
+
+ # TODO - Currently staging exceptions are not handled well.
+ # Convert to using exit status and return value on a case-by-case basis.
+ raise "Unable to determine Erlang startup command" unless appname
+ appname
+ end
+end
60 lib/vcap/stager/plugin/otp_rebar/runtime_info.yml
@@ -0,0 +1,60 @@
+# Metadata about the various supported runtimes, used to validate and configure
+# staged applications.
+
+R14B02:
+ erts_version: 5.8.3
+ builtins:
+ - appmon-2.1.13
+ - cosFileTransfer-1.1.10
+ - debugger-3.2.6
+ - erts-5.8.3
+ - inets-5.5.2
+ - observer-0.9.9
+ - pman-2.7.1
+ - ssh-2.0.4
+ - tools-2.6.6.3
+ - asn1-1.6.16
+ - cosNotification-1.1.16
+ - dialyzer-2.4.2
+ - et-1.4.2
+ - inviso-0.6.2
+ - orber-3.6.20
+ - public_key-0.11
+ - ssl-4.1.4
+ - tv-2.1.4.6
+ - common_test-1.5.3
+ - cosProperty-1.1.13
+ - docbuilder-0.9.8.98
+ - eunit-2.1.6
+ - jinterface-1.5.4
+ - os_mon-2.2.5
+ - reltool-0.5.5
+ - stdlib-1.17.3
+ - typer-0.9
+ - compiler-4.7.3
+ - cosTime-1.1.10
+ - edoc-0.7.7
+ - gs-1.5.13
+ - kernel-2.14.3
+ - otp_mibs-1.0.6
+ - runtime_tools-1.8.5
+ - syntax_tools-1.6.7
+ - webtool-0.8.7
+ - cosEvent-2.1.10
+ - cosTransactions-1.2.10
+ - erl_docgen-0.2.4
+ - hipe-3.7.9
+ - megaco-3.15.1
+ - parsetools-2.0.5
+ - sasl-2.1.9.3
+ - test_server-3.4.3
+ - wx-0.98.9
+ - cosEventDomain-1.1.10
+ - crypto-2.0.2.1
+ - erl_interface-3.7.3
+ - ic-4.2.26
+ - mnesia-4.4.17
+ - percept-0.8.5
+ - snmp-4.19
+ - toolbar-1.4.1
+ - xmerl-1.2.8
5 lib/vcap/stager/plugin/otp_rebar/stage
@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+require File.expand_path('../../common', __FILE__)
+plugin_class = StagingPlugin.load_plugin_for('otp_rebar')
+plugin_class.validate_arguments!
+plugin_class.new(*ARGV).stage_application
78 lib/vcap/stager/plugin/rails3/database_support.rb
@@ -0,0 +1,78 @@
+module RailsDatabaseSupport
+ # Prepares a database.yml file for the app, if needed.
+ # Returns the service binding that was used for the 'production' db entry.
+ def configure_database
+ bindings = bound_databases
+ case bindings.size
+ when 0
+ # nothing to do
+ when 1
+ write_database_yaml(bindings.first)
+ else
+ configure_multiple_databases(bindings)
+ end
+ end
+
+ # Actually lay down a database.yml in the app's config directory.
+ def write_database_yaml(binding)
+ data = database_config_for(binding)
+ conf = File.join(destination_directory, 'app', 'config', 'database.yml')
+ File.open(conf, 'w') do |fh|
+ fh.write(YAML.dump('production' => data))
+ end
+ binding
+ end
+
+ def configure_multiple_databases(