Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Add erlang support using otp_rebar.

Also includes details of deploying a non-trivial Erlang/OTP app, and supports
multiple isolated Erlang installs.
  • Loading branch information...
commit 0afdb2b26afecc0adc84a580c080f90be6203d30 1 parent bf4a28c
@paulj paulj authored pbozeman committed
View
3  AUTHORS
@@ -2,7 +2,7 @@
# =========
#
# This is a reasonably complete list of everyone who has committed
-# code to the Cloud Foundry project. The list is just ordered by first
+# code to the Cloud Foundry project. The list is just ordered by first
# name here, but the file is (and should be kept) YAML-formatted.
#
# Please feel free to add any personal details.
@@ -20,6 +20,7 @@
- Nicholas Kushmerick { email: nicholask@vmware.com}
- Oleg Shaldybin { email: olegs@vmware.com}
- Patrick Bozeman { email: pbozeman@vmware.com}
+- Paul Jones { email: pauljones23@gmail.com }
- Peter Kukol { email: peterk@vmware.com }
- Pieter Noordhuis { email: pcnoordhuis@gmail.com }
- Ramnivas Laddad { email: rladdad@vmware.com }
View
4 cloud_controller/app/models/app.rb
@@ -19,8 +19,8 @@ class App < ActiveRecord::Base
AppStates = %w[STOPPED STARTED]
PackageStates = %w[PENDING STAGED FAILED]
- Runtimes = %w[ruby18 ruby19 java node]
- Frameworks = %w[sinatra rails3 spring grails node unknown]
+ Runtimes = %w[ruby18 ruby19 java node erlangR14B02]
+ Frameworks = %w[sinatra rails3 spring grails node otp_rebar unknown]
validates_presence_of :name, :framework, :runtime
View
2  cloud_controller/staging/common.rb
@@ -149,7 +149,7 @@ def self.load_plugin_for(framework)
# manifest directory, and it finds a framework.yml file, it will replace this.
manifest_path = File.join(manifest_root, "#{framework}.yml")
load_manifest(manifest_path)
- Object.const_get("#{framework.capitalize}Plugin")
+ Object.const_get("#{framework.camelize}Plugin")
end
def self.load_manifest(path)
View
12 cloud_controller/staging/manifests/otp_rebar.yml
@@ -0,0 +1,12 @@
+---
+name: "otp_rebar"
+runtimes:
+ - erlangR14B02:
+ version: 'R14B02'
+ description: 'Erlang R14B02'
+ executable: /var/vcap/runtimes/erlang-R14B02/bin/erl
+ default: true
+app_servers:
+detection:
+ - "releases/*/*.rel": '.'
+staged_services:
View
120 cloud_controller/staging/otp_rebar/plugin.rb
@@ -0,0 +1,120 @@
+class OtpRebarPlugin < StagingPlugin
+ def runtime_info_for(runtime_name)
+ unless @runtime_info
+ @runtime_info = YAML::load_file(File.expand_path('../runtime_info.yml', __FILE__))
+ end
+
+ @runtime_info[runtime_name]
+ end
+
+ # TODO - Is there a way to avoid this without some kind of 'register' callback?
+ # e.g. StagingPlugin.register('sinatra', __FILE__)
+ def framework
+ 'otp_rebar'
+ end
+
+ def stage_application
+ Dir.chdir(destination_directory) do
+ create_app_directories
+ copy_source_files
+ create_startup_script
+ rewrite_libs
+ update_vm_args
+ end
+ end
+
+ def start_command
+ command_lines = []
+
+ # Users can create a vmc.args file with shell variables (like $VMC_APP_PORT). The contents of
+ # this will be appended to the vm.args file.
+ if File.exists? 'app/etc/vmc.args'
+ File.read('app/etc/vmc.args').lines.each do |l|
+ command_lines << "echo #{l.strip.inspect} >>etc/vm.args"
+ end
+ end
+
+ # Always generate a node name
+ command_lines << "echo \"-name erl$VMC_APP_PORT@`hostname`\" >>etc/vm.args"
+
+ # Finally, we can start the app
+ command_lines << "sh bin/#{detect_app_name} console"
+
+ command_lines.join("\n")
+ end
+
+ private
+ def startup_script
+ vars = environment_hash
+ generate_startup_script(vars)
+ end
+
+ # We can't always assume that the libraries being pointed to in the release are compatible with our
+ # platform. For instance, if the release is built on a Mac, then included shared libraries won't work
+ # on Linux. So we'll rewrite all of the libs that are builtin to be symlinks to the runtime.
+ def rewrite_libs
+ runtime_version = runtime['version']
+ runtime_info = runtime_info_for(runtime_version)
+ runtime_dir = "/var/vcap/runtimes/erlang-#{runtime_version}"
+
+ # Ensure that our runtime matches the one that the libraries were packaged for
+ start_erl_data = File.read('app/releases/start_erl.data')
+ expected_erts = start_erl_data.split(/ /).first
+ runtime_erts = runtime_info['erts_version']
+ unless expected_erts == runtime_erts
+ raise "Application was released with different Erlang version to runtime. Selected Runtime ERTS: #{runtime_erts}, Packaged ERTS: #{expected_erts}"
+ end
+
+ # Link in the system runtime
+ FileUtils.rm_rf "app/erts-#{runtime_erts}"
+ FileUtils.ln_s "#{runtime_dir}/lib/erlang/erts-#{runtime_erts}", "app/erts-#{runtime_erts}"
+
+ builtin = runtime_info['builtins']
+ Dir['app/lib/*'].each do |lib_name|
+ base_lib_name = File.basename(lib_name)
+ if builtin.include? base_lib_name
+ # Candidate for replacement
+ FileUtils.rm_rf lib_name
+ FileUtils.ln_s "#{runtime_dir}/lib/erlang/lib/#{base_lib_name}", lib_name
+ end
+ end
+ end
+
+ # We want to alter the VM so that it doesn't want input, and that it doesn't need the double INT to close.
+ def update_vm_args
+ existing_args = File.read('app/etc/vm.args')
+
+ # We need to remove any -name declarations, since that would prevent us running multiple instances
+ cleaned_args = existing_args.lines.map { |l| if l =~ /^-name .*/ then '' else l end }
+ File.open('app/etc/vm.args', 'w') do |f|
+ f.puts cleaned_args.join
+ f.puts
+ f.puts "+B"
+ f.puts "-noinput"
+ end
+ end
+
+ # Detect the name of the application by looking for a startup script matching the .rel files.
+ def detect_app_name
+ app_files = app_files_matching_patterns
+
+ # We may have multiple releases. Look for app names where we also have a script in bin/ to boot them
+ interesting_app_files = app_files.select do |app_file|
+ app_name = File.basename(app_file)[0..-5] # Remove the .rel suffix
+ File.exists? "app/bin/#{app_name}"
+ end
+
+ appname = if interesting_app_files.length == 1
+ File.basename(interesting_app_files.first)[0..-5] # Remove the .rel suffix
+ elsif interesting_app_files.length == 0
+ raise "No valid Erlang releases with start scripts found. Cannot start application."
+ else
+ raise "Multiple Erlang releases with different names found. Cannot start application. (Found: #{interesting_app_files.inspect})"
+ end
+
+ # TODO - Currently staging exceptions are not handled well.
+ # Convert to using exit status and return value on a case-by-case basis.
+ raise "Unable to determine Erlang startup command" unless appname
+ appname
+ end
+end
View
60 cloud_controller/staging/otp_rebar/runtime_info.yml
@@ -0,0 +1,60 @@
+# Metadata about the various supported runtimes, used to validate and configure
+# staged applications.
+
+R14B02:
+ erts_version: 5.8.3
+ builtins:
+ - appmon-2.1.13
+ - cosFileTransfer-1.1.10
+ - debugger-3.2.6
+ - erts-5.8.3
+ - inets-5.5.2
+ - observer-0.9.9
+ - pman-2.7.1
+ - ssh-2.0.4
+ - tools-2.6.6.3
+ - asn1-1.6.16
+ - cosNotification-1.1.16
+ - dialyzer-2.4.2
+ - et-1.4.2
+ - inviso-0.6.2
+ - orber-3.6.20
+ - public_key-0.11
+ - ssl-4.1.4
+ - tv-2.1.4.6
+ - common_test-1.5.3
+ - cosProperty-1.1.13
+ - docbuilder-0.9.8.98
+ - eunit-2.1.6
+ - jinterface-1.5.4
+ - os_mon-2.2.5
+ - reltool-0.5.5
+ - stdlib-1.17.3
+ - typer-0.9
+ - compiler-4.7.3
+ - cosTime-1.1.10
+ - edoc-0.7.7
+ - gs-1.5.13
+ - kernel-2.14.3
+ - otp_mibs-1.0.6
+ - runtime_tools-1.8.5
+ - syntax_tools-1.6.7
+ - webtool-0.8.7
+ - cosEvent-2.1.10
+ - cosTransactions-1.2.10
+ - erl_docgen-0.2.4
+ - hipe-3.7.9
+ - megaco-3.15.1
+ - parsetools-2.0.5
+ - sasl-2.1.9.3
+ - test_server-3.4.3
+ - wx-0.98.9
+ - cosEventDomain-1.1.10
+ - crypto-2.0.2.1
+ - erl_interface-3.7.3
+ - ic-4.2.26
+ - mnesia-4.4.17
+ - percept-0.8.5
+ - snmp-4.19
+ - toolbar-1.4.1
+ - xmerl-1.2.8
View
5 cloud_controller/staging/otp_rebar/stage
@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+require File.expand_path('../../common', __FILE__)
+plugin_class = StagingPlugin.load_plugin_for('otp_rebar')
+plugin_class.validate_arguments!
+plugin_class.new(*ARGV).stage_application
View
5 dea/config/dea.yml
@@ -37,3 +37,8 @@ runtimes:
version: 1.6.0
version_flag: '-version'
environment:
+ erlangR14B02:
+ executable: /var/vcap/runtimes/erlang-R14B02/bin/erl
+ version: ".* 5.8.3"
+ version_flag: '-version'
+ environment:
View
31 docs/erlang-otp.md
@@ -0,0 +1,31 @@
+# Erlang/OTP Support
+
+Erlang/OTP applications are supported when using the rebar build tool and appropriate configuration to generate releases.
+
+## Demonstration
+
+Due to the somewhat irregular nature of Erlang application layouts, support currently only exists for applications
+using Rebar, and packaged into Erlang releases (usually via the rebar generate command).
+
+To demonstrate Erlang deployment, we'll deploy the Sean Cribb's riak\_id application (https://github.com/seancribbs/riak_id).
+
+- On your development system, ensure that you have Erlang R14B02 installed.
+- In an appropriate directory:
+ - <code>git clone https://github.com/seancribbs/riak\_id</code>
+ - <code>cd riak\_id</code>
+ - <code>wget --no-check-certificate https://github.com/downloads/basho/rebar/rebar</code>
+ - <code>echo -riak\_core http \'[{\"0.0.0.0\", '$VMC\_APP\_PORT'}]\' >rel/files/vmc.args</code>
+ - Add <code>{copy, "files/vmc.args", "etc/vmc.args"},</code> to rel/reltool.config directly before <code>{template, "files/app.config", "etc/app.config"}</code>
+ - <code>make rel</code>
+ - <code>cd rel/riak\_id</code>
+ - <code>vmc push riak\_id --url riak\_id.vcap.me -n</code>
+
+Visit http://riak_id.vcap.me/id. You should see a generated number. Multiple successive refreshes should generate new numbers.
+See the project page for an explanation of the meaning of these.
+
+Notable things in this deployment:
+
+- The application code was untouched
+- A file vmc.args was added to the release template. This file details the additional vm arguments that will be provided as the application is launched, thereby allowing for customisation of items such as the http port.
+- Multiple instances are not supported on the same machine, as the underlying riak\_core also requires a unique handoff port, which we're unable to allocate.
+- An application containing custom nifs won't deploy correctly unless the development machine is of an equivalent architecture to the deployment host (eg linux x64).
View
32 setup/vcap_setup
@@ -229,7 +229,7 @@ if [[ $DEA == true || $ALL == true ]]; then
fi
fi
- # Node
+ # Node and Erlang frameworks
pushd .
cd /tmp
@@ -245,9 +245,33 @@ if [[ $DEA == true || $ALL == true ]]; then
cd ..
rm node-v$NODE_VERSION.tar.gz
rm -fr node-v$NODE_VERSION
- popd
fi
+ ERLANG_VERSION=R14B02
+
+ if [ ! -d "/var/vcap/runtimes/erlang$ERLANG_VERSION" ]; then
+ if [[ $PLATFORM == 'Linux' ]]; then
+ apt-get -qqy install build-essential libncurses5-dev openssl libssl-dev
+ elif [[ $PLATFORM == 'MacOSX' ]]; then
+ echo -e "\033[31mFIXME: Install MacOSX dependencies \033[0m"
+ fi
+
+ wget http://erlang.org/download/otp_src_$ERLANG_VERSION.tar.gz
+ tar zxvf otp_src_$ERLANG_VERSION.tar.gz
+ cd otp_src_$ERLANG_VERSION
+
+ if [[ $PLATFORM == 'Linux' ]]; then
+ apt-get -qqy install build-essential libncurses5-dev openssl libssl-dev
+ elif [[ $PLATFORM == 'MacOSX' ]]; then
+ echo -e "\033[31mFIXME: Install MacOSX dependencies \033[0m"
+ fi
+
+ ./configure --prefix /var/vcap/runtimes/erlang-$ERLANG_VERSION
+ make
+ make install
+ fi
+ popd
+
echo "App Platform Support"
echo "===================="
@@ -261,6 +285,10 @@ if [[ $DEA == true || $ALL == true ]]; then
node -v
echo ""
+ echo -e "\nErlang should now be installed -->"
+ erl -version
+ echo ""
+
echo "Setting up Ruby System Gems and Support"
if [[ $PLATFORM == 'Linux' ]]; then
apt-get install -qqy ruby-dev libmysql-ruby libmysqlclient-dev libpq-dev postgresql-client
Please sign in to comment.
Something went wrong with that request. Please try again.