Permalink
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
518 lines (327 sloc) 24.3 KB

Ruby Scaffolding

Ruby Scaffolding is a Habitat package which helps you build your Ruby-based web applications, services, and processes (hereafter referred to as "apps") into a package which runs consistently on a wide range of containers, virtual machines, or servers via the Habitat Supervisor. The Supervisor will facilitate clustering, discovery of database services, dynamically update configuration and credentials, coordinate reliable rolling updates, and a lot more!

For more about Habitat, you can check out Try Habitat. For more about building your apps, read on!

Use Ruby Scaffolding for Your App

Check out the Ruby Scaffolding QuickStart Guide.

Scaffolding Detection

To properly load and use this Scaffolding, the implementation looks for a Gemfile in your app's root directory. Either directory structure is supported:

A Plan in a habitat/ subdirectory:

.
├── Gemfile
└── habitat
    └── plan.sh

A Plan in the same directory as the Gemfile:

.
├── Gemfile
└── plan.sh

Gemfile

A Gemfile must be present for this Scaffolding to function correctly. If no Gemfile.lock is present, a lock file will be generated by calling bundle lock. While committing a Gemfile.lock to source control is highly recommended, it is not strictly required.

Package Dependencies

Most non-trivial apps need more than their own codebase to run correctly. Many have gem dependencies which require compiling native gem extensions. Others require certain software present for shelling out to (for example: ImageMagick). This Scaffolding uses app detection logic to conditionally inject some common Habitat packages into your Plan. However, if more packages are required, it is generally easy to add these to your Plan to fully describe your app's build and runtime dependencies.

Default Dependencies

The following Habitat package dependencies will be injected into your app's Plan:

  • core/busybox-static: Used by process bins to have valid shebangs and a consistent minimal command set. Will be injected into your Plan's pkg_deps array.
  • core/git: Used to detect if your app exists within a git repository to better support installing your app while honoring the .gitignore file. Will be injected into your Plan's pkg_build_deps array.

Detected Dependencies

The following gems will checked for in the Gemfile.lock to conditionally inject commonly known Habitat packages into your app's Plan:

  • If the sqlite3 gem is detected, then sqlite-related Habitat packages will be injected into your Plan's pkg_deps array.
  • If any of several PostgreSQL-related gems are detected, then PostgreSQL-related Habitat packages will be injected into your Plan's pkg_deps array. See the PostgreSQL Database Detection section for more details.
  • If the nokogiri gem is detected, then XML/XSLT-related Habitat packages will be injected into your Plan's pkg_deps array.
  • If the execjs gem is detected, then Node-related Habitat packages will be injected into your Plan's pkg_deps array.
  • If the webpacker gem is detected, then Yarn-related Habitat packages will be injected into your Plan's pkg_deps array.

Additional checks performed by this scaffolding are:

  • The app's version of Ruby will be determined by checking several source locations. See the Ruby Version section for more details.

Specifying Run Dependencies in Your Plan

Generally speaking, your app needs additional Habitat packages present at runtime for one of two reasons:

  1. Your app expects certain programs present which it may shell out or bind to in some way.
  2. Your Gemfile has gem dependencies with native extensions, requiring additional linked libraries.

In both cases, this means your app needs additional Habitat packages installed at runtime. For this, we use the pkg_deps array in your Plan.

For this example, let's suppose you are using the rbnacl gem for bindings to the libsodium library. Additionally, your app resizes images to generate thumbnails by using the mogrify program from the ImageMagick software. To add both of these packages, update your pkg_deps line to the following below:

pkg_name=my_app
pkg_origin=acmecorp
pkg_version=0.1.0
 # ...
pkg_scaffolding=core/scaffolding-ruby
pkg_deps=(core/imagemagick core/libsodium)

Specifying Build Dependencies in Your Plan

Finally, your app may require additional software present in order to build your app that is not present by default.

For this example, let's suppose you are using the Helix project to write some Ruby classes in Rust. For this you need the Rust compiler present, but only at build time. To add the Rust compiler package, update your pkg_build_deps line to the following below:

pkg_name=my_app
pkg_origin=acmecorp
pkg_version=0.1.0
 # ...
pkg_scaffolding=core/scaffolding-ruby
pkg_build_deps=(core/rust)

Ruby Version

Selecting a Version of Ruby

By default the latest version of the MRI-based core/ruby package will be injected into your Plan's pkg_deps array. To specify a non-default version of Ruby, there are two locations you can do this:

  1. Use the ruby keyword in your app's Gemfile
  2. Set the scaffolding_ruby_pkg variable in your Plan with a valid Habitat package identifier corresponding to a package with a ruby program

A set Plan variable will win over setting a version in your Gemfile, however it is recommended to the Gemfile strategy first as this is portable across other Ruby app build and deployment solutions.

Specifying a Ruby Version in Your Gemfile

You can use the ruby keyword in your app's Gemfile to specify a version of Ruby. For example:

source "https://rubygems.org"

ruby "2.4.1"

 # ...

The value of this keyword will be used to determine the version of the Habitat core/ruby package. For example, the Gemfile above would result in core/ruby/2.4.1 being injected into your Plan's pkg_deps array.

Currently only releases of the core/ruby package will work so if further customizing is required you may need to specify a version of Ruby in your Plan as described below. Additionally, only MRI versions of Ruby are supported, however future support for JRuby implementations of Ruby is possible. Consequently, this means that the :engine and :engine_version options are not used.

Specifying a Ruby Version in Your Plan

You can set the scaffolding_ruby_pkg variable in your Plan to specify a version of Ruby. For example:

pkg_name=my_app
pkg_origin=acmecorp
pkg_version=0.1.0
 # ...
pkg_scaffolding=core/scaffolding-ruby

scaffolding_ruby_pkg=core/ruby/2.4.1

The value of this variable will be used to determine the Habitat package to satisfy the role of your app's Ruby implementation.

Currently only MRI versions of Ruby are supported, however future support for JRuby implementations of Ruby is possible.

Process Bins

Your app may have one or more top-level processes which map to a running service or ephemeral task. Each of these processes will be wrapped up in a small script which sets up a suitable app environment and invokes a command. By convention the main process bin which the package's run hook will invoke is the web process.

Default Process Bins

By default, the detected app type will determine some of the process bins to be created (see the specify app type documentation for more details). The following additional default process bins may be created.

Default Rake Process Bin

If your Gemfile.lock has the rake gem and a Rakefile exists in your app's root directory (also valid Rakefile names are rakefile, rakefile.rb, and Rakefile.rb), a rake process bin will be generated:

  • rake: bundle exec rake

Default Shell Process Bin

A bare-bones shell process bin will be generated for all apps. Due to the process bin wrapping logic, this shell session will have its $PATH correctly set, all appropriate environment variables set and will be running in the app's installed root path.

  • sh: sh

Specifying Process Bins

By default, the detected app type will determine some of the process bins to be created. To customize your app's process bins, there are two locations you can do this:

  1. Use a Procfile in your app's root directory
  2. Setup the scaffolding_process_bins hash in your Plan

For each process bin name a set Plan hash entry will win over a Procfile entry, however it is recommended to use the Procfile strategy first as this is portable across other Ruby app build and deployment solutions.

Specifying Process Bins in a Procfile

You can override default process bins or even add new ones by including a Procfile in your app's root directory. By convention, the web entry will be invoked by your package's run hook and will therefore be considered your package's main service. Additional entries will generate additional process bins in your package. For example, let's take an app whose package name is set to "my_app" (by setting pkg_name="my_app" in your plan.sh) and a Procfile containing:

web: bundle exec puma -t 5:5 -p $PORT
release: bundle exec rake db:migrate

The Scaffolding will produce my_app-web and my_app-release process bins in the resulting package under $pkg_prefix/bin which will be in the package's $PATH environment. Note that additional process bins may also be generated depending on the app type detected.

Specifying Process Bins in Your Plan

You can override default process bins or even add new ones by creating the scaffolding_process_bins hash in your Plan and setting one or more entries. By convention, the web entry will be invoked by your package's run hook and will therefore be considered your package's main service. Additional entries will generate addition process bins in your package. For example, let's take an app whose package name is set to "my_app" (by setting pkg_name="my_app" in your plan.sh):

pkg_name=my_app
pkg_origin=acmecorp
pkg_version=0.1.0
 # ...
pkg_scaffolding=core/scaffolding-ruby

 # Declare the associative array (hash) in bash
declare -A scaffolding_process_bins
 # Override the default web process
scaffolding_process_bins[web]='bundle exec puma -t 5:5 -p ${PORT}'
 # Add an addition process bin called release
scaffolding_process_bins[release]='bundle exec rake db:migrate'

The scaffolding will produce my_app-web and my_app-release process bins in the resulting package under $pkg_prefix/bin which will be in the package's $PATH environment. Note that additional process bins may also be generated depending on the app type detected.

Note: future work may make the Bash associative array creation an easier task.

App Environment Variables

In order to run correctly, your app may require several environment variables set up which it would consume on start. At build time, a Plan hash called scaffolding_env can be created to set up more app environment variables. The Scaffolding writes all of the app's environment variables to a config template which the Supervisor will compute and render at runtime. Each process bin will source the runtime-computed version of this config file before running itself meaning that all process bins have access to these variables.

Default App Environment Variables

By default, the detected app type will determine some of the app's environment variables (see the specific app type documentation for more details). The following default app environment variables are created:

  • LANG: {{cfg.lang}}
  • PORT: {{cfg.app.port}}

The values of these environment variables use handlebars templating, meaning that their values will be computed and rendered by the Supervisor at runtime.

Specifying App Environment Variables

By default, the detected app type will determine some of the app's environment variables to be created. To customize your app's environment variables, you would setup the scaffolding_env hash in your Plan.

For each environment variable name, a set Plan hash entry will win over a default entry.

Specifying App Environment Variables in Your Plan

You can override default app environment variables or even add new ones by creating the scaffolding_env hash in your Plan and setting one or more entries. Let's see an example:

pkg_name=my_app
pkg_origin=acmecorp
pkg_version=0.1.0
 # ...
pkg_scaffolding=core/scaffolding-ruby

 # Declare the associative array (hash) in bash
declare -A scaffolding_env
 # Add an addition variable which is hardcoded at build time
scaffolding_env[MY_PKG_VERSION]="$pkg_version/$pkg_release"
 # Add addition variables which is uses runtime config values
scaffolding_env[AWS_DEFAULT_REGION]="{{cfg.aws_default_region}}"

Note that in the above example we can choose to consume runtime configuration by writing the value with handlebars templating. This will be computed and rendered by the Supervisor at runtime and provided to your app. However, if the value is not "tunable" (i.e. would you change this setting in different environments or in production?), you can immutably set the variable by simply not using any handlebars templating syntax.

Config Settings

To make your app's package useful in more environments and contexts, your package will contain config settings. These settings will be computed at runtime and can be used to dynamically generate app config files, environment variable settings, etc.

Default Config Settings

The following config settings will be created for each app:

  • lang: Used to set the $LANG environment variable when the app runs. Defaults to "en_US.UTF-8".
  • app.port: Used to set the listen port number for your app when it runs. It is consumed to set the $PORT environment variable and to set the value of the exported port configuration. This enforces the port binding contract.

Specifying Config Settings

By default, the detected app type will determine some of the app's config settings to be created. To customize your app's config settings, you would create a default.toml file with your Plan.

Specifying Config Settings in default.toml

You can add more config settings by creating a default.toml file in the same directory containing your plan.sh file. Either directory structure is supported:

A Plan in a habitat/ subdirectory:

.
├── Gemfile
└── habitat
    ├── default.toml
    └── plan.sh

A Plan in the same directory as the Gemfile:

.
├── Gemfile
├── default.toml
└── plan.sh

Service Bindings

Service bindings are a powerful way to declare your app's service dependencies which the Supervisor will honor when running your app.

Default Service Bindings

By default, the detected app type or databases may generate service bindings, however none are generated to start with.

Specifying Service Bindings in Your Plan

You can add service bindings in your Plan by setting an entry in the pkg_binds hash. Let's see an example of declaring a required binding on an Elasticsearch service group:

pkg_name=my_app
pkg_origin=acmecorp
pkg_version=0.1.0
 # ...
pkg_scaffolding=core/scaffolding-ruby

 # We require both the HTTP and transport ports from this
 # service binding
pkg_binds[elasticsearch]="http-port transport-port"

This allows your app to dynamically bind to the desired Elasticsearch service group at runtime. For example, if your app's target Elasticsearch service group was "es.my_app", then you would start it with:

hab start acmecorp/my_app --bind elastic search:es.my_app

App Type Detection

Several popular Ruby-based frameworks are detected and supported with additional dependencies, configurations, etc. See below for details on the state of each app type.

Rails 5.x Applications

Detection

Rails 5 app type will be detected if the app's Gemfile.lock contains a railties gem, and if that gem's version is greater than 5.0.0 but less than 6.0.0.

Default Process Bins

The following default process bins will be generated:

  • web: bundle exec rails server -p $PORT
  • console: bundle exec rails console

Default App Environment Variables

The following default app environment variables will be created:

  • RAILS_ENV: {{cfg.rails_env}}
  • RACK_ENV: {{cfg.rack_env}}
  • RAILS_LOG_TO_STDOUT: enabled
  • RAILS_SERVE_STATIC_FILES: enabled
  • SECRET_KEY_BASE: {{cfg.secret_key_base}}

The RAILS_LOG_TO_STDOUT is used to help stream app logs, which Rails honors by default.

Note: all app environment variable values which use handlebars templating will be computed and rendered by the Supervisor at run time.

Default Config Settings

The following default config settings will be created:

  • rails_env: Used to set the $RAILS_ENV environment variable when the app runs. Defaults to "production".
  • rack_env: Used to set the $RACK_ENV environment variable when the app runs. Defaults to "production".

Rails 4.2.x Applications

Detection

A Rails 4.2.x app type will be detected if the app's Gemfile.lock contains a railties gem, and if that gem's version is greater than or equal to 4.2.0 but less than 5.0.0.

Additional Gems

This app uses the rails_12factor gem to enable support for serving static files and logging to standard out. If this gem is not detected, the build will fail instructing you to add this gem to the app's Gemfile.

Default Process Bins

The following default process bins will be generated:

  • web: bundle exec rails server -p $PORT
  • console: bundle exec rails console

Default App Environment Variables

The following default app environment variables will be created:

  • RAILS_ENV: {{cfg.rails_env}}
  • RACK_ENV: {{cfg.rack_env}}
  • RAILS_LOG_TO_STDOUT: enabled
  • RAILS_SERVE_STATIC_FILES: enabled
  • SECRET_KEY_BASE: {{cfg.secret_key_base}}

The RAILS_LOG_TO_STDOUT is used to help stream app logs, which Rails honors by default with help from the rails_12factor gem.

Note: all app environment variable values which use handlebars templating will be computed and rendered by the Supervisor at run time.

Default Config Settings

The following default config settings will be created:

  • rails_env: Used to set the $RAILS_ENV environment variable when the app runs. Defaults to "production".
  • rack_env: Used to set the $RACK_ENV environment variable when the app runs. Defaults to "production".

Rails 4.x Applications

Note: This app type is not fully supported yet, but may be in the future. If your use case is interesting, please join our public Slack and let us know!

Detection

A Rails 4 app type will be detected if the app's Gemfile.lock contains a railties gem, and if that gem's version is greater than or equal to 3.0.0 but less than 4.0.0.

Rails 3.x Applications

Note: This app type is not fully supported yet, but may be in the future. If your use case is interesting, please join our public Slack and let us know!

Detection

A Rails 3 app type will be detected if the app's Gemfile.lock contains a railties gem, and if that gem's version is greater than or equal to 3.0.0 but less than 4.0.0.

Rails 2.x Applications

Note: This app type is not fully supported yet, but may be in the future. If your use case is interesting, please join our public Slack and let us know!

Detection

A Rails 2 app type will be detected if the app's Gemfile.lock contains a railties gem, and if that gem's version is greater than or equal to 2.0.0 but less than 3.0.0.

Rack Applications

Detection

A Rack app type will be detected if the app's Gemfile.lock contains a rack gem.

Default Process Bins

The following default process bins will be generated:

  • web: bundle exec rackup config.ru -p $PORT
  • console: bundle exec irb

Default App Environment Variables

The following default app environment variables will be created:

  • RACK_ENV: {{cfg.rack_env}}

Default Config Settings

The following default config settings will be created:

  • rack_env: Used to set the $RACK_ENV environment variable when the app runs. Defaults to "production".

Ruby Applications

Note: This app type is not fully supported yet, but may be in the future. If your use case is interesting, please join our public Slack and let us know!

Detection

A Ruby app type will be detected if no other suitable app type can be determined.

PostgreSQL Database Detection

Your app's use of a PostgreSQL database is detected by inspecting the Gemfile.lock for the presence of the following gems:

  • pg
  • activerecord-jdbcpostgresql-adapter
  • jdbc-postgres
  • jdbc-postgresql
  • jruby-pg
  • rjack-jdbc-postgres
  • tgbyte-activerecord-jdbcpostgresql-adapter

NOTE: If you are using activerecord-postgis-adapter you will need to set the db.adapter config option to postgis in order to access the postgis functions in PostgreSQL.

Default App Environment Variables

  • DATABASE_URL: $adapter://$user:$password@$host:$port/$name

Where $adapter, $user, $password, $host, $port, and $name will be determined at runtime.

Default Config Settings

  • db.adapter: The connecting database adapter for this app. Defaults to the value of postgresql if no value is provided.
  • db.name: The database name on the database server for this app. Defaults to the value of "${pkg_name}_production".
  • db.user: The connecting database user for this app. Defaults to the value of "$pkg_name".
  • db.password: The connecting database password for this app. Defaults to the value of "${pkg_name}". It is strongly recommended to use a different, randomly generated password when running your app in production.
  • db.host: (Only when bind is not used) The hostname/IP address of the database. There is no default.
  • db.port: (Only when bind is not used) The listen port of the database. The default will be 5432 if no value is provided.

Default Service Bindings

If a database requirement is detected, an optional binding called database will be generated which requires the port configuration. This allows your app to dynamically bind to the desired database service group at runtime.

  • pkg_binds[database]="port"

To use the service binding, you will need to add the following option when loading and/or starting the service:

  • --bind database:<SERVICE_GROUP>

Where <SERVICE_GROUP> is a service group running PostgreSQL exporting the port configuration. For example:

hab start acmecorp/my_app --bind database:postgresql.default

If your database is not currently running in a Habitat ring under a Supervisor, you may omit adding the --bind database:... option when loading and/or starting your app service. This means that you must provide at least one additional config setting: db.host (as explained above).