Merge branch 'release/0.6.0' into stable

.circleci/config.yml

@@ -1,4 +1,8 @@
 version: 2
+general:
+  branches:
+    ignore:
+      - gh-pages
 jobs:
   build:
     machine: true
@@ -7,16 +11,16 @@ jobs:
       - checkout
       - run: echo 'export INSTALL_PATH="$HOME/dependencies";export PATH="$INSTALL_PATH/bin:$PATH";export MIX_ENV=test;export VERSION_CIRCLECI=2' >> $BASH_ENV
       - restore_cache:
-          key: environment-cache-{{ checksum ".circleci/config.yml" }}-{{ checksum "script/ci/prepare.sh" }}
+          key: environment-cache-{{ checksum ".circleci/config.yml" }}-{{ checksum "script/ci/prepare.sh" }}-{{ arch }}
       - run:
           name: Install Elixir
           command: script/ci/prepare.sh
       - save_cache:
-          key: environment-cache-{{ checksum ".circleci/config.yml" }}-{{ checksum "script/ci/prepare.sh" }}
+          key: environment-cache-{{ checksum ".circleci/config.yml" }}-{{ checksum "script/ci/prepare.sh" }}-{{ arch }}
           paths:
             - ~/dependencies
       - restore_cache:
-          key: dependencies-cache-{{ checksum ".circleci/config.yml" }}-{{ checksum "mix.lock" }}
+          key: dependencies-cache-{{ checksum ".circleci/config.yml" }}-{{ checksum "mix.lock" }}-{{ arch }}
       - run:
           name: Preparing dependencies
           command: |
@@ -27,7 +31,7 @@ jobs:
             mix dialyzer --plt;
           no_output_timeout: 10m
       - save_cache:
-          key: dependencies-cache-{{ checksum ".circleci/config.yml" }}-{{ checksum "mix.lock" }}
+          key: dependencies-cache-{{ checksum ".circleci/config.yml" }}-{{ checksum "mix.lock" }}-{{ arch }}
           paths:
             - ~/.mix
             - _build

.gitignore

@@ -18,3 +18,5 @@ erl_crash.dump
 
 # Also ignore archive artifacts (built via "mix archive.build").
 *.ez
+/_site
+/.sass-cache

README.md

@@ -4,18 +4,18 @@
 
 Simple deployment and server automation for Elixir.
 
-**Bootleg** is a simple set of commands that attempt to simplify building and deploying elixir applications. The goal of the project is to provide an extensible framework that can support many different deploy scenarios with one common set of commands.
+**Bootleg** is a simple set of commands that attempt to simplify building and deploying Elixir applications. The goal of the project is to provide an extensible framework that can support many different deployment scenarios with one common set of commands.
 
-Out of the box, Bootleg provides remote build and remote server automation for your existing [Distillery](https://github.com/bitwalker/distillery) releases. Bootleg assumes your project is committed into a `git` repository and some of the build steps use this assumption
-to handle code in some steps of the build process. If you are using an scm other than git, please consider contributing to Bootleg to
+Out of the box, Bootleg provides remote build and remote server automation for your [Distillery](https://github.com/bitwalker/distillery) releases. Bootleg assumes your project is committed into a **git** repository and some of the build steps use this assumption
+to handle code within the build process. If you are using another source control management (SCM) tool please consider contributing to Bootleg to
 add additional support.
 
 ## Installation
 
-```elixir
+```
 def deps do
-  [{:distillery, "~> 1.3",
-   {:bootleg, "~> 0.5"}]
+  [{:distillery, "~> 1.5",
+   {:bootleg, "~> 0.6"}]
 end
 ```
 
@@ -24,7 +24,6 @@ end
 In order to build your project, Bootleg requires that your build server be set up to compile
 Elixir code. Make sure you have already installed Elixir on any build server you define.
 
-
 ## Quick Start
 
 ### Initialize your project
@@ -138,6 +137,7 @@ by Bootleg:
   * `password` - ssh password
   * `identity` - unencrypted private key file path (passphrases are not supported at this time)
   * `port` - ssh port (default `22`)
+  * `replace_os_vars` - controls the `REPLACE_OS_VARS` environment variable used by Distillery for release configuration (default `true`)
 
 #### Examples
 
@@ -395,29 +395,50 @@ end
 
 ## Phoenix Support
 
-Bootleg builds elixir apps, if your application has extra steps required make use of the hooks
+If your application has extra steps required, you may make use of the hooks
 system to add additional functionality. A common case is for building assets for Phoenix
-applications. To build phoenix assets during your build, include the additional package
-`bootleg_phoenix` to your `deps` list. This will automatically perform the additional steps required
-for building phoenix releases.
+applications.
+
+### Using the bootleg_phoenix package
+
+To run these steps automatically you may include the additional package
+`bootleg_phoenix` in your `deps` list. This package provides the build hook commands required to build most Phoenix releases.
 
 ```elixir
 # mix.exs
 def deps do
-  [{:distillery, "~> 1.3"},
-  {:bootleg, "~> 0.5"},
-  {:bootleg_phoenix, "~> 0.1"}]
+  [{:distillery, "~> 1.5"},
+  {:bootleg, "~> 0.6"},
+  {:bootleg_phoenix, "~> 0.2"}]
 end
 ```
 
-For more about `bootleg_phoenix` see: https://github.com/labzero/bootleg_phoenix
+See also: [labzero/bootleg_phoenix](https://github.com/labzero/bootleg_phoenix).
 
-## Sharing Tasks
+### Using your own deploy configuration and hooks
 
-Sharing is a good thing. We love to share, especially awesome code we write. Bootleg supports loading
-tasks from packages in a manner very similar to `Mix.Task`. Just define your module under `Bootleg.Tasks`,
-`use Bootleg.Task` and pass it a block of Bootleg DSL. The contents will be discovered and executed
-automatically at launch.
+Similar to how `bootleg_phoenix` is implemented, you can make use of the hooks system to run some commands on the build server around compile time.
+
+```elixir
+task :phoenix_digest do
+  remote :build do
+    "npm install"
+    "./node_modules/brunch/bin/brunch b -p"
+    "MIX_ENV=prod mix phoenix.digest"
+  end
+  UI.info "Phoenix asset digest generated"
+end
+
+after_task :compile, :phoenix_digest
+```
+
+
+## Task Providers
+
+Sharing is a good thing. Bootleg supports loading
+tasks from packages in a manner very similar to `Mix.Task`. 
+
+You can create and share custom tasks by namespacing a module under `Bootleg.Tasks` and passing a block of Bootleg DSL:
 
 ```elixir
 defmodule Bootleg.Tasks.Foo do
@@ -431,30 +452,27 @@ defmodule Bootleg.Tasks.Foo do
 end
 ```
 
-See `Bootleg.Task` for more details.
+See also: [Bootleg.Task](https://hexdocs.pm/bootleg/Bootleg.Task.html#content) for additional examples.
 
 ## Help
 
-If something goes wrong, retry with the `--verbose` option.
 For detailed information about the Bootleg commands and their options, try `mix bootleg help <command>`.
 
+We're usually around on Slack where you can find us on [elixir-lang's #bootleg channel](http://elixir-lang.slack.com/messages/bootleg/) if you have any questions.
+
 -----
 
 ## Acknowledgments
 
 Bootleg makes heavy use of the [bitcrowd/SSHKit.ex](https://github.com/bitcrowd/sshkit.ex)
-library under the hood. We would like to acknowledge the effort from the bitcrowd team that went into
-creating SSHKit.ex as well as for them prioritizing our requests and providing a chance to collaborate
-on ideas for both the SSHKit.ex and Bootleg projects.
+library under the hood. We are very appreciative of the efforts of the bitcrowd team for both creating SSHKit.ex and being so attentive to our requests. We're also grateful for the opportunity to collaborate
+on ideas for both projects!
 
 ## Contributing
 
-We welcome everyone to contribute to Bootleg and help us tackle existing issues!
-
-Use the [issue tracker][issues] for bug reports or feature requests.
-Open a [pull request][pulls] when you are ready to contribute.
+We welcome all contributions to Bootleg, whether they're improving the documentation, implementing features, reporting issues or suggesting new features.
 
-If you are planning to contribute documentation, please check
+If you'd like to contribute documentation, please check
 [the best practices for writing documentation][writing-docs].
 
 

lib/bootleg/config.ex

@@ -8,8 +8,9 @@ defmodule Bootleg.Config do
 
   defmacro __using__(_) do
     quote do
-      import Bootleg.Config, only: [role: 2, role: 3, config: 2, config: 0, before_task: 2,
-        after_task: 2, invoke: 1, task: 2, remote: 1, remote: 2, remote: 3, load: 1, upload: 3]
+      import Bootleg.Config, only: [role: 2, role: 3, config: 2, config: 1, config: 0,
+        before_task: 2, after_task: 2, invoke: 1, task: 2, remote: 1, remote: 2,
+        remote: 3, load: 1, upload: 3, download: 3]
     end
   end
 
@@ -51,7 +52,14 @@ defmodule Bootleg.Config do
       |> Keyword.put(:user, user)
       # identity needs to be present in both options lists
       |> Keyword.put(:identity, ssh_options[:identity])
-      |> Enum.filter(fn {_, v} -> v end)
+      |> Keyword.get_and_update(:identity, fn val ->
+          if val || Keyword.has_key?(ssh_options, :identity) do
+            {val, val || ssh_options[:identity]}
+          else
+            :pop
+          end
+        end)
+      |> elem(1)
 
     quote bind_quoted: binding() do
       hosts =
@@ -94,6 +102,42 @@ defmodule Bootleg.Config do
     end
   end
 
+  @doc """
+  Fetches the value for the supplied key from the Bootleg configuration. If the provided
+  key is a `Tuple`, the first element is considered the key, the second value is considered
+  the default value (and returned without altering the config) in case the key has not
+  been set. This uses the same semantics as `Keyword.get/3`.
+
+  ```
+  use Bootleg.Config
+  config :foo, :bar
+  
+  # local_foo will be :bar
+  local_foo = config :foo
+
+  # local_foo will be :bar still, as :foo already has a value
+  local_foo = config {:foo, :car}
+
+  # local_hello will be :world, as :hello has not been defined yet
+  local_hello = config {:hello, :world}
+
+  config :hello, nil
+  # local_hello will be nil, as :hello has a value of nil now
+  local_hello = config {:hello, :world}
+  ```
+  """
+  defmacro config({key, default}) do
+    quote bind_quoted: binding() do
+      Keyword.get(Bootleg.Config.Agent.get(:config), key, default)
+    end
+  end
+
+  defmacro config(key) do
+    quote bind_quoted: binding() do
+      Keyword.get(Bootleg.Config.Agent.get(:config), key)
+    end
+  end
+
   @doc """
   Sets `key` in the Bootleg configuration to `value`.
 
@@ -106,6 +150,7 @@ defmodule Bootleg.Config do
 
   config :app, :my_cool_app
   config :version, "1.0.0"
+  ```
   """
   defmacro config(key, value) do
     quote bind_quoted: binding() do
@@ -399,10 +444,10 @@ defmodule Bootleg.Config do
   # runs for hosts found in :build first, then for hosts in :app
   remote [:build, :app], do: "hostname"
 
-  # only runs on `host1.example.com`
   role :build, "host2.example.com"
-  role :build, "host1.example.com", filter: [primary: true, another_attr: :cat]
+  role :build, "host1.example.com", primary: true, another_attr: :cat
 
+  # only runs on `host1.example.com`
   remote :build, filter: [primary: true] do
     "hostname"
   end
@@ -491,6 +536,59 @@ defmodule Bootleg.Config do
     end
   end
 
+  @doc """
+  Downloads files from remote hosts to the local machine.
+
+  Downloading works much like `remote/3`, but instead of transferring shell commands over SSH,
+  it transfers files via SCP. The remote host does need to support SCP, which should be provided
+  by most SSH implementations automatically.
+
+  `role` can either be a single role name, a list of roles, or a list of roles and filter
+  attributes. The special `:all` role is also supported. See `remote/3` for details. Note that
+  if multiple hosts match, files will be downloaded from all matching hosts, and any duplicate
+  file names will result in collisions. The exact semantics of how that works are handled by
+  `SSHKit.SCP`, but in general the file transfered last wins.
+
+  `local_path` is a path to local directory or file where the downloaded files(s) should be placed.
+  Absolute paths will be respected, relative paths will be resolved relative to the current working
+  directory of the invoking shell. If the `local_path` does not exist in the local file system, an
+  attempt will be made to create the missing directory. This does not handle nested directories,
+  and a `File.Error` will be raised.
+
+  `remote_path` is the file or directory to be copied from the remote hosts. If a directory is
+  specified, its contents will be recursively copied. Relative paths will be resolved relative to
+  the remote workspace, absolute paths will be respected.
+
+  The files on the local host are created using the current user's `uid`/`gid` and `umask`.
+
+  ```
+  use Bootleg.Config
+
+  # copies ./my_file from the remote host to ./new_name locally
+  download :app, "my_file", "new_name"
+
+  # copies ./my_file from the remote host to the file ./a_dir/my_file locally
+  download :app, "my_file", "a_dir"
+
+  # recursively copies ./some_dir on the remote host to ./new_dir locally, ./new_dir
+  # will be created if missing
+  download :app, "some_dir", "new_dir"
+
+  # copies /foo/my_file on the remote host to /tmp/foo locally
+  download :app, "/foo/my_file", "/tmp/foo"
+  """
+  defmacro download(role, remote_path, local_path) do
+    {roles, filters} = split_roles_and_filters(role)
+    roles = unpack_role(roles)
+    quote bind_quoted: binding() do
+      Enum.each(roles, fn role ->
+        role
+        |> SSH.init([], filters)
+        |> SSH.download(remote_path, local_path)
+      end)
+    end
+  end
+
   @doc false
   @spec get_config(atom, any) :: any
   def get_config(key, default \\ nil) do