Ansible playbooks to install common platforms and tools (e.g. JVM, Ruby, Postgres etc.)
Clone or download
Permalink
Failed to load latest commit information.
group_vars EC2 group_vars only relate to neo4j-demo playbook Mar 23, 2017
roles Add Java 10 and remove old unsupported versions (#128) Oct 2, 2018
LICENSE-2.0.txt Added Apache License 2.0 Jan 14, 2014
README.md Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
android.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
aws-cli-and-psql.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
aws-tools.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
boto.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
boto3.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
consul-0.7.5.yml Update Consul version to 0.7.5 (closes #118) Mar 23, 2017
consul-0.8.3.yml Consul 0.8.3 May 23, 2017
consul.yml Update Consul version to 0.7.5 (closes #118) Mar 23, 2017
docker.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
elasticsearch-reindex.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
elasticsearch.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
gatling.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
gcloud.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
golang-1.10.2.yml add support for golang 1.10.2 Sep 3, 2018
golang-1.8.3.yml Added support for GoLang version 1.8.3 Aug 21, 2017
golang-1.8.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
golang-1.9.2.yml Added latest Golang Dec 24, 2017
golang.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
gradle.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
gradle2.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
grunt.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
gulp.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
invoke.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
java10.yml Add Java 10 and remove old unsupported versions (#128) Oct 2, 2018
java8.yml Add Java 10 and remove old unsupported versions (#128) Oct 2, 2018
jq.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
leiningen.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
lzo.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
mountebank.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
neo4j-demo.yml EC2 group_vars only relate to neo4j-demo playbook Mar 23, 2017
neo4j.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
ngrok.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
nodejs.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
oh-my-zsh.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
packer.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
postgres-client.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
postgres.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
pygerduty.yml Added Pygerduty Install Playbook Apr 30, 2015
python-data-science.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
python-filechunkio.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
python-pyenv.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
python-sphinx.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
python-virtualenv.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
r-studio-server.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
ruby-rvm-new.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
ruby-rvm.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
rust.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
sbt-13.8.yml Update SBT playbook to use version Mar 23, 2017
sbt.yml Update SBT playbook to use version Mar 23, 2017
scala-2.10.5.yml Update Scala playbook to use version Mar 23, 2017
scala-2.11.6.yml Update Scala playbook to use version Mar 23, 2017
scala.yml Update Scala playbook to use version Mar 23, 2017
snowplow-batch-pipeline.yml Add Java 10 and remove old unsupported versions (#128) Oct 2, 2018
snowplow-javascript-tracker.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
snowplow-python-tracker.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
snowplow-realtime-pipeline.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
snowplow-website-new.yml Tweaks Jul 26, 2017
snowplow-website.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
spark.yml Add Java 10 and remove old unsupported versions (#128) Oct 2, 2018
thrift-0.10.0.yml Update Thrift to v0.10.0 (closes #111) Mar 23, 2017
thrift.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
typescript.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017
wine.yml Cleanup formatting in playbooks, roles and README (closes #119) Mar 23, 2017

README.md

Snowplow Ansible Playbooks

Ansible playbooks to install common tools, including:

  • The JVM ecosystem
  • Ruby
  • Postgres

Currently these are all used to configure Vagrant VMs (specifically dev-environment) to hack on specific parts of the Snowplow stack. Going forwards, we plan to extend the playbooks to also manage server environments at Snowplow Analytics.

We have tried to write each playbook in a generic way, so that this may be a useful resource if you want to set up similar development environments that are not related to Snowplow specifically.

Repo structure

Plays to install individual bundles of software e.g. Ruby / RVM, Java, Scala, SBT, Thrift, NodeJS etc. have each been bundled into separate roles.

Those roles are then combined into larger playbooks that are saved in the project root directory. For example, the snowplow-batch-pipeline.yml runs the base, Java, Scala, SBT and Ruby / RVM roles required to enable development on the Snowplow Hadoop-based data pipeline.

This structure makes it simple to compose new playbooks out of the roles. For example, to create a playbook that installs Ruby / RVM and PostgreSQL, we'd create a new playbook file e.g. ruby-postgres.yml with the following contents:

---
- hosts: vagrant
  remote_user: vagrant
  roles:
    - base
    - ruby-rvm
    - postgres

Running the playbooks to install packages on local machine (e.g. Vagrant VM)

Once you've SSHed into Vagrant,

ansible-playbook boto.yml -i /vagrant/home/ansible/ansible_hosts --connection=local

Note you can update the reference to the hosts file (/vagrant/home/ansible/ansible_hosts) to point at any file on the VM with the following contents:

[vagrant]
127.0.0.1:2222

Installing specific versions of different packages e.g. Postgres

We are moving to making it possible to decide, at run time, what version of each package you wish to install. This has been implemented with the Postgres playbook, for example.

By default, running the Postgres playbook will install version 8.4. (That default is specified in ansible-playbooks/roles/postgres/defaults/main.yml).

If instead you'd like to install Postgres 9.3, simply execute:

ansible-playbook /vagrant/ansible-playbooks/postgres.yml -i /vagrant/home/ansible/ansible_hosts --connection=local --extra-vars "postgres_version=9.3"

For more information on roles in Ansible playbooks, consult the Ansible documentation

Running the playbooks against a remote server (e.g. on EC2)

To use the different roles in this repo to install the applications on a remote server e.g. on EC2:

  1. Update your hosts / inventory file with details of the server you wish to install the apps on
  2. Create a new playbook pointing at the appropriate host in your inventory file, and set to use the appropriate user
  3. Run it! Note that in general, we add the ---ask-sudo-pass flag,

For example, to install any of the applications to a box called 'snowplow-external-server-1', make sure that the ~/.ssh/config file contains something like the following:

...
Host snowplow-external-server-1
  Hostname ec2-11-222-333-4444.compute-1.amazonaws.com
  User username_with_sudo_permissions
  IdentityFile "~/.ssh/id_rsa"
  Port 22
  ForwardAgent yes
 ...

Then Update the hosts file:

[snowplow-external-server-1]
snowplow-external-server-1

Then create a playbook like the following:

- hosts: snowplow-external-server-1
  remote_user: username_with_sudo_permissions
  
  roles:
   - role: oracle-java 

A note on RStudio setup

RStudio uses PAM to do user management. Once you've run the RStudio Server playbook, create a new user to log in with:

$ sudo adduser rstudio

Assign that user a password. On your host machine, you'll then be able to log in (e.g. to localhost:8788) with the credentials you just created on the Ubuntu guest.

Copyright and license

Snowplow Ansible Playbooks is copyright 2014-2017 Snowplow Analytics Ltd.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this software except in compliance with the License.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.