Skip to content

Commit

Permalink
Resolve merge conflicts between release-0.6.1 and release
Browse files Browse the repository at this point in the history
  • Loading branch information
JONBRWN committed Dec 31, 2018
2 parents 7836542 + 2940c88 commit 23babad
Show file tree
Hide file tree
Showing 314 changed files with 8,113 additions and 6,080 deletions.
4 changes: 2 additions & 2 deletions .for_maintainers/release/AMI_RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ This will `ssh` you into the running Wallaroo box.
This will do the following:

1. Check out Wallaroo at `<BRANCH>`
2. Clean up and then build: `machida`,`cluster_shutdown`,
2. Clean up and then build: `machida3`,`cluster_shutdown`,
`data_receiver`, and `sender`.
3. Launch an EC2 instance of Ubuntu 16.04 and install the
minimum necessary set of packages to run Wallaroo apps
Expand Down Expand Up @@ -94,7 +94,7 @@ The build process comprises the following files:
* [`template.json`](../../.release/ami/template.json): The [Packer template][packer_template] for the AMI
* [`ami/initial_system_setup.sh`](../../.release/ami/initial_system_setup.sh):
This script is run on the EC2 instance. It is responsible for installing the
runtime dependencies for `machida`, and setting up an `/etc/rc.local` script to run
runtime dependencies for `machida3`, and setting up an `/etc/rc.local` script to run
on system startup.
* [`regions`](../../.release/ami/regions): A text file listing the AWS regions
where the public AMI will be available.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,50 +1,50 @@
# Building and pushing Wallaroo Documentation Gitbook
# Building and pushing Wallaroo Documentation

This document is aimed at members of the Wallaroo team who might be building and pushing the Wallaroo Documentation Gitbook for a release candidate, release, or master branch. It serves as a checklist that can take your through the gitbook release process step-by-step.
This document is aimed at members of the Wallaroo team who might be building and pushing the Wallaroo Documentation for a release candidate, release, or master branch. It serves as a checklist that can take your through the documentation release process step-by-step.

To learn more about our release process, see [RELEASE.md](RELEASE.md).

## Prerequisites for building and pushing Wallaroo Documentation Gitbook
## Prerequisites for building and pushing Wallaroo Documentation

In order to build and push the Wallaroo Documentation Gitbook, you absolutely must have:
In order to build and push the Wallaroo Documentation, you absolutely must have:

* Provisioned the Wallaroo Release Vagrant box as described in [PROVISION_VAGRANT.md](PROVISION_VAGRANT.md) and ssh'ed into it.

## Building and pushing Wallaroo Documentation Gitbook
## Building and pushing Wallaroo Documentation

Please note that this document was written with the assumption that you are using a clone of the `wallaroolabs/wallaroo` repo. This process will not work without modification if you try to use a fork rather than a clone of the repo. The `github-release.sh` script assumes you are using the `release` branch, `master` branch, or a release candidate branch that follows the `release-*` format.

### Pull latest changes for your branch

From within the Wallaroo Vagrant box, you'll want to run a `git pull` for the branch you plan to use to build and release the Wallaroo Documentation Gitbook like so:
From within the Wallaroo Vagrant box, you'll want to run a `git pull` for the branch you plan to use to build and release the Wallaroo Documentation like so:

```bash
cd ~/wallaroo
git checkout origin/RELEASE_BRANCH
git pull
```

So if you were going to release the Wallaroo Documentation Gitbook using `release-0.4.0`, you'd run the following:
So if you were going to release the Wallaroo Documentation using `release-0.4.0`, you'd run the following:

```bash
cd ~/wallaroo
git checkout origin/release-0.4.0
git pull
```

### Building and pushing the Wallaroo Documentation Gitbook
### Building and pushing the Wallaroo Documentation

From within the Wallaroo Vagrant box run the following:

```bash
cd /users/ubuntu/wallaroo
bash .release/gitbook-release.sh VERSION COMMIT
bash .release/documentation-release.sh VERSION COMMIT
```

So, for example, if your version is release-0.4.0 and your commit is `0xa0ece`, you'd run:

```bash
bash .release/gitbook-release.sh release-0.4.0 0xa0ece
bash .release/documentation-release.sh release-0.4.0 0xa0ece
```

This will then build and push the Wallaroo Documentation Gitbook to the `wallaroolabs/docs.wallaroolabs.com` Github repository.
This will then build and push the Wallaroo Documentation to the `wallaroolabs/docs.wallaroolabs.com` Github repository.
4 changes: 2 additions & 2 deletions .for_maintainers/release/RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,14 @@ Our release process features a few primary steps
* Building and releasing RC artifacts including:
* Building and releasing the Wallaroo RC source archive and Metrics UI RC AppImage to Bintray
* Building and releasing the Wallaroo and Metrics UI RC Docker images on Bintray
* Building and pushing RC Documentation Gitbook
* Building and pushing RC Documentation
* Testing of the release candidate
* Promoting of a release candidate to a release
* Building and releasing artifacts including:
* Building and releasing the Wallaroo source archive and Metrics UI AppImage to Bintray
* Building and releasing the Wallaroo and Metrics UI Docker images on Bintray
* Building and publishing a publicly accesible AMI
* Building and pushing Documentation Gitbook
* Building and pushing Documentation

* Post release process

Expand Down
164 changes: 6 additions & 158 deletions .for_maintainers/release/RELEASE_ISSUE_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,15 +34,15 @@ Any issues that arise during testing should be created using the following forma

##### Docker Installation Instructions

Follow the instructions for [setting up and installing Wallaroo](https://wallaroo-docs-rc.netlify.com/book/getting-started/docker-setup.html).
Follow the instructions for [setting up and installing Wallaroo](https://wallaroo-docs-rc.netlify.com/python-installation/python-docker-installation-guide/).

- [ ] MacOS -
- [ ] Windows -
- [ ] Linux -

##### Docker "Run a Wallaroo Application" Instructions

Follow the instructions for [Run a Wallaroo Application in Docker](https://wallaroo-docs-rc.netlify.com/book/getting-started/run-a-wallaroo-application-docker.html).
Follow the instructions for [Run a Wallaroo Application in Docker](https://wallaroo-docs-rc.netlify.com/python-installation/python-docker-installation-guide/#validate-your-installation).

- [ ] MacOS -
- [ ] Windows -
Expand All @@ -60,7 +60,7 @@ Run through all the Python examples and make sure they work. This means everythi

##### Installing From Wallaroo Up for Python Instructions

Follow the instructions for [setting up and installing Wallaroo](https://wallaroo-docs-rc.netlify.com/book/getting-started/wallaroo-up.html).
Follow the instructions for [setting up and installing Wallaroo](https://wallaroo-docs-rc.netlify.com/python-installation/python-wallaroo-up-installation-guide/).

- [ ] Ubuntu Xenial -
- [ ] Ubuntu Trusty -
Expand All @@ -71,7 +71,7 @@ Follow the instructions for [setting up and installing Wallaroo](https://wallaro

##### Python From Wallaroo Up "Run a Wallaoo Application" Instructions

Follow the instructions for the Python From Wallaroo Up [Run a Wallaroo Application](https://wallaroo-docs-rc.netlify.com/book/getting-started/run-a-wallaroo-application-wallaroo-up.html).
Follow the instructions for the Python From Wallaroo Up [Run a Wallaroo Application](https://wallaroo-docs-rc.netlify.com/python-installation/python-wallaroo-up-installation-guide/#validate-your-installation).

- [ ] Ubuntu Xenial -
- [ ] Ubuntu Trusty -
Expand All @@ -95,49 +95,19 @@ Making Sure it Works:
- [ ] Debian Stretch -
- [ ] CentOS 7 -

#### Python From Source

##### Installing From Source for Python Instructions

Follow the instructions for [setting up and installing Wallaroo](https://wallaroo-docs-rc.netlify.com/book/getting-started/linux-setup.html).

- [ ] Xenial -
- [ ] Trusty -
- [ ] Bionic -

##### Python From Source "Run a Wallaoo Application" Instructions

Follow the instructions for the Python From Source [Run a Wallaroo Application](https://wallaroo-docs-rc.netlify.com/book/getting-started/run-a-wallaroo-application.html).

- [ ] Xenial -
- [ ] Trusty -
- [ ] Bionic -

##### From Source Python Examples

Run through all the Python examples and make sure they work. This means everything in `examples/python`. You can run the automated version using `<wallaroo-install-path>/misc/example-tester.sh python`. This will run all examples except the kafka ones.

Making Sure it Works:
- Building and running all `examples/python` applications
- verifying all applications build and run as expected

- [ ] Xenial -
- [ ] Trusty -
- [ ] Bionic -

#### Vagrant for Python

##### Vagrant Installation Instructions

Follow the instructions for [setting up and installing Wallaroo](https://wallaroo-docs-rc.netlify.com/book/getting-started/vagrant-setup.html).
Follow the instructions for [setting up and installing Wallaroo](https://wallaroo-docs-rc.netlify.com/python-installation/python-vagrant-installation-guide/).

- [ ] MacOS -
- [ ] Windows -
- [ ] Linux -

##### Vagrant "Run a Wallaroo Application" Instructions

Follow the instructions for [Run a Wallaroo Application in Vagrant](https://wallaroo-docs-rc.netlify.com/book/getting-started/run-a-wallaroo-application-vagrant.html).
Follow the instructions for [Run a Wallaroo Application in Vagrant](https://wallaroo-docs-rc.netlify.com/python-installation/python-vagrant-installation-guide/#validate-your-installation).

- [ ] MacOS -
- [ ] Windows -
Expand All @@ -151,128 +121,6 @@ Run through all the Python examples and make sure they work. This means everythi
- [ ] Windows -
- [ ] Linux -

#### Docker for Go

##### Docker Installation Instructions

Follow the instructions for [setting up and installing Wallaroo](https://wallaroo-docs-rc.netlify.com/book/go/getting-started/docker-setup.html).

- [ ] MacOS -
- [ ] Windows -
- [ ] Linux -

##### Docker "Run a Wallaroo Application" Instructions

Follow the instructions for [Run a Wallaroo Go Application in Docker](https://wallaroo-docs-rc.netlify.com/book/go/getting-started/run-a-wallaroo-go-application-docker.html).

- [ ] MacOS -
- [ ] Windows -
- [ ] Linux -

##### Docker Go Examples

Run through all the Go examples and make sure they work. This means everything in `examples/go`. You can run the automated version using `<wallaroo-install-path>/misc/example-tester.sh go`. This will run all examples except the kafka ones.

- [ ] MacOS -
- [ ] Windows -
- [ ] Linux -

#### Go From Wallaroo Up

##### Installing From Wallaroo Up for Go Instructions

Follow the instructions for [setting up and installing Wallaroo](https://wallaroo-docs-rc.netlify.com/book/go/getting-started/wallaroo-up.html).

- [ ] Ubuntu Xenial -
- [ ] Ubuntu Trusty -
- [ ] Ubuntu Bionic -
- [ ] Fedora 28 -
- [ ] Debian Stretch -
- [ ] CentOS 7 -

##### Go From Wallaroo Up "Run a Wallaoo Application" Instructions

Follow the instructions for the Go From Wallaroo Up [Run a Wallaroo Application](https://wallaroo-docs-rc.netlify.com/book/go/getting-started/run-a-wallaroo-go-application-wallaroo-up.html).

- [ ] Ubuntu Xenial -
- [ ] Ubuntu Trusty -
- [ ] Ubuntu Bionic -
- [ ] Fedora 28 -
- [ ] Debian Stretch -
- [ ] CentOS 7 -

##### From Wallaroo Up Go Examples

Run through all the Go examples and make sure they work. This means everything in `examples/go`. You can run the automated version using `<wallaroo-install-path>/misc/example-tester.sh go`. This will run all examples except the kafka ones.

Making Sure it Works:
- Building and running all `examples/go` applications
- verifying all applications build and run as expected

- [ ] Ubuntu Xenial -
- [ ] Ubuntu Trusty -
- [ ] Ubuntu Bionic -
- [ ] Fedora 28 -
- [ ] Debian Stretch -
- [ ] CentOS 7 -

#### Go From Source

##### Installing From Source for Go Instructions

Follow the instructions for [setting up and installing Wallaroo](https://wallaroo-docs-rc.netlify.com/book/go/getting-started/linux-setup.html).

- [ ] Xenial -
- [ ] Trusty -
- [ ] Bionic -

##### Go "Run a Wallaoo Application" Instructions

Follow the instructions for [Run a Wallaroo Go Application](https://wallaroo-docs-rc.netlify.com/book/go/getting-started/run-a-wallaroo-go-application.html).

- [ ] Xenial -
- [ ] Trusty -
- [ ] Bionic -

##### From Source Go Examples

Run through all the Go examples and make sure they work. This means everything in `examples/go`. You can run the automated version using `<wallaroo-install-path>/misc/example-tester.sh go`. This will run all examples except the kafka ones.

Making Sure it Works:
- Building and running all `examples/go` applications
- verifying all applications build and run as expected

- [ ] Xenial -
- [ ] Trusty -
- [ ] Bionic -

#### Vagrant for Go

##### Vagrant Installation Instructions

Follow the instructions for [setting up and installing Wallaroo](https://wallaroo-docs-rc.netlify.com/book/go/getting-started/vagrant-setup.html).

- [ ] MacOS -
- [ ] Windows -
- [ ] Linux -

##### Vagrant "Run a Wallaroo Application" Instructions

Follow the instructions for [Run a Wallaroo Go Application in Vagrant](https://wallaroo-docs-rc.netlify.com/book/go/getting-started/run-a-wallaroo-go-application-vagrant.html).

- [ ] MacOS -
- [ ] Windows -
- [ ] Linux -

##### Vagrant Go Examples

Run through all the Go examples and make sure they work. This means everything in `examples/go`. You can run the automated version using `<wallaroo-install-path>/misc/example-tester.sh go`. This will run all examples except the kafka ones.

- [ ] MacOS -
- [ ] Windows -
- [ ] Linux -


## Phase 2

Review all issues that arise from Phase 1. Address issues that we feel are worth including or can be included to meet release target date. Dependent on the issues addressed, another round of testing may be needed.
Expand Down
4 changes: 2 additions & 2 deletions .for_maintainers/release/RELEASE_NOTES_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Overview of release and what is included.
In all cases below, if you run into issues, please reach out to us! We’re available on [twitter](https://twitter.com/wallaroolabs), [IRC](https://webchat.freenode.net/?channels=#wallaroo), [Github](https://github.com/WallarooLabs/wallaroo), [by email](mailto:hello@wallaroolabs.com), or [our mailing list](https://groups.io/g/wallaroo).
We love questions!

**If you have made no changes to Wallaroo or Pony since installation, your best bet will be to start from scratch, following the [instructions](https://docs.wallaroolabs.com/book/getting-started/choosing-an-installation-option.html) of your choice.**
**If you have made no changes to Wallaroo or Pony since installation, your best bet will be to start from scratch, following the [instructions](https://docs.wallaroolabs.com/python-installation/) of your choice.**

Below are instructions for [Upgrading Wallaroo when compiled from source](#upgrading-wallaroo-when-compiled-from-source), [Upgrading Wallaroo in Docker](#upgrading-wallaroo-in-docker), [Upgrading Wallaroo in Vagrant](#upgrading-wallaroo-in-vagrant), and [Upgrading Wallaroo via Wallaroo Up](#upgrading-wallaroo-up).

Expand Down Expand Up @@ -125,7 +125,7 @@ mkdir c:\wallaroo-docker\wallaroo-src
<a name="upgrading-wallaroo-in-vagrant"></a>
### Upgrading Wallaroo in Vagrant

To upgrade your Wallaroo installation in Vagrant, you’ll want to follow the latest installation instructions for [Wallaroo in Vagrant](https://docs.wallaroolabs.com/book/getting-started/vagrant-setup.html).
To upgrade your Wallaroo installation in Vagrant, you’ll want to follow the latest installation instructions for [Wallaroo in Vagrant](https://docs.wallaroolabs.com/python-installation/python-vagrant-installation-guide/).

Finally, to provision your new Vagrant box, run the following commands:

Expand Down
2 changes: 1 addition & 1 deletion .for_maintainers/release/UPDATE_RELEASE_ARTIFACTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,4 +64,4 @@ This will then ensure the following are built and released:

* the Wallaroo RC/release source archive and Metrics UI RC AppImage to Bintray (documented in [DOCKER_RELEASE.md](DOCKER_RELEASE.md)).
* the Wallaroo and Metrics UI RC/release Docker images on Bintray ([BINTRAY_ARTIFACTS_RELEASE.md](BINTRAY_ARTIFACTS_RELEASE.md)).
* the RC/release Documentation Gitbook ([GITBOOK_RELEASE.md](GITBOOK_RELEASE.md)).
* the RC/release Documentation ([DOCUMENTATION_RELEASE.md](DOCUMENTATION_RELEASE.md)).
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "documentation/themes/ananke"]
path = documentation/themes/ananke
url = https://github.com/budparr/gohugo-theme-ananke.git
12 changes: 6 additions & 6 deletions .release/ami-release.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,18 +32,18 @@ compile_for_sandy_bridge() {
git checkout "$RC_BRANCH_NAME"
make clean

make build-machida "$PONYC_OPTS" resilience=on
mv machida/build/machida machida-resilience
make clean-machida
make build-machida "$PONYC_OPTS" resilience=off
make build-machida3 "$PONYC_OPTS" resilience=on
mv machida3/build/machida3 machida3-resilience
make clean-machida3
make build-machida3 "$PONYC_OPTS" resilience=off
(cd utils/data_receiver && make "$PONYC_OPTS")
(cd utils/cluster_shutdown && make "$PONYC_OPTS")
(cd utils/cluster_shrinker && make "$PONYC_OPTS")
(cd giles/sender && make "$PONYC_OPTS")

zip -j9 .release/wallaroo_bin.zip \
machida/build/machida \
machida-resilience \
machida3/build/machida3 \
machida3-resilience \
utils/data_receiver/data_receiver \
utils/cluster_shutdown/cluster_shutdown \
utils/cluster_shrinker/cluster_shrinker \
Expand Down
11 changes: 9 additions & 2 deletions .release/ami/initial_system_setup.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,15 @@
#!/bin/sh
set -x #echo on
export DEBIAN_FRONTEND=noninteractive
export TERM=linux

# grub-pc currently causes apt-get upgrade to pause indefinitely
# holding so it does not get upgraded
sudo apt-mark hold grub-pc
# Install common tools
sudo apt-get update && sudo apt-get -y upgrade
sudo apt-get install -y python-dev build-essential python-pip unzip zip curl \
sudo apt-get -y update
sudo apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" upgrade
sudo apt-get install -y python3-dev build-essential python3-pip unzip zip curl \
libsnappy-dev liblz4-dev libz-dev libssl-dev \
cpuset numactl
sudo -H python -m pip install pytest==3.2.2
Expand Down
3 changes: 1 addition & 2 deletions .release/ami/regions
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
ap-northeast-1
ap-northeast-2
ap-south-1
ap-southeast-1
ap-southeast-2
ca-central-1
Expand All @@ -12,4 +11,4 @@ sa-east-1
us-east-1
us-east-2
us-west-1
us-west-2
us-west-2

0 comments on commit 23babad

Please sign in to comment.