Skip to content
Permalink
Browse files
Significant updates to the release process
Makes lots of updates related to us now being a top-level project, and
no longer in the Incubator.
  • Loading branch information
rdowner committed Jun 29, 2016
1 parent 9f2f0cf commit b08fded4d94389d0b6760591e3966759d3c64b99
Show file tree
Hide file tree
Showing 9 changed files with 104 additions and 156 deletions.
@@ -11,11 +11,17 @@ releases. To allow these example commands to run unmodified, set these environme
# The version currently set on the master branch (BROOKLYN_VERSION_BELOW)
OLD_MASTER_VERSION=0.10.0-SNAPSHOT
# The next version to be set on the master branch
NEW_MASTER_VERSION=0.NNN+1.0-SNAPSHOT
NEW_MASTER_VERSION=0.10.0-SNAPSHOT

# The version we are releasing now.
VERSION_NAME=0.NNN.0
VERSION_NAME=0.9.0

# The release candidate number we are making now.
RC_NUMBER=1

# Modules and submodules - these will come in handy later
SUBMODULES="$( perl -n -e 'if ($_ =~ /path += +(.*)$/) { print $1."\n" }' < .gitmodules )"
MODULES=". ${SUBMODULES}"
{% endhighlight %}

Alternatively, use the command `eval $( ./brooklyn-dist/release/environment.sh )` to set these values automatically.
@@ -11,11 +11,9 @@ children:
- { path: publish-temp.md }
- { path: vote.md }
- { path: fix-release.md }
- { path: vote-ipmc.md }
- { path: publish.md }
- { path: announce.md }
---
0. Review and update these instructions, post-graduation from incubation!
1. [Prerequisites](prerequisites.html) - steps that a new release manager must do (but which only need to be done once)
2. [Set environment variables](environment-variables.html) - many example snippets here use environment variables to
avoid repetition - this page describes what they are
@@ -25,7 +23,5 @@ children:
5. [Publish the release artifacts to the staging area](publish-temp.html)
6. [Vote on the dev@brooklyn list](vote.html)
1. If the vote fails - [fix the release branch](fix-release.html) and resume from step 3
7. [Vote on the general@incubator list](vote-ipmc.html)
1. If the vote fails - [fix the release branch](fix-release.html) and resume from step 3
8. [Publish the release artifacts to the public location](publish.html)
9. [Announce the release](announce.html)
7. [Publish the release artifacts to the public location](publish.html)
8. [Announce the release](announce.html)
@@ -15,26 +15,26 @@ The release script will:
The script has a single required parameter `-r` which is given the release candidate number - so `-r1` will create
release candidate 1 and will name the artifacts appropriately.

Before running this however, it is a good idea to check the `apache-release` profile build is healthy.
This will catch glitches such as PGP or javadoc problems without doing time-consuming uploads:
The script takes a `-n` parameter to work in *dry run* mode; in this mode, the script will NOT upload Maven artifacts
or commit the release to the Subversion repository. This speeds up the process (the Maven deploy in particular slows
down the build) and will catch any problems such as PGP or javadoc problems much sooner.

{% highlight bash %}
mvn clean install -Papache-release
{% endhighlight %}

To run the script:
# A dry run to test everything is OK
./brooklyn-dist/release/make-release-artifacts.sh -r$RC_NUMBER -n

{% highlight bash %}
# The real build, which will publish artifacts
./brooklyn-dist/release/make-release-artifacts.sh -r$RC_NUMBER
{% endhighlight %}

It will show you the release information it has deduced, and ask yes-or-no if it can proceed. Please note that the
script will thoroughly clean the Git workspace of all uncommited and unadded files.

**You really probably want to run this against a secondary checkout.** It will wipe `.project` files and other IDE metadata, and bad things can happen if an IDE tries to write while the script is running. Also as it takes a long time, this means your workspace is not tied up. One quick and easy way to do this is to `git clone` the local directory of your primary checkout to a secondary location.
It will show you the release information it has deduced, and ask yes-or-no if it can proceed. Then you will be prompted
for the passphrase to your GnuPG private key. You should only be asked this question once; the GnuPG agent will cache
the password for the remainder of the build.

A few minutes into the script you will be prompted for the passphrase to your GnuPG private key. You should only be
asked this question once; the GnuPG agent will cache the password for the remainder of the build.
Please note that the script will thoroughly clean the Git workspace of all uncommitted and unadded files **even in dry
run mode**. Therefore **you really want to run this against a secondary checkout.** It will wipe `.project` files and
other IDE metadata, and bad things can happen if an IDE tries to write while the script is running. Consider using the
Vagrant configuration provided.

Please note that uploading to the Nexus staging repository is a slow process. Expect this stage of the build to take
2 hours.
@@ -49,10 +49,3 @@ The release script will:
5. For each of the produced files, produce MD5, SHA1, SHA256 and GnuPG signatures

At the end of the script, it will show you the files it has produced and their location.

Make a signed tag for this release, and then push the tag:

{% highlight bash %}
git tag -s -m "Tag release ${VERSION_NAME} release candidate ${RC_NUMBER}" apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER}
git push apache apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER}
{% endhighlight %}
@@ -36,6 +36,8 @@ The following software packages are required during the build. Make sure you hav

- A Java Development Kit, version 1.7
- `maven` and `git`
- Go Language 1.6 - usually provided by the `golang` package on popular distributions
- The `rpmbuild` command - usually provided by the `rpm` package on popular distributions
- `xmlstarlet` is required by the release script to process version numbers in `pom.xml` files;
on mac, `port install xmlstarlet` should do the trick.
- `zip` and `unzip`
@@ -45,6 +47,7 @@ The following software packages are required during the build. Make sure you hav
- `md5sum` and `sha1sum` - these are often present by default on Linux, but not on Mac;
`port install md5sha1sum` should remedy that.
- if `gpg` does not resolve (it is needed for maven), create an alias or script pointing at `gpg2 "$@"`
- the `mmv` command (usually in a package named `mmv`) will help with the final steps of the release process


GPG keys
@@ -79,16 +82,26 @@ cd apache-dist-release-brooklyn
svn --username $SVN_USERNAME commit -m 'Update brooklyn/KEYS for $GPG_KEY'
{% endhighlight %}

References:
References:

* [Post on the general@incubator list](https://mail-archives.apache.org/mod_mbox/incubator-general/201410.mbox/%3CCAOGo0VawupMYRWJKm%2Bi%2ByMBqDQQtbv-nQkfRud5%2BV9PusZ2wnQ%40mail.gmail.com%3E)
* [GPG cheatsheet](http://irtfweb.ifa.hawaii.edu/~lockhart/gpg/gpg-cs.html)

We recommend the use of the `gpg-agent`, as the release process invokes gpg to sign a large number of artifacts, one at
a time. The agent stores its configuration in `~/.gnupg/gpg-agent.conf`. A sample configuration is shown below; it uses
the Mac OSX `pinentry-mac` program which can be obtained through MacPorts or other sources. For other platforms you will
need to change this; sometimes you can omit it completely and your OS will pick a suitable alternative. The following
two lines cause your passphrase to be cached in memory for a limited period; it will expire from the cache 30 minutes
after it was most recently accessed, or 4 hours after it was first cached.

```
pinentry-program /Applications/MacPorts/pinentry-mac.app/Contents/MacOS/pinentry-mac
default-cache-ttl 1800
max-cache-ttl 14400
```

If you experience trouble with PGP subsequently (when running maven):
* On Mac make sure that `~/.gnupg/gpg-agent.conf` refers to the right file for `pinentry-program`;
it must point at a *binary* and preferably a pop-up which is keychain-aware,
such as with macports `/Applications/MacPorts/pinentry-mac.app/Contents/MacOS/pinentry-mac`
* When the `pinentry` dialog pops up, tick to "Save to keychain", otherwise it will keep popping up
and may time out and fail the build

* See [GnuPG/Pinentry Enigmail debugging](https://www.enigmail.net/support/gnupg2_issues.php) for tips on diagnosing gpg-agent communication (from the process to this agent and from this agent to the pinentry program)
* See [GnuPG Agent Options](https://www.gnupg.org/documentation/manuals/gnupg/Agent-Options.html) for extended gpg-agent debug

@@ -4,6 +4,22 @@ title: Publish to the staging area
navgroup: developers
---

Update the canonical Git repository
-----------------------------------

Make a signed tag for this release candidate:

{% highlight bash %}
for m in ${MODULES}; do ( cd $m && git tag -s -m "Tag release ${VERSION_NAME} release candidate ${RC_NUMBER}" rel/apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER} ); done
{% endhighlight %}

Now push the release branch and release candidate tag:

{% highlight bash %}
for m in ${MODULES}; do ( cd $m && git push apache-git $VERSION_NAME && git push apache-git rel/apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER} ); done
{% endhighlight %}


Publish the source and binary distributions to the pre-release area
-------------------------------------------------------------------

@@ -17,8 +33,8 @@ In your workspace for the `dist.apache.org` repo, create a directory with the ar
mkdir apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER}
{% endhighlight %}

Copy into this directory all of the artifacts from the previous step - `-src` and `-bin`, `.tar.gz` and `.zip`, and all
associated `.md5`, `.sha1`, `.sha256` and `.asc` signatures. Then commit:
Copy into this directory all of the artifacts from the previous step - `-src` and `-bin`, `.tar.gz`, `.zip` and `.rpm`,
and all associated `.md5`, `.sha1`, `.sha256` and `.asc` signatures. Then commit:

{% highlight bash %}
svn add apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER}
@@ -4,6 +4,21 @@ title: Publish to the public
navgroup: developers
---

Update the canonical Git repository
-----------------------------------

Make a signed tag for this release:

{% highlight bash %}
for m in ${MODULES}; do ( cd $m && git tag -s -m "Tag release ${VERSION_NAME}" rel/apache-brooklyn-${VERSION_NAME} rel/apache-brooklyn-${VERSION_NAME}-rc${RC_NUMBER} ); done
{% endhighlight %}

Now push the release tag:

{% highlight bash %}
for m in ${MODULES}; do ( cd $m && git push apache-git rel/apache-brooklyn-${VERSION_NAME} ); done
{% endhighlight %}

Publish the source and binary distributions to the pre-release area
-------------------------------------------------------------------

@@ -40,8 +55,7 @@ Note that the PGP signatures do not embed the filename so they do not need to be
As a final check, re-test the hashes and signatures:

{% highlight bash %}
for ext in -src.tar.gz -src.zip -bin.tar.gz -bin.zip; do
artifact=apache-brooklyn-${VERSION_NAME}${ext}
for artifact in *.tar.gz *.zip *.rpm; do
md5sum -c ${artifact}.md5
shasum -a1 -c ${artifact}.sha1
shasum -a256 -c ${artifact}.sha256
@@ -7,15 +7,13 @@ navgroup: developers
This will allow development to continue on master without affecting the release; it also allows quick-fixes to the
release branch to address last-minute problems (which must of course be merged/cherry-picked back into master later).

Determine the correct name for the version. Note that while in incubation, we must include “incubating” in our release
name - common practice for this is to append “-incubating” to the release version.

Do not use -rc1, -rc2 etc. in version strings. Use the version that will be the actual published version. (The artifacts
that get uploaded to the dist.apache.org/dev will include “-rc1” etc. in the folder name, but the contents will be *as
final*. Therefore, turning the rc into the final is simply a case of taking the rc file and publishing it to the release
folder with the correct name.)

References:

- [Post on general@incubator](https://mail-archives.apache.org/mod_mbox/incubator-general/201409.mbox/%3CCAK2iWdS1H9dkJcSdohky6hFqJdP0XyuhAG%2B%3D1Aspxcjt5RmnJw%40mail.gmail.com%3E)
- [Post on general@incubator](https://mail-archives.apache.org/mod_mbox/incubator-general/201409.mbox/%3CCAOGo0VaEz4cEUbgMgqhh3hiiiubnspiGkQ%3DQv08bOwPqRtzAvQ%40mail.gmail.com%3E)

@@ -28,20 +26,26 @@ Create a branch with the same name as the version, based off master:
{% highlight bash %}
git checkout master
git pull --rebase # assumes that the Apache canonical repository is the default upstream for your master - amend if necessary
git checkout -b $VERSION_NAME
git push -u apache $VERSION_NAME
git submodule update --remote --merge --recursive
for m in $MODULES; do ( cd $m && git checkout master && git checkout -b $VERSION_NAME ); done
{% endhighlight %}

Now change the version numbers in this branch throughout the project using the script `brooklyn-dist/release/change-version.sh` and commit it:

{% highlight bash %}
./brooklyn-dist/release/change-version.sh BROOKLYN $OLD_MASTER_VERSION $VERSION_NAME
git add .
# Now inspect the staged changes and ensure there are no surprises
git commit -m "Change version to $VERSION_NAME"
git push
# Now inspect the changes and ensure there are no surprises
find . -name "*.bak" -delete
for m in $SUBMODULES; do ( cd $m && git add . && git commit -m "Change version to $VERSION_NAME" ); done
git add $SUBMODULES && git commit -m "Update submodules to $VERSION_NAME"
git add . && git commit -m "Change version to $VERSION_NAME"
{% endhighlight %}

If you are happy with the changes, push them:

{% highlight bash %}
for m in $MODULES; do ( cd $m && git push apache-git $VERSION ); done
{% endhighlight %}


Update the version on master
@@ -55,10 +59,9 @@ The release notes should be cleared out and the version history augmented with t
Example:

{% highlight bash %}
git checkout master
for m in $MODULES; do ( cd $m && git checkout master ); done
./brooklyn-dist/release/change-version.sh BROOKLYN $OLD_MASTER_VERSION $NEW_MASTER_VERSION
git add .
# Now inspect the staged changes and ensure there are no surprises
# Now inspect the changes and ensure there are no surprises
{% endhighlight %}

Open `brooklyn-docs/guide/misc/release-notes.md` and `brooklyn-docs/website/meta/versions.md` in your favourite editor and amend.
@@ -68,8 +71,16 @@ and putting some placeholder text elsewhere.
Then:

{% highlight bash %}
git commit -m "Change version to $NEW_MASTER_VERSION"
git push
find . -name "*.bak" -delete
for m in $SUBMODULES; do ( cd $m && git add . && git commit -m "Change version to $NEW_MASTER_VERSION" ); done
git add $SUBMODULES && git commit -m "Update submodules to $NEW_MASTER_VERSION"
git add . && git commit -m "Change version to $NEW_MASTER_VERSION"
{% endhighlight %}

If you are happy with the changes, push them:

{% highlight bash %}
for m in $MODULES; do ( cd $m && git push apache-git master ); done
{% endhighlight %}


@@ -79,5 +90,5 @@ Switch back to the release branch
Move back to the release branch:

{% highlight bash %}
git checkout $VERSION_NAME
for m in $MODULES; do ( cd $m && git checkout $VERSION_NAME ); done
{% endhighlight %}

This file was deleted.

0 comments on commit b08fded

Please sign in to comment.