Releasing a Debian package

How to release a perfSONAR Debian package?

Here is the documentation of the process to release a new perfSONAR Debian package that correspond to a new perfSONAR RPM package. This process is making use of the conventions described in our Debian packaging guidelines

Releasing an official perfSONAR package for Debian is a two steps process that uses the staging and the stable (or testing) suites defined in our perfSONAR Debian repository.

Building and staging

Building a package from a frozen code source, identified by a git tag, and posting it to the staging repository.

Process

We'll consider that the upstream source of our package is in the master branch and that the Debian package files are in the debian/wheezy branch. We also have a tag on the master branch identifying the upstream release.

1. You need to have your git tree up to date, both in the master branch and in the debian/wheezy branch with no pending changes.
2. The master branch, up to the release tag, is merged into the debian/wheezy branch (if there exist any newer commits after the release tags, they must not be merged now). The merge needs to be cleaned and committed to the debian/wheezy branch.
3. The Debian changelog is updated with:

• the list of the latest changes
• the corresponding Debian package version and release numbers, this needs to comply with rules from Versioning and also needs to have a version-release number that is bigger than the previously released Debian package so that the upgrade happens smoothly on the user side
• an updated distribution, changed from UNRELEASED to perfsonar-wheezy-staging

1. Locally commit your changes to actually freeze the Debian release:

• use a commit message such as: Releasing Debian package x.y-c
• add a git tag corresponding to the Debian version and release numbers and that correctly references the upstream tag (this is important for Jenkins to be able to build the package cleanly, it will use this tag to build the upstream tarball)

1. Make sure everything is building fine under git-buildpackage environment, that the patches are applied cleanly and that the working tree is clean after the patches are reverted (this is part of the gbp build process). If you build with gbp you'll use the same build process as on Jenkins.
2. When you have confirmed the build is good, push your changes and the new tag to the remote repository.
3. Launch a build in Jenkins, from the -source job, using your newly created tag and using the force-upstream flag if you're building for a RC.
4. Once the build finishes, the resulting packages (source and arch) are added to the staging repository and this repository is signed with the perfSONAR Debian Archive Automatic Signing Key. The staging repository is then rsynced to the download servers.

Sample commands

Here is some examples of the usual commands you'll need to use to perform a release. We'll consider the 3.5-1.rc1 release

• Have an up to date repository and merge the 3.5-1.rc1 upstream tag

cd code-repo
git checkout master; git pull
git checkout debian/wheezy; git pull
git merge --no-commit --no-ff 3.5-1.rc1

• Solve any occurring conflict and commit the merge
• Apply all quilt patches and see if it needs updating
  • Correct the patches if need be
  • Add or remove patches if need be
• Update the debian/changelog
  • with the list of latest upstream changes and Debian specific changes
  • with the correct version and release numbers
  • replacing the UNRELEASED distribution with perfsonar-wheezy-staging
  • update your signature with the current timestamp
• Commit your changes with appropriate message (see above) and a tag

git commit -a
git tag debian/wheezy/3.5-1.rc1-1

• Check build with gbp with commands such as (these are the same commands as used by the Jenkins scripts):

tag=debian/wheezy/3.5-1.rc1-1
ntag=${tag##*\/}
git-buildpackage -nc --git-force-create --git-ignore-new --git-ignore-branch -S -us -uc --git-verbose --git-builder=/bin/true --git-cleaner=/bin/true --git-export-dir= --git-upstream-tree=tag --git-upstream-tag=${ntag%-*}
dpkg-buildpackage -uc -us -nc -d -S -i -I --source-option=--unapply-patches

• Publish your changes

git push
git push --tags

• Build the source package in Jenkins and, if all goes well, the resulting packages will get automatically added to the staging repository.

Further development

After creating a release, we need to put the Debian branch back into a situation where the SNAPSHOT builds can be generated automatically.

There are a few things needed for this to happen smoothly:

• the master branch needs to merge fine with the Debian one
• the debian/changelog must be ready for the next Debian package release:
  • a new changelog entry must be done (unless we released an alpha or an RC, because then we'll reuse the same changelog entry for each RC up to the final, adding more elements to the list when needed)
  • the version number need to be incremented by something equal or less than the minimum upstream unit (preparing for the next upstream release):
    • if upstream has already decided on the next version number, use that
    • otherwise add a .0 the current version
    • beware of alpha and RC releases that don't necessarily increase numbers in the same pattern, see Versioning
  • the distribution field must be set to UNRELEASED
  • update your signature with the current timestamp

Official release

Releasing an official package from the staging repository to the testing or stable repository.

Process

We'll consider here releasing a set of newly built bwctl packages from the staging repository to the testing repository. We're using reprepro to manage the repositories and we keep the repository signing key on the developer's machine (this signing key is not on the server).

1. We need to tunnel the GPG agent connection from our development host to the downloads server.
2. We prevent the continuous build server to push anything to the repository while we work on it, otherwise the repository on the continuous build server and the download server will get desynchronised.
3. We copy the packages from the staging repository to the testing repository.
4. reprepro will ask us to sign the new release file.
5. We regenerate the repository content file
6. We tell the continous build server to pull our newly released package and the testing repository content from us before adding any new package to the snapshot or staging repository.

Sample commands

This needs to be run as the jenkins user on the master download server.

• You need to setup GPG agent tunelling, usually with socat. Instructions about that probably depends on your local configuration and is difficult to describe here.
• Then, connect to the download server and follow the process.
• Jenkins cannot push anything to us:

rm /var/www/html/repository/ok-to-push

• Looking for packages and copying to testing, with signing (reprepro doesn't follow convention of copying from source to destination...). The copymatched command will ask you for the GPG key passphrase.

reprepro listmatched perfsonar-wheezy-release bwctl*
reprepro listmatched perfsonar-wheezy-staging bwctl*
reprepro copymatched perfsonar-wheezy-release perfsonar-wheezy-staging bwctl*
reprepro listmatched perfsonar-wheezy-release bwctl*

• Regenerate repository content file

/home/antoine/deb-repo-info.pl -html > /var/www/html/repo.html

• Tell Jenkins to pull content from us before any new build.

touch /var/www/html/repository/collect-from-me