Skip to content
Permalink
Browse files
Add release scripts (#290)
* Add release scripts

fix #287

* dev/release/release_rc.sh: This prepares RC related artifacts and upload
  them to https://dist.apache.org/repos/dist/dev/arrow .
* dev/release/release.sh: This publishes voted RC to
  https://dist.apache.org/repos/dist/release/arrow .
* dev/release/README.md: This describes how to use the above scripts.
* .github/*: Add missing license headers.

They are confirmed as much as possible on local but some processes
can't be confirmed on local. There may be still some problems but they
will be fixed in the next release process.

* Describe how to use JuliaTagBot

Co-authored-by: Jacob Quinn <quinn.jacobd@gmail.com>

* Fix a typo

* Fix form

* Fix wrong author

Co-authored-by: Eric Hanson <5846501+ericphanson@users.noreply.github.com>

Co-authored-by: Jacob Quinn <quinn.jacobd@gmail.com>
Co-authored-by: Eric Hanson <5846501+ericphanson@users.noreply.github.com>
  • Loading branch information
3 people committed Feb 25, 2022
1 parent 84b959f commit 31dc7079a7d86d76f2e29eaf5c5520b3231c2e6d
Showing 7 changed files with 304 additions and 1 deletion.
@@ -1,3 +1,20 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

name: TagBot
on:
issue_comment:
@@ -1,3 +1,20 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

name: CI
on:
pull_request:
@@ -16,5 +16,6 @@
# under the License.

/apache-rat-*.jar
/dist/
/filtered_rat.txt
/rat.xml
@@ -0,0 +1,98 @@
<!---
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# Release

## Overview

1. Test the revision to be released
2. Increment version number in `Project.toml`
3. Prepare RC and vote (detailed later)
4. Publish (detailed later)

### Prepare RC and vote

Run `dev/release/release_rc.sh` on working copy of `git@github.com:apache/arrow-julia` not your fork:

```console
$ git clone git@github.com:apache/arrow-julia.git
$ dev/release/release_rc.sh ${RC}
(Send a vote email to dev@arrow.apache.org.
You can use a draft shown by release_rc.sh for the email.)
```

Here is an example to release RC1:

```console
$ dev/release/release_rc.sh 1
```

The argument of `release_rc.sh` is the RC number. If RC1 has a problem, we'll increment the RC number such as RC2, RC3 and so on.

Requirements to run `release_rc.sh`:

* You must be an Apache Arrow committer or PMC member
* You must prepare your PGP key for signing

If you don't have a PGP key, https://infra.apache.org/release-signing.html#generate may be helpful.

Your PGP key must be registered to the followings:

* https://dist.apache.org/repos/dist/dev/arrow/KEYS
* https://dist.apache.org/repos/dist/release/arrow/KEYS

See the header comment of them how to add a PGP key.

Apache arrow committers can update them by Subversion client with their ASF account. e.g.:

```console
$ svn co https://dist.apache.org/repos/dist/dev/arrow
$ cd arrow
$ editor KEYS
$ svn ci KEYS
```

### Publish

We need to do the followings to publish a new release:

* Publish to apache.org
* Publish to the Julia General registry

Run `dev/release/release.sh` to publish to apache.org:

```console
$ dev/release/release.sh ${VERSION} ${RC}
```

Here is an example to release 2.2.1 RC1:

```console
$ dev/release/release.sh 2.2.1 1
```

To publish the release to the Julia General registry, navigate to the GitHub commit where the project version was incremented in the Project.toml file (step 2 above), then post a comment on the commit with the following:

`@JuliaRegistrator register()`

JuliaRegistrator will respond saying it has opened a pull request to the General registry and under normal circumstances, will be merged automatically.

### Verify

TODO
@@ -0,0 +1,40 @@
#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

set -eu

if [ "$#" -ne 2 ]; then
echo "Usage: $0 <version> <rc>"
echo " e.g.: $0 2.2.1 1"
exit 1
fi

version=$1
rc=$2

id="apache-arrow-julia-${version}"
echo "Copying dev/ to release/"
svn \
cp \
-m "Apache Arrow Julia ${version}" \
https://dist.apache.org/repos/dist/dev/arrow/${id}-rc${rc} \
https://dist.apache.org/repos/dist/release/arrow/${id}

echo "Success! The release is available here:"
echo " https://dist.apache.org/repos/dist/release/arrow/${id}"
@@ -0,0 +1,130 @@
#!/bin/bash
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

set -eu

SOURCE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SOURCE_TOP_DIR="$(cd "${SOURCE_DIR}/../../" && pwd)"

if [ "$#" -ne 1 ]; then
echo "Usage: $0 <rc>"
echo " e.g.: $0 1"
exit 1
fi

rc=$1

cd "${SOURCE_TOP_DIR}"

git_origin_url="$(git remote get-url origin)"
if [ "${git_origin_url}" != "git@github.com:apache/arrow-julia.git" ]; then
echo "This script must be ran with working copy of apache/arrow-julia."
echo "The origin's URL: ${git_origin_url}"
exit 1
fi

echo "Ensure using the latest commit"
git checkout main
git pull --rebase --prune

version=$(grep -o '^version = ".*"' "Project.toml" | \
sed -e 's/^version = "//' \
-e 's/"$//')

rc_tag="v${version}-rc${rc}"
echo "Tagging for RC: ${rc_tag}"
git tag -a -m "${version} RC${rc}" "${rc_tag}"
git push origin "${rc_tag}"

rc_hash="$(git rev-list --max-count=1 "${rc_tag}")"

id="apache-arrow-julia-${version}"
rc_id="${id}-rc${rc}"
dev_dist_url="https://dist.apache.org/repos/dist/dev/arrow"
dev_dist_dir="dev/release/dist"
tar_gz="${id}.tar.gz"
tar_gz_path="${dev_dist_dir}/${rc_id}/${tar_gz}"
rc_url="${dev_dist_url}/${rc_id}/"

echo "Checking out ${dev_dist_url}"
rm -rf "${dev_dist_dir}"
svn co --depth=empty "${dev_dist_url}" "${dev_dist_dir}"

echo "Attempting to create ${tar_gz} from tag ${rc_tag}"
mkdir -p "$(dirname "${tar_gz_path}")"
git archive "${rc_hash}" --prefix "${id}/" --output "${tar_gz_path}"

pushd "${dev_dist_dir}/${rc_id}"

echo "Running Rat license checker on ${tar_gz}"
../../run_rat.sh ${tar_gz}

echo "Signing tar.gz and creating checksums"
gpg --armor --output ${tar_gz}.asc --detach-sig ${tar_gz}

if type shasum >/dev/null 2>&1; then
sha256_generate="shasum -a 256"
sha512_generate="shasum -a 512"
else
sha256_generate="sha256sum"
sha512_generate="sha512sum"
fi
${sha256_generate} ${tar_gz} > ${tar_gz}.sha256
${sha512_generate} ${tar_gz} > ${tar_gz}.sha512

echo "Uploading to ${rc_url}"
svn add .
svn ci -m "Apache Arrow Julia ${version} ${rc}"

popd

echo "Removing temporary directory"
rm -rf "${dev_dist_dir}"

echo "Draft email for dev@arrow.apache.org mailing list"
echo ""
echo "---------------------------------------------------------"
cat <<MAIL
To: dev@arrow.apache.org
Subject: [VOTE][Julia] Release Apache Arrow Julia ${version} RC${rc}
Hi,
I would like to propose the following release candidate (RC${rc}) of
Apache Arrow Julia version ${version}.
This release candidate is based on commit:
${rc_hash} [1]
The source release rc${rc} is hosted at [2].
Please download, verify checksums and signatures, run the unit tests,
and vote on the release. See [3] for how to validate a release candidate.
The vote will be open for at least 24 hours.
[ ] +1 Release this as Apache Arrow Julia ${version}
[ ] +0
[ ] -1 Do not release this as Apache Arrow Julia ${version} because...
[1]: https://github.com/apache/arrow-julia/tree/${rc_hash}
[2]: ${rc_url}
[3]: https://github.com/apache/arrow-julia/blob/main/dev/release/README.md#verify
MAIL
echo "---------------------------------------------------------"
@@ -19,7 +19,7 @@

set -eu

RELEASE_DIR=$(cd "$(dirname "$BASH_SOURCE")"; pwd)
RELEASE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

RAT_VERSION=0.13

0 comments on commit 31dc707

Please sign in to comment.