Build: Publish SNAPSHOT releases

[nastra]

The idea is that we do x.y.z-SNAPSHOT releases whenever something is pushed to master. This requires that we have NEXUS_USER / NEXUS_PW secrets stored.

I also created https://issues.apache.org/jira/browse/INFRA-22437 in order to have a general Maven user/password for publishing the snapshots.

[kbendick]

The idea is that we do x.y.z-SNAPSHOT releases whenever something is pushed to master. This requires that we have MAVEN_USER / MAVEN_PASSWORD secrets stored.

Thinking on this further, is it correct to say that a nightly build from the master branch is considered a 0.12.1 SNAPSHOT? For the patch release, I had expected we might cherry-pick certain commits and apply them on top of the 0.12.0 release (though I’m not entirely sure if that’s the case or not so please don’t quote me on that - just an assumption on my part).

If we don’t always build the next release (specifically point releases) from master, this might confuse some users, particularly those users who integrate Iceberg into their company’s systems and might try out SNAPSHOT releases before a RC is published.

I definitely like the idea of publishing SNAPSHOT, but wondering if this will potentially be an issue.

[nastra]

Thinking on this further, is it correct to say that a nightly build from the master branch is considered a 0.12.1 SNAPSHOT? For the patch release, I had expected we might cherry-pick certain commits and apply them on top of the 0.12.0 release (though I’m not entirely sure if that’s the case or not so please don’t quote me on that - just an assumption on my part).

Going forward, we would probably have to maintain the current development version in a version.txt file. This is bumped to the next version whenever a release was done. So after a 0.12.1 release we would bump the version in version.txt to either 0.12.2 or 0.13.0 (whatever the next version would be). Therefore, we would always publish snapshots with the current development version

If we don’t always build the next release (specifically point releases) from master, this might confuse some users, particularly those users who integrate Iceberg into their company’s systems and might try out SNAPSHOT releases before a RC is published.

I'm not entirely sure I can follow here. The latest SNAPSHOT release is whatever is there on master so people have a chance of trying to prepare integration much earlier into their systems. I think this is a big plus and we already had quite a lot occassions where we wished that there would be snapshots of the current version that's being developed :)

[nastra]

@rdblue suggested that we try and extract the version info from existing tags.
I did some experimenting with the existing and other gradle git plugins. Most plugins were not exactly doing what we need or didn't really have a good way of being configured or were quite old and not maintained anymore, so I ended up using the existing git plugin that we already have (com.palantir.git-version).

It supports searching for tag prefixes via gitVersion(prefix: 'apache-iceberg-') but that requires that our tags are annotated slightly different as the prefix must match a regex of the form [/@]?([A-Za-z]+[/@-])+, thus I adjusted the part where we annotate the tags from Apache Iceberg x.y.z to apache-iceberg-x.y.z.

Also gitVersion(prefix: 'apache-iceberg-') just spits out a plain string and doesn't give any flexibility of extracing the MAJOR.MINOR.PATCH parts, thus I had to do some regex matching to extract what we need.

[rdblue]

The tag name is already present later in the command, $tagrc. And the git docs don't say that -a takes any arguments. When I run a test of this, it fails with fatal: too many params, so I don't think that this actually works because it is adding a tag name multiple times.

It looks like you're trying to work around one of two things:

1. The tag isn't on the master branch, it is on the commit that adds verison.txt that is never in master
2. The tag name includes the -rcX suffix

I think it is more likely that you're working around the first because we add a new tag without the -rcX suffix when the release is final. You can see an example here: https://github.com/apache/iceberg/releases/tag/apache-iceberg-0.12.0

[nastra]

you're right, I don't know why I ended up assuming that the tagging annotation part was wrong

[rdblue]

@nastra and I talked on Slack and created a release-base-0.12.0 tag in master to mark where 0.12.0 was branched. That way we can keep the master version based on a tag, but it is obvious that it isn't a release tag.

[rdblue]

Thanks, @nastra!