Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add automated releases to Sonatype Nexus through CI #98

Merged
merged 12 commits into from
May 22, 2019

Conversation

rbreslow
Copy link
Contributor

@rbreslow rbreslow commented Apr 30, 2019

Overview

  • Set version in ThisBuild automatically using the output of $ git describe
  • Add build.sbt configuration for publishing artifacts to Sonatype Nexus
  • Update to sbt 1.2.8
  • Update sbt plugin versions and remove unused plugins
  • Add cipublish STRTA that is branch restricted to master and develop
  • Add GitHub issue template for releases

Resolves #86

Checklist

  • Description of PR is in an appropriate section of the CHANGELOG and grouped with similar changes if possible

Notes

We're using sbt-gpg instead of sbt-pgp because the CircleCI build environment is using GnuPG 2.x, and there's no way for sbt-pgp to read a GnuPG 2.x private key.

This came with a few challenges:

  • sbt-gpg is only targeting Scala 2.12 which necessitated the sbt 1.2.8 bump.
  • There isn't an easy way to use password protected private keys with sbt-gpg. The recommended approach in their README is to create a dedicated signing key for CI that does not have a password and encrypt it with the Travis CLI tool. CircleCI doesn't support encryption in the same way as Travis, but I figured it'd be safe to base64 the key as a secret environment variable. The difficult part was exporting the signing sub key without a passphrase. I had to use GnuPG 1.4 with the export-reset-subkey-passwd export option.

Also, I have a concern that the cache could possibly grow forever. For example, if we are restoring the old cache, dependencies change, and then we cache the whole ~/.ivy2 directory again, when do the old artifacts ever get cleaned? It's possible sbt is intelligent enough that it is already doing this.

Finally, I am not sure the best approach to resolving the line item:

WIP PR commits can be optionally (via a commit message signal) released as SNAPSHOTS with a Git SHA suffixes; bypass tests flag

It isn't clear cut in the CircleCI configuration how you'd filter based on PR/commit names, etc. This may require additional logic in the STRTA. I think it deserves it's own issue to fully explore.

Testing Instructions

There isn't a way to test automated releases to Maven Central without cutting a real release. I think the best way to do this would be to walk through the release checklist together after merging this PR.

I have tested that if the value of version in ThisBuild does not end with -SNAPSHOT, the publish task will publish artifacts to the staging repository. I discarded the artifacts after doing this. From here, if I had ran sonatypeRelease, the staging repository would sync with Maven Central.

For everything else, see CI checks below.

scripts/test Outdated Show resolved Hide resolved
build.sbt Show resolved Hide resolved
build.sbt Show resolved Hide resolved
only:
- develop
- master
- feature/jrb/automated-releases-to-sonatype
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reminder to remove before merging.

@rbreslow rbreslow changed the title [wip] Add automated releases to Sonatype Nexus through CI Add automated releases to Sonatype Nexus through CI Apr 30, 2019
@rbreslow rbreslow self-assigned this Apr 30, 2019
.circleci/config.yml Show resolved Hide resolved
.github/ISSUE_TEMPLATE/release.md Show resolved Hide resolved
.circleci/config.yml Show resolved Hide resolved
build.sbt Outdated Show resolved Hide resolved
scripts/test Outdated Show resolved Hide resolved
.circleci/config.yml Show resolved Hide resolved
@rbreslow rbreslow requested a review from hectcastro May 2, 2019 17:29
Copy link
Contributor

@hectcastro hectcastro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I have a concern that the cache could possibly grow forever. For example, if we are restoring the old cache, dependencies change, and then we cache the whole ~/.ivy2 directory again, when do the old artifacts ever get cleaned? It's possible sbt is intelligent enough that it is already doing this.

I believe that sbt is smart enough to deal with this. This is effectively how we are treating the Ivy cache in CI for other projects (Raster Foundry, Histo). They only get larger as we upgrade dependency versions.

Finally, I am not sure the best approach to resolving the line item:

WIP PR commits can be optionally (via a commit message signal) released as SNAPSHOTS with a Git SHA suffixes; bypass tests flag

It isn't clear cut in the CircleCI configuration how you'd filter based on PR/commit names, etc. This may require additional logic in the STRTA. I think it deserves it's own issue to fully explore.

I think that we can safely ignore this for now.

Copy link
Contributor

@hectcastro hectcastro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we're in a good spot. For next steps, I think it would be good to:

  • Assign someone from the GeoTrellis team to review (I think @pomadchin would be good)
  • After that, schedule a short meeting to go through the steps necessary for a release with all 3 of us

@rbreslow rbreslow requested a review from pomadchin May 6, 2019 13:35
@hectcastro hectcastro requested review from echeipesh and removed request for pomadchin May 8, 2019 13:16
@hectcastro
Copy link
Contributor

@rbreslow, there are some conflicts in this PR now. It might be good to resolve them. Afterwards, @echeipesh, we would appreciate a review.

Copy link
Collaborator

@echeipesh echeipesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, the main question for me was the artifact name com.azavea.geotrellis:maml-jvm_S:x.y.z is consistent with what we've been doing and will be good name of the artifact regardless of the source project.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add automated releases to Sonatype Nexus through CI
3 participants