Adopt GHA Scala Library Release Workflow #299

rtyley · 2023-12-13T14:41:39Z

This replaces the old release process which had developers manually running sbt release on their own laptops - each developer had to obtain their own PGP key and Sonatype credentials, which was an elaborate & fiddly process.

This PR adopts gha-scala-library-release-workflow, which provides an automated release process that uses single shared set of release credentials, available through GitHub Organisation Secrets (like we already do with NPM) - this workflow allows you to release a new library version with a click of a button:

RunReleaseWorkflow.mov

Required changes

The changes required to adopt the automated workflow:

No need to set these sbt configuration keys, as they're now taken care of by the workflow:
- sonatypeProfileName
- publishTo
- scmInfo & pomExtra
Remove the sign, publish, release & push steps of sbt-release's releaseProcess configuration, because the workflow does those now, and the workflow only wants sbt release to create the versioning commits, and tag them appropriately. The workflow does the rest itself.
Remove sbt-pgp plugin because it's no longer used - the workflow does the signing using gpg directly.
Grant this repo access to the GitHub Organisation Secrets containing the Maven Release credentials with https://github.com/guardian/github-secret-access/pull/21
Unusually, drop running the tests as part of the release for now, as the tests in this project require special credentials (see Add Continuous Integration (CI) as GitHub Action #272)

Additionally, we add automatic version numbering based on compatibility assessment performed by sbt-version-policy:

Add the sbt-version-policy plugin, to allow it to do the compatibility assessment. This also sets the versionScheme for this library to early-semver, which is the recommended versioning for Scala libraries, and sbt-version-policy & correct sbt-eviction-issue-detection pretty much depend on the versionScheme being early-semver. Thus we also need to switch to using a new semver version number for our library version!
Add the releaseVersion := fromAggregatedAssessedCompatibilityWithLatestRelease().value sbt setting, which will intelligently set the release version based on sbt-version-policy's compatibility assessment, thanks to Simplify integration with sbt-release scalacenter/sbt-version-policy#187 .
Use publish / skip := true, rather than other hacks like publish := {} or publishArtifact := false, to tell sbt not to publish modules that we don't want published (typically, the 'root' module) - this is important because sbt-version-policy won't understand those hacks, but will understand publish / skip := true means that it doesn't need to assess compatibility there.

Recent prior example of adding gha-scala-library-release-workflow to a repo: guardian/play-googleauth#208

rtyley · 2023-12-13T16:36:04Z

.github/workflows/release.yml

+      AUTOMATED_MAVEN_RELEASE_PGP_SECRET: ${{ secrets.AUTOMATED_MAVEN_RELEASE_PGP_SECRET }}
+      AUTOMATED_MAVEN_RELEASE_SONATYPE_PASSWORD: ${{ secrets.AUTOMATED_MAVEN_RELEASE_SONATYPE_PASSWORD }}


These GitHub Organisation Secrets will be granted to this repo by https://github.com/guardian/github-secret-access/pull/21.

rtyley · 2023-12-13T16:42:45Z

build.sbt

-      </developers>
-    ),
+  licenses := Seq("Apache V2" -> url("https://www.apache.org/licenses/LICENSE-2.0.html")),
+  releaseVersion := fromAggregatedAssessedCompatibilityWithLatestRelease().value,


fromAggregatedAssessedCompatibilityWithLatestRelease() will intelligently set the release version based on sbt-version-policy's compatibility assessment, thanks to scalacenter/sbt-version-policy#187 - so a 'major', 'minor' or 'patch' version number bump, according to Scala's recommended semver guidelines.

This means that sbt 1.5.0 and above will be able to tell when incompatible versions of this library are being used in a project.

rtyley · 2023-12-13T16:44:24Z

build.sbt

-    releaseStepCommandAndRemaining("+publishSigned"),
-    releaseStepCommand("sonatypeBundleRelease"),


For projects using gha-scala-library-release-workflow, the 'sign' & 'release' steps are taken care of by the workflow, rather than by the sbt-release plugin, so that the necessary credentials can be isolated from the library build/test process.

This means we can remove publishSigned & sonatypeBundleRelease here.

rtyley · 2023-12-13T16:45:56Z

build.sbt

    setNextVersion,
-    commitNextVersion,
-    pushChanges


For projects using gha-scala-library-release-workflow, the 'push' step is taken care of by the workflow, rather than by the sbt-release plugin, so that the library build/test process does not need to be given 'write' access to the repository.

rtyley · 2023-12-13T16:48:53Z

build.sbt

-    publishArtifact := false,
    publish / skip := true,


publishArtifact := false is unnecessary when publish / skip := true is set.

publish / skip is the recommended key for this purpose, and is the key examined by sbt-version-policy when it checks to see if it can skip calculating compatibility for a particular subproject.

rtyley · 2023-12-13T16:50:42Z

build.sbt

@@ -70,12 +47,12 @@ def baseProject(module: String, majorMinorVersion: String) = Project(s"$module-p
    scalaVersion := "2.13.11",
    crossScalaVersions := Seq(scalaVersion.value, "2.12.18"),
    scalacOptions := Seq(
+        "-release:11",


gha-scala-library-release-workflow is currently using Java 17 for compiling everything, so it's important to specify that you want Java 11-compatible bytecode if you're going to be using this library in projects running Java 11.

rtyley · 2023-12-13T16:52:53Z

build.sbt

        "-feature",
        "-deprecation",
        "-Xfatal-warnings"
    ),
    libraryDependencies += scalaTest,
-    publishTo := sonatypePublishToBundle.value,


gha-scala-library-release-workflow will set publishTo to the appropriate value (as it happens, a local-filesystem staging repository), no need to set it here.

rtyley · 2023-12-13T16:57:03Z

project/plugins.sbt

-addSbtPlugin("com.github.sbt" % "sbt-pgp" % "2.2.1")
+addSbtPlugin("ch.epfl.scala" % "sbt-version-policy" % "3.2.0")


We don't need the sbt-pgp plugin, as gha-scala-library-release-workflow will sign the artifacts directly using GPG... but we do need sbt-version-policy, to intelligently work out the correct version number based on compatibility with the latest release:

releaseVersion := fromAggregatedAssessedCompatibilityWithLatestRelease().value

Note also that our project automatically has versionScheme set to early-semver by sbt-version-policy - this is the only versionScheme currently supported by sbt-version-policy.

rtyley · 2023-12-13T17:00:07Z

build.sbt

-  scmInfo := Some(ScmInfo(
-    url("https://github.com/guardian/facia-scala-client"),
-    "scm:git:git@github.com:guardian/facia-scala-client.git"
-  )),
-  pomExtra := (
-    <url>https://github.com/guardian/facia-scala-client</url>
-      <developers>
-        <developer>
-          <id>janua</id>
-          <name>Francis Carr</name>
-          <url>https://github.com/janua</url>
-        </developer>
-        <developer>
-          <id>adamnfish</id>
-          <name>Adam Fisher</name>
-          <url>https://github.com/adamnfish</url>
-        </developer>
-      </developers>
-    ),


All the required metadata deleted here is now provided by gha-scala-library-release-workflow (quite concisely, thanks to sbt-sonatype's GitHubHosting helper!).

rtyley · 2023-12-13T17:06:15Z

build.sbt

  releaseCrossBuild := true, // true if you cross-build the project for multiple Scala versions
  releaseProcess := Seq[ReleaseStep](
    checkSnapshotDependencies,
    inquireVersions,
    runClean,
-    runTest,


This is a bit unusual, the other 3 repos that that have adopted gha-scala-library-release-workflow haven't needed to drop this line, but the unit tests in this project, facia-scala-client, require special credentials (see #272) and passing them through in a generic way to gha-scala-library-release-workflow is a bit tricky.

I have a couple of ideas on how to solve this, but would like to address it in a subsequent PR.

This replaces the old release process which had developers manually running `sbt release` on their own laptops - each developer had to obtain their own PGP key and Sonatype credentials, which was an elaborate & fiddly process. Now there's a single set of release credentials, available through GitHub Organisation Secrets, like we already have with NPM. ### Required changes The changes required to adopt the automated workflow: * No need to set these sbt configuration keys, as they're now taken care of by the workflow: * `sonatypeProfileName` * `publishTo` * `scmInfo` & `pomExtra` * Remove the sign, publish, release & push steps of sbt-release's `releaseProcess` configuration, because the workflow does those now, and the workflow only wants `sbt release` to create the versioning commits, and tag them appropriately. The workflow does the rest itself. * Remove `sbt-pgp` plugin because it's no longer used - the workflow does the signing using `gpg` directly. * Grant this repo access to the GitHub Organisation Secrets containing the Maven Release credentials with guardian/github-secret-access#21 * Unusually, drop running the tests as part of the release for now, as the tests in this project require special credentials (see #272) Additionally, we add **automatic version numbering** based on compatibility assessment performed by `sbt-version-policy`: * Add the `sbt-version-policy` plugin, to allow it to do the compatibility assessment. This also sets the `versionScheme` for this library to `early-semver`, which is the recommended versioning for Scala libraries, and `sbt-version-policy` & correct sbt-eviction-issue-detection pretty much depend on the `versionScheme` being `early-semver`. Thus we also need to switch to using a new semver version number for our library version! * Add the `releaseVersion := fromAggregatedAssessedCompatibilityWithLatestRelease().value` sbt setting, which will intelligently set the release version based on `sbt-version-policy`'s compatibility assessment, thanks to scalacenter/sbt-version-policy#187 . * Use `publish / skip := true`, rather than other hacks like `publish := {}` or `publishArtifact := false`, to tell sbt not to publish modules that we don't want published (typically, the 'root' module) - this is important because `sbt-version-policy` won't understand those hacks, but _will_ understand `publish / skip := true` means that it doesn't need to assess compatibility there. Recent prior example of adding `gha-scala-library-release-workflow` to a repo: guardian/play-googleauth#208

davidfurey

lgtm

rtyley · 2023-12-13T18:26:59Z

Now merged, this has successfully performed a release, v4.0.7 - all looks good!

Further to #102 (comment), these are a few more tweaks: * Delete the entire `sonatype.sbt` file that was providing these keys that `gha-scala-library-release-workflow` now takes care of for us: * `sonatypeProfileName` - see https://github.com/guardian/gha-scala-library-release-workflow/blob/7d278d4d44e30b4b4c0f6791053bdeb40b8159cb/.github/workflows/reusable-release.yml#L258 * `publishTo` - see guardian/facia-scala-client#299 (comment) * `scmInfo` & `pomExtra` - see guardian/facia-scala-client#299 (review) * `homepage` - https://github.com/xerial/sbt-sonatype/blob/da75ed2efe29e7d674a6d6af5103b4aa1cecefb8/src/main/scala/xerial/sbt/Sonatype.scala#L368 * Specify Java 11-compatible bytecode - see https://github.com/guardian/facia-scala-client/pull/299/files#r1425630691 As part of some of these changes, I've also removed the vestiges of the build-configuration that adapted the build for Scala 2.11 & 2.12 (eg the `versionDependent()` helper method) - these are no longer need since #78 removed support for Scala 2.12 in January 2023.

- Adopt scala library release GHA workflow Following [the example from play-secret-rotation](guardian/play-secret-rotation#416), rather than set up a new developer credential to release the next version of this library, this commit sets up [guardian/gha-scala-library-release-workflow](https://github.com/guardian/gha-scala-library-release-workflow). - Further `gha-scala-library-release-workflow` tweaks Further to #102 (comment), these are a few more tweaks: * Delete the entire `sonatype.sbt` file that was providing these keys that `gha-scala-library-release-workflow` now takes care of for us: * `sonatypeProfileName` - see https://github.com/guardian/gha-scala-library-release-workflow/blob/7d278d4d44e30b4b4c0f6791053bdeb40b8159cb/.github/workflows/reusable-release.yml#L258 * `publishTo` - see guardian/facia-scala-client#299 (comment) * `scmInfo` & `pomExtra` - see guardian/facia-scala-client#299 (review) * `homepage` - https://github.com/xerial/sbt-sonatype/blob/da75ed2efe29e7d674a6d6af5103b4aa1cecefb8/src/main/scala/xerial/sbt/Sonatype.scala#L368 * Specify Java 11-compatible bytecode - see https://github.com/guardian/facia-scala-client/pull/299/files#r1425630691 As part of some of these changes, I've also removed the vestiges of the build-configuration that adapted the build for Scala 2.11 & 2.12 (eg the `versionDependent()` helper method) - these are no longer need since #78 removed support for Scala 2.12 in January 2023. Co-authored-by: Roberto Tyley <roberto.tyley@gmail.com>

rtyley mentioned this pull request Dec 13, 2023

Adopt GHA Scala Library Release Workflow guardian/play-googleauth#208

Merged

rtyley commented Dec 13, 2023

View reviewed changes

rtyley force-pushed the adopt-GHA-Scala-Library-Release-Workflow branch from aef541d to dfb698f Compare December 13, 2023 17:18

rtyley marked this pull request as ready for review December 13, 2023 17:20

rtyley requested a review from davidfurey December 13, 2023 17:20

rtyley mentioned this pull request Dec 13, 2023

Upgrade to play-json 3.x #300

Merged

7 tasks

davidfurey approved these changes Dec 13, 2023

View reviewed changes

rtyley merged commit eb8d956 into main Dec 13, 2023
1 check passed

rtyley mentioned this pull request Dec 15, 2023

Adopt scala library release GHA workflow guardian/marley#102

Merged

rtyley mentioned this pull request Dec 20, 2023

Adopt GHA Scala Library Release Workflow guardian/mobile-apps-api-models#55

Merged

rtyley mentioned this pull request Jan 10, 2024

Adoption of gha-scala-library-release-workflow across the Guardian guardian/gha-scala-library-release-workflow#20

Open

davidfurey mentioned this pull request Feb 20, 2024

Adopt GHA Scala Library Release Workflow guardian/tags-thrift-schema#44

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adopt GHA Scala Library Release Workflow #299

Adopt GHA Scala Library Release Workflow #299

rtyley commented Dec 13, 2023 •

edited

rtyley Dec 13, 2023

rtyley Dec 13, 2023

rtyley Dec 13, 2023 •

edited

rtyley Dec 13, 2023

rtyley Dec 13, 2023

rtyley Dec 13, 2023

rtyley Dec 13, 2023

rtyley Dec 13, 2023 •

edited

rtyley Dec 13, 2023

rtyley Dec 13, 2023

davidfurey left a comment

rtyley commented Dec 13, 2023

		AUTOMATED_MAVEN_RELEASE_PGP_SECRET: ${{ secrets.AUTOMATED_MAVEN_RELEASE_PGP_SECRET }}
		AUTOMATED_MAVEN_RELEASE_SONATYPE_PASSWORD: ${{ secrets.AUTOMATED_MAVEN_RELEASE_SONATYPE_PASSWORD }}

		releaseStepCommandAndRemaining("+publishSigned"),
		releaseStepCommand("sonatypeBundleRelease"),

		addSbtPlugin("com.github.sbt" % "sbt-pgp" % "2.2.1")
		addSbtPlugin("ch.epfl.scala" % "sbt-version-policy" % "3.2.0")

Adopt GHA Scala Library Release Workflow #299

Adopt GHA Scala Library Release Workflow #299

Conversation

rtyley commented Dec 13, 2023 • edited

Required changes

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rtyley Dec 13, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rtyley Dec 13, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

davidfurey left a comment

Choose a reason for hiding this comment

rtyley commented Dec 13, 2023

rtyley commented Dec 13, 2023 •

edited

rtyley Dec 13, 2023 •

edited

rtyley Dec 13, 2023 •

edited