-
-
Notifications
You must be signed in to change notification settings - Fork 421
Add PublishCredentialsModule, MavenPublish and MavenPublishModule #6023
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+241
−85
Merged
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
b481792
Add SonatypeCredentialsModule, MavenPublsh and MavenPublishModule
maxstreese 0023385
[autofix.ci] apply automated fixes
autofix-ci[bot] 5c4f48a
Add scalalib alias and export for MavenPublishModule
maxstreese af98ac1
Fix a bug in choosing between snapshot and release uri
maxstreese 9de0f54
remove unroll annotation
maxstreese 1a2155f
Apply code review; Rename and refactor
maxstreese d3036c5
[autofix.ci] apply automated fixes
autofix-ci[bot] 92629e7
Remove bundleName parameter (has no effect in publishing)
maxstreese fda8ffc
Document the MavenPublishModule
maxstreese fa7796c
Merge branch 'main' into maven-publish
maxstreese ff06f45
Merge branch 'main' into maven-publish
maxstreese File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| package mill.javalib | ||
|
|
||
| import com.lumidion.sonatype.central.client.core.SonatypeCredentials | ||
| import mill.api.daemon.Logger | ||
| import mill.javalib.PublishModule.PublishData | ||
| import mill.javalib.internal.MavenWorkerSupport as InternalMavenWorkerSupport | ||
|
|
||
| private[mill] trait MavenPublish { | ||
|
|
||
| def mavenPublishDatas( | ||
| publishDatas: Seq[PublishData], | ||
| credentials: SonatypeCredentials, | ||
| releaseUri: String, | ||
| snapshotUri: String, | ||
| taskDest: os.Path, | ||
| log: Logger, | ||
| env: Map[String, String], | ||
| worker: InternalMavenWorkerSupport.Api | ||
| ): Unit = { | ||
| val dryRun = env.get("MILL_TESTS_PUBLISH_DRY_RUN").contains("1") | ||
|
|
||
| val (snapshots, releases) = publishDatas.partition(_.meta.isSnapshot) | ||
|
|
||
| releases.map(_ -> false).appendedAll(snapshots.map(_ -> true)).foreach { (data, isSnapshot) => | ||
| mavenPublishData( | ||
| dryRun = dryRun, | ||
| publishData = data, | ||
| isSnapshot = isSnapshot, | ||
| credentials = credentials, | ||
| releaseUri = releaseUri, | ||
| snapshotUri = snapshotUri, | ||
| taskDest = taskDest, | ||
| log = log, | ||
| worker = worker | ||
| ) | ||
| } | ||
| } | ||
|
|
||
| def mavenPublishData( | ||
| dryRun: Boolean, | ||
| publishData: PublishData, | ||
| isSnapshot: Boolean, | ||
| credentials: SonatypeCredentials, | ||
| releaseUri: String, | ||
| snapshotUri: String, | ||
| taskDest: os.Path, | ||
| log: Logger, | ||
| worker: InternalMavenWorkerSupport.Api | ||
| ): Unit = { | ||
| val uri = if (isSnapshot) snapshotUri else releaseUri | ||
| val artifacts = MavenWorkerSupport.RemoteM2Publisher.asM2ArtifactsFromPublishDatas( | ||
| publishData.meta, | ||
| publishData.payloadAsMap | ||
| ) | ||
|
|
||
| if (isSnapshot) { | ||
| log.info( | ||
| s"Detected a 'SNAPSHOT' version for ${publishData.meta}, publishing to Maven Repository at '$uri'" | ||
| ) | ||
| } | ||
|
|
||
| /** Maven uses this as a workspace for file manipulation. */ | ||
| val mavenWorkspace = taskDest / "maven" | ||
|
|
||
| if (dryRun) { | ||
| val publishTo = taskDest / "repository" | ||
| val result = worker.publishToLocal( | ||
| publishTo = publishTo, | ||
| workspace = mavenWorkspace, | ||
| artifacts | ||
| ) | ||
| log.info(s"Dry-run publishing to '$publishTo' finished with result: $result") | ||
| } else { | ||
| val result = worker.publishToRemote( | ||
| uri = uri, | ||
| workspace = mavenWorkspace, | ||
| username = credentials.username, | ||
| password = credentials.password, | ||
| artifacts | ||
| ) | ||
| log.info(s"Publishing to '$uri' finished with result: $result") | ||
| } | ||
| } | ||
|
|
||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| package mill.javalib | ||
|
|
||
| import mill.* | ||
| import mill.api.* | ||
| import mill.util.Tasks | ||
|
|
||
| /** | ||
| * External module to publish artifactes to Maven repositories other than `central.sonatype.org` | ||
| * (e.g. a private Maven repository). | ||
| */ | ||
| object MavenPublishModule extends ExternalModule, DefaultTaskModule, MavenWorkerSupport, | ||
| PublishCredentialsModule, MavenPublish { | ||
|
|
||
| def defaultTask(): String = "publishAll" | ||
|
|
||
| def publishAll( | ||
| publishArtifacts: mill.util.Tasks[PublishModule.PublishData] = | ||
| Tasks.resolveMainDefault("__:PublishModule.publishArtifacts"), | ||
| username: String = "", | ||
| password: String = "", | ||
| releaseUri: String, | ||
| snapshotUri: String | ||
| ): Command[Unit] = Task.Command { | ||
| val artifacts = Task.sequence(publishArtifacts.value)() | ||
|
|
||
| val credentials = getPublishCredentials("MILL_MAVEN", username, password)() | ||
|
|
||
| mavenPublishDatas( | ||
| artifacts, | ||
| credentials, | ||
| releaseUri = releaseUri, | ||
| snapshotUri = snapshotUri, | ||
| taskDest = Task.dest, | ||
| log = Task.log, | ||
| env = Task.env, | ||
| worker = mavenWorker() | ||
| ) | ||
| } | ||
|
|
||
| lazy val millDiscover: Discover = Discover[this.type] | ||
|
|
||
| } | ||
42 changes: 42 additions & 0 deletions
42
libs/javalib/src/mill/javalib/PublishCredentialsModule.scala
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| package mill.javalib | ||
|
|
||
| import com.lumidion.sonatype.central.client.core.SonatypeCredentials | ||
| import mill.api.* | ||
|
|
||
| /** | ||
| * Internal module to Retrieve credentials for publishing to Maven repositories | ||
| * (e.g. `central.sonatype.org` or a private Maven repository). | ||
| */ | ||
| private[mill] trait PublishCredentialsModule extends Module { | ||
| def getPublishCredentials( | ||
| envVariablePrefix: String, | ||
| usernameParameterValue: String, | ||
| passwordParameterValue: String | ||
| ): Task[SonatypeCredentials] = Task.Anon { | ||
| val username = | ||
| getPublishCredential(usernameParameterValue, "username", s"${envVariablePrefix}_USERNAME")() | ||
| val password = | ||
| getPublishCredential(passwordParameterValue, "password", s"${envVariablePrefix}_PASSWORD")() | ||
| Result.Success(SonatypeCredentials(username, password)) | ||
| } | ||
|
|
||
| private def getPublishCredential( | ||
| credentialParameterValue: String, | ||
| credentialName: String, | ||
| envVariableName: String | ||
| ): Task[String] = Task.Anon { | ||
| if (credentialParameterValue.nonEmpty) { | ||
| Result.Success(credentialParameterValue) | ||
| } else { | ||
| (for { | ||
| credential <- Task.env.get(envVariableName) | ||
| } yield { | ||
| Result.Success(credential) | ||
| }).getOrElse( | ||
| Result.Failure( | ||
| s"No $credentialName set. Consider using the $envVariableName environment variable or passing `$credentialName` argument" | ||
| ) | ||
| ) | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,7 @@ | ||
| package mill.scalalib | ||
|
|
||
| object Dependency extends mill.api.ExternalModule.Alias(mill.javalib.Dependency) | ||
| object MavenPublishModule | ||
| extends mill.api.ExternalModule.Alias(mill.javalib.MavenPublishModule) | ||
| object SonatypeCentralPublishModule | ||
| extends mill.api.ExternalModule.Alias(mill.javalib.SonatypeCentralPublishModule) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add some scaladoc to this trait and its companion object, and explain what is the difference between this and the normal
trait PublishModuleandobject SonatypeCentralPublishModulewe typically tell people to use?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
traitat all? Ideally we should just have everyone extendPublishModuleand use different companion object external modules to publish them to different placesThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Regarding "do we need this
traitat all": I was just exactly following what is already done intraitSonatypeCentralPublishModuleand its companionobject. Therefore it seems clean to me to not break with that design?Regarding what the difference is: What
SonatypeCentralPublishModuledoes for Sonatype Central,MavenPublishModuledoes for "plain" Maven. I can certainly make this distinction clear by providing a scaladoc for both modules.To try to explain my understanding as best as possible:
SonatypeCentralPublishModuleis specifically for publishing to the service https://central.sonatype.com/. Because of at least staging releases (not sure if something else), this is not exactly the same as publishing to a repository that "just" follows the Maven2 Repository Layout (even if that repo is managed by the Sonatype software). That is the reason whyMavenPublishModulehas been brought to life. Now to go even deeper into this, I believe it should be possible and in fact probably also cleaner and closer to reality, to have all of this functionality in one module and/orpublishtask, which can be configured in a multitude of ways (e.g. publish to central or private; specify credentials or not; cryptographically sign or not). This is what was actually done in the deprecatedpublishAlltask of thePublishModuleitself and is stated in the Mill docs (though - if I remember my testing correctly - was not 100 % compliant for "plain" Maven publishing because it did not createmaven-metadata.xml). Implementing this unified approach is something that I consider both out-of-scope for this PR and a bit out of my comfort zone. Though I'd certainly be open to try if you agree :)Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the important thing is to make sure
./mill mill.javalib.MavenPublishModule/works for publishing normalmill.javalib.publish.PublishModules, and not just this particular trait. The same way that./mill mill.javalib.SonatypeCentralPublishModule/works on normalmill.javalib.publish.PublishModules without requiring the individual modulesextend mill.javalib.SonatypeCentralPublishModuleIf that works, then we can probably do without this
traitand just ask people to use the companion objectThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In my understanding
./mill mill.javalib.MavenPublishModule/works for publishing normalmill.javalib.publish.PublishModules with the current implementation in this PR, becauseMavenPublishModuleis anExternalModule, it's default task ispublishAllandpublishAllspublishArtifactsparameter by default resolves to__:PublishModule.publishArtifacts(following exactly what is done inSonatypeCentralPublishModule). Or am I missing something here?I would propose to still keep the
MavenPublishModuletrait for two reasons:SonatypeCentralPublishModuleand therefore is uniform./mill mill.javalib.MavenPublishModule/or (for example)./mill __.publishMaven. I think giving users the second option is important, because of the following scenario: What if I want to publish some of my modules to Sonatype Central and some to a private repo? Or what if I want to publish to different private repos? When keeping theMavenPublishModuletrait, this is easy to express in the build by extending it in different modules and supplying different parameters. I think with the external module alone this would be more tricky?Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just to clarify, the external module's command accepting a
--publish-artifactsoption itself accepting a task query like__.publishArtifactsor<my-module>.publishArtifactsshould already allow to publish selected modules via different methods.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah I was missing this part, thanks! So what I could do to put this in - say - a shared module in a build is something like the below right?
In this case should I provide this option in the docs? And related: Is there a different way than using the
Tasksutil for this?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think expecting people to pass the values via the CLI or env variables would be fine for now, that's what we mostly recommend for
SonatypePublishModuleand it seems to work