Replace nexus publishing plugin with direct call to central sonatype uploading URL #3723
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…loading URL This replaces the usage of the nexus publishing plugin with direct calls to the sonatype central publishing API. It makes the following modifications: 1. Each published project will write to a central staging repo if we have enabled central publishing. 1. A new task, `createMavenCentralBundle`, will go through and scoop up all of the relevant files from the staging repo to produce the final bundle 1. An additional task, `publishMavenCentralBundle`, will upload the bundle to the central publishing API (see: https://central.sonatype.com/api-doc and https://central.sonatype.org/publish/publish-portal-api/#uploading-a-deployment-bundle) 1. The release action now calls the new `publishMavenCentralBundle` task This should allow us to retire our usage of the legacy API.
alecgrieser
force-pushed
the
use-curl-directly
branch
from
e0efeb3 to
d681a33
Compare
alecgrieser
changed the title
ScottDugas
approved these changes
Nov 5, 2025
Collaborator
ScottDugas
left a comment
ScottDugas
left a comment
There was a problem hiding this comment.
Mostly comments to future proof and prevent unlikely and surprising failures
| connection.setDoOutput(true) | ||
| connection.setRequestProperty('Authorization', "Bearer ${encodedCredentials}") | ||
|
|
||
| def boundary = "----GradleBoundary${System.currentTimeMillis()}" |
Collaborator
There was a problem hiding this comment.
Given that we are uploading a zip, would it be worthwhile to add a uuid other this to help ensure that the zip file itself doesn't have this sequence of bytes? (at least if I understand multipart correctly)
Or would it be worthwhile to validate on our end that the string we use for the boundary is not in the file?
Collaborator
Author
There was a problem hiding this comment.
Yeah, I could see that. It may be overkill, and that is sort of what the current time millis is for. I think a UUID is probably safe enough, though we could do a search for it, I suppose
| destinationDirectory = layout.buildDirectory.dir('distributions') | ||
|
|
||
| from(stagingRepo) { | ||
| include "${rootProject.group.replaceAll('.', '/')}/**/${project.version}/*" |
Collaborator
There was a problem hiding this comment.
This means if a sub-project publishes to the staging repo, but doesn't put the results in org/foundationdb we just won't publish it, right? Probably ok, at least for the time being to not protect against that, I can't foresee us messing that up, and I think the only improvement would be to error.
Collaborator
Author
There was a problem hiding this comment.
Yes, though that's somewhat deliberate, as the bundle we produce is tagged with the group ID of the project, and I believe that central may actually reject our artifacts if it's in a different group
alecgrieser
added a commit
to alecgrieser/fdb-record-layer
that referenced
this pull request
Nov 5, 2025
This updates the logic in the `publishMavenCentralBundle` task so that it no longer loads the full contents of the file into memory. It does this by maintaining a buffer and then writing the contents of the buffer into the stream, which allows us to avoid materializing the full element into memory. This is to try and fix a Java heap memory error that we saw previously (see: https://github.com/FoundationDB/fdb-record-layer/actions/runs/19108664941/job/54605649248). While I was here, I also updated the boundary for the HTTP multi-part form from the current time millis to a UUID, in response to a comment from the previous PR (FoundationDB#3723 (comment)).
alecgrieser
added a commit
that referenced
this pull request
Nov 5, 2025
This updates the logic in the `publishMavenCentralBundle` task so that it no longer loads the full contents of the file into memory. It does this by maintaining a buffer and then writing the contents of the buffer into the stream, which allows us to avoid materializing the full element into memory. This is to try and fix a Java heap memory error that we saw previously (see: https://github.com/FoundationDB/fdb-record-layer/actions/runs/19108664941/job/54605649248). While I was here, I also updated the boundary for the HTTP multi-part form from the current time millis to a UUID, in response to a comment from the previous PR (#3723 (comment)).
alecgrieser
added a commit
to alecgrieser/fdb-record-layer
that referenced
this pull request
Nov 6, 2025
…onatype uploading URL (FoundationDB#3723)" This reverts commit da406a3.
arnaud-lacurie
pushed a commit
that referenced
this pull request
Nov 7, 2025
This reverts #3723 and #3724 and returns us back to publishing via the nexus publishing API. The impetus for this is that the last approach got stuck in a publishing state after the upload (see: https://github.com/FoundationDB/fdb-record-layer/actions/runs/19117047380/job/54633288001, which succeeded, but for which we didn't get artifacts actually published despite the server upload succeeding). When we are in this mode, we expect the build to fail with a 400 error letting us know that we didn't upload anything. However, if we didn't `close` the staging repositories (which is the step that is failing), then legacy API would not be expected to upload anything to the central repo. So we need to continue to do that part even though we think it will fail.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This replaces the usage of the nexus publishing plugin with direct calls to the sonatype central publishing API. It makes the following modifications:
createMavenCentralBundle, will go through and scoop up all of the relevant files from the staging repo to produce the final bundlepublishMavenCentralBundle, will upload the bundle to the central publishing API (see: https://central.sonatype.com/api-doc and https://central.sonatype.org/publish/publish-portal-api/#uploading-a-deployment-bundle)publishMavenCentralBundletaskThis should allow us to retire our usage of the legacy API. Locally, I was able to validate that the bundle we get back from
createMavenCentralBundlehas the same layout as one generated by the JReleaser tool. Running the upload task, I was able to validate that I get an invalid token error, which is as good as I could get without giving it the proper credentials.This resolves #3709