Intermittent failure when publishing to Maven Central

[t0yv0]

What happened?

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':closeSonatypeStagingRepository'.
> Staging repository is still transitioning after defined time. Consider its increment. StagingRepository(id=compulumi-1379, state=open, transitioning=true)

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org/

BUILD FAILED in 15m 33s
12 actionable tasks: 12 executed
Error: Gradle build failed: process exited with status 1

Steps to reproduce

Run publish on Gradle projects generated through java codegen. Providers team does this a lot and sporadically runs into the step failing.

Expected Behavior

Reliably publishes to Maven Central.

Actual Behavior

Publishing breaks.

Output of pulumi about

N/A

Additional context

N/A

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[guineveresaenger]

I encountered this today when running the SDK publish job for pulumi-mongodbatlas: https://github.com/pulumi/pulumi-mongodbatlas/actions/runs/3085502151/jobs/4989190896#step:23:402

A re-trigger of the publish job failed with the same error.

[thomas11]

Here's one from today: https://github.com/pulumi/pulumi-azure-native/actions/runs/3128392081/attempts/3

[t0yv0]

https://github.com/pulumi/pulumi-azure-native/actions/runs/3128392081/jobs/5099098500#step:23:410 mentions compulumi-1585. I've logged into Nexus UI and I see compulumi-1537 with google-native content in it "stuck", but no azure-native ones. Something deleted it. But I'm starting to wonder if we're colliding on the random "integers" since everything is called compulumi. I can try disambiguating this name per provider.

[t0yv0]

Got emailed this vuln report for compulumi-1586, which seems to have failed CI also. https://sbom.lift.sonatype.com/report/T1-a0368c8f29fdaa555824-8592a30c9ae431-1664316336-bf8055af23fe4f55b0e3fad356c510b3

[t0yv0]

https://github.com/pulumi/pulumi-azure-native/actions/runs/3128392081/jobs/5099370433#step:23:409 confirms Attempt #4 used compulumi-1586.

[t0yv0]

Some notes here: this is a very annoying bug that is not easy to reproduce and may be quite painful to fix, I was not able to fix quickly. It is very likely either a bug or a misconfiguration of the gradle-nexusplugin. How to move forward:

1. it might be possible to enable debug logging in the plugin, and then wait for an incidence in the providers pipeline that will provide full debug logs, which will be sufficient to file an issue with the plugin and maybe move forward to a fix

2. another possibility to consider is bypassing the problem; currently SDK generator for Java fully controls the companion build files it generates and it chooses to generate Gradle build files. It could switch to Maven. In that case we'd be using Maven plugins to accomplish the publish to Nexus task. I expect Maven ecosystem to be more mature so hopefully the plugin will be more reliable or have better error messages when misconfigured.

We might want 2 anyway for product reasons in case the SDK generator needs to run on non-Pulumi owned contexts.