Skip to content

Retry Maven Central flakes when building the release jars - #5500

Merged
shai-almog merged 2 commits into
masterfrom
ci-retry-release-jars
Jul 31, 2026
Merged

Retry Maven Central flakes when building the release jars#5500
shai-almog merged 2 commits into
masterfrom
ci-retry-release-jars

Conversation

@shai-almog

Copy link
Copy Markdown
Collaborator

What happened

Run 30594669157 (release 7.0.264) failed the release-jars job 30s in:

[ERROR] Plugin org.apache.maven.plugins:maven-compiler-plugin:3.11.0 ... could not be resolved:
  status code: 429, reason phrase: Too Many Requests (429)

A transient Maven Central 429 from the runner CDN edge, at plugin resolution, before anything compiled. The job's last step attaches the ParparVM jars to the published release, so it never ran and 7.0.264 shipped without them — every release since 7.0.261 has ByteCodeTranslator-1.0-SNAPSHOT.jar and JavaAPI-1.0-SNAPSHOT.jar, 7.0.264 had only the guide and javadocs. Anything fetching releases/latest/download/... for those jars got a 404.

The sibling vm-tests job on the same run passed — it already retries this exact flake (added for the 403 variant). release-jars called mvn bare.

This job only runs on release-published events, so it fails rarely, silently, and exactly when it matters.

The change

Lift the inline retry out of vm-tests into scripts/ci/retry.sh, next to the existing scripts/ci/apt-get-update.sh, and use it from both jobs. Bounded at 3 attempts / 30s, overridable via RETRY_ATTEMPTS and RETRY_DELAY_SECONDS; the last attempt's exit status propagates. A real build failure fails identically on every attempt, so retrying only costs time on a genuinely broken build.

Verified locally: success passes through, a flaky command succeeds on attempt 2, a hard failure exits with the command's own status (7), a missing command exits 127, and no arguments exits 2.

Already done separately

7.0.264's assets were restored by re-running the failed job — the release page now carries both jars again. This PR is only about it not recurring.

Noted, not fixed here

  • softprops/action-gh-release@v1 in this job targets the deprecated Node 20 runtime (warning on the successful rerun). Same failure class — if it breaks, a release silently ships without jars — but a v1v2 bump is its own change.
  • HeavyLoadBenchmarkTest logs WARNING: Failed to compile HelloCodenameOne on every run and continues: it compiles scripts/hellocodenameone with JDK 8, but that source is Java 17 (records, switch expressions, text blocks). The benchmark has never included the app payload it intends to. Pre-existing and unrelated to the release failure.

🤖 Generated with Claude Code

The release-jars job builds the ParparVM JavaAPI and ByteCodeTranslator
jars and attaches them to the published GitHub release. It called mvn
bare, so a single transient Maven Central 403/429 from the runner CDN
edge killed it at plugin resolution, ~30s in, before compiling anything.

That is what happened on 7.0.264: maven-compiler-plugin:3.11.0 came back
429 Too Many Requests, the job failed, and the release published without
its two jars -- every release from 7.0.261 on had them. The job only runs
on release-published events, so it fails rarely, silently, and precisely
when it matters. (7.0.264's assets have since been restored by re-running
the job.)

The sibling vm-tests job already worked around the same flake with an
inline retry helper. Lift that into scripts/ci/retry.sh, alongside the
existing scripts/ci/apt-get-update.sh, and use it from both jobs so the
release path gets the protection the test path already had.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 70f2628bd8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/parparvm-tests.yml
Comment thread scripts/ci/retry.sh Outdated
…nges

Two review findings from the Codex pass:

RETRY_ATTEMPTS=0 or a non-numeric value made `seq` emit nothing (GNU) or
error, the loop body never ran, `status` stayed 0, and the wrapper exited
successfully *without ever running the command* -- the worst possible
failure for a release-jars guard, and a direct contradiction of the
script's own "always attempted at least once" contract. Both overrides
are now validated as integers and rejected with exit 2, consistent with
the existing no-command usage error; these are set in workflow YAML, so
a typo should be loud rather than silently clamped.

While confirming it: BSD and GNU seq disagree on degenerate ranges
(`seq 1 0` yields nothing on GNU but "1 0" on BSD, so attempts=0 ran the
command twice on macOS and zero times in CI). Replaced with an arithmetic
while-loop, which runs the body exactly `attempts` times on both.

Also added scripts/ci/retry.sh to this workflow's pull_request and push
path filters, so a future change to the helper actually exercises the
jobs that consume it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

✅ Continuous Quality Report

Test & Coverage

Static Analysis

  • SpotBugs [Report archive]
    • ByteCodeTranslator: 0 findings (no issues)
    • android: 0 findings (no issues)
    • codenameone-maven-plugin: 0 findings (no issues)
    • core-unittests: 0 findings (no issues)
    • ios: 0 findings (no issues)
  • PMD: 0 findings (no issues) [Report archive]
  • Checkstyle: 0 findings (no issues) [Report archive]

Generated automatically by the PR CI workflow.

@github-actions

Copy link
Copy Markdown
Contributor

✅ ByteCodeTranslator Quality Report

Test & Coverage

  • Tests: 414 total, 0 failed, 14 skipped

Benchmark Results

  • Execution Time: 21115 ms

  • Hotspots (Top 20 sampled methods):

    • 16.71% java.util.ArrayList.indexOf (299 samples)
    • 4.53% java.lang.StringBuilder.append (81 samples)
    • 4.02% com.codename1.tools.translator.ByteCodeClass.markDependent (72 samples)
    • 3.86% com.codename1.tools.translator.BytecodeMethod.equals (69 samples)
    • 3.75% com.codename1.tools.translator.ByteCodeClass.hasDeclaredMethod (67 samples)
    • 3.58% com.codename1.tools.translator.BytecodeMethod.optimize (64 samples)
    • 3.30% com.codename1.tools.translator.Parser.cn1EnsureSubclassIndex (59 samples)
    • 2.63% com.codename1.tools.translator.bytecodes.Ldc.addToConstantPool (47 samples)
    • 2.40% org.objectweb.asm.tree.analysis.Analyzer.findSubroutine (43 samples)
    • 2.18% com.codename1.tools.translator.Parser.classIndex (39 samples)
    • 1.79% java.lang.System.identityHashCode (32 samples)
    • 1.57% org.objectweb.asm.tree.analysis.Analyzer.analyze (28 samples)
    • 1.45% com.codename1.tools.translator.bytecodes.Invoke.findMethodUp (26 samples)
    • 1.45% java.lang.StringCoding.encode (26 samples)
    • 1.45% com.codename1.tools.translator.Parser.generateClassAndMethodIndexHeader (26 samples)
    • 1.34% com.codename1.tools.translator.BytecodeMethod.appendCMethodPrefix (24 samples)
    • 1.34% java.util.HashMap.hash (24 samples)
    • 1.17% org.objectweb.asm.ClassReader.readCode (21 samples)
    • 1.06% java.lang.Object.hashCode (19 samples)
    • 1.01% com.codename1.tools.translator.bytecodes.Invoke.resolveDirectTarget (18 samples)
  • ⚠️ Coverage report not generated.

Static Analysis

  • ✅ SpotBugs: no findings (report was not generated by the build).
  • ⚠️ PMD report not generated.
  • ⚠️ Checkstyle report not generated.

Generated automatically by the PR CI workflow.

@shai-almog
shai-almog merged commit 8c6dc61 into master Jul 31, 2026
17 of 18 checks passed
@shai-almog
shai-almog deleted the ci-retry-release-jars branch July 31, 2026 04:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant