Skip to content

[improve][build] Replace check-binary-license.sh with a Gradle task#25673

Merged
merlimat merged 1 commit into
apache:masterfrom
lhotari:lh-improve-binary-license-check
May 5, 2026
Merged

[improve][build] Replace check-binary-license.sh with a Gradle task#25673
merlimat merged 1 commit into
apache:masterfrom
lhotari:lh-improve-binary-license-check

Conversation

@lhotari
Copy link
Copy Markdown
Member

@lhotari lhotari commented May 5, 2026

Motivation

Validating LICENSE/NOTICE coverage of the binary distributions used to require a manual two-step ritual: build with ./gradlew assemble, then invoke src/check-binary-license.sh against each tarball. The shell script depends on bash/tar/grep/sed, sits outside the Gradle build lifecycle, and is wired into CI as a separate post-build step.

This change ports the validation into a first-class Gradle task so:

  • a single ./gradlew checkBinaryLicense builds the tarballs (if needed) and runs the checks,
  • per-distribution invocation works (:distribution:pulsar-server-distribution:checkBinaryLicense, :distribution:pulsar-shell-distribution:checkBinaryLicense),
  • the task is configuration-cache compatible and reuses Gradle's incremental/cached behaviour,
  • the validation can be reused by any distribution module via a build-logic plugin.

Modifications

  • Add CheckBinaryLicenseTask (build-logic/conventions/src/main/kotlin/CheckBinaryLicenseTask.kt): a @CacheableTask that mirrors the legacy bash logic — enumerates bundled jars (with the same path/name exclusions), parses jar references from LICENSE/NOTICE, and runs the three checks (every non-pulsar bundled jar must appear in LICENSE; every LICENSE jar reference must be bundled; every NOTICE jar reference except checker-qual.jar must be bundled). Reads the tarball through Gradle's injected ArchiveOperations, so the task action does not capture Project and survives configuration-cache serialization. LICENSE/NOTICE jar references are matched as regex against the bundled-jar set to preserve the existing wildcard semantics that bash grep already provided (e.g. org.rocksdb.*.jar).
  • Add the pulsar.binary-license-check-conventions precompiled-script plugin in build-logic. It registers checkBinaryLicense and exposes a binaryLicenseCheck { archive = ... } extension so consumers can wire the producing tarball lazily through a Provider chain. No cross-project access — configure-on-demand safe.
  • Apply the plugin in distribution/server/build.gradle.kts and distribution/shell/build.gradle.kts, wired to serverDistTar/shellDistTar. The provider chain auto-derives the task dependency, so ./gradlew checkBinaryLicense builds the tarballs first.
  • Add a root aggregation task checkBinaryLicense in build.gradle.kts that depends on both per-distribution tasks via string-path dependsOn, mirroring the existing serverDistTar aggregation pattern (configure-on-demand safe).
  • Update .github/workflows/pulsar-ci.yaml so the existing "Check binary licenses" step in the build job invokes ./gradlew checkBinaryLicense --no-configuration-cache (matching the assemble step's CC opt-out). Remove the duplicate post-Docker-build invocation in the docker job since the build-job step already covers both distributions.
  • Document the new task in README.md next to the existing checkstyle / spotless instructions, framed around the use case of confirming that newly added/changed runtime dependencies are properly recorded.
  • Delete src/check-binary-license.sh.

Verifying this change

  • Make sure that the change passes the CI checks.

This change is already covered by existing tests, namely the CI "Check binary licenses" step, which now runs the new Gradle task end-to-end.

Verified locally:

  • ./gradlew checkBinaryLicense builds both distributions and runs the checks; both pass.
  • ./gradlew :distribution:pulsar-server-distribution:checkBinaryLicense and the shell variant work in isolation.
  • Re-running shows UP-TO-DATE on both per-distribution tasks and the root aggregation.
  • ./gradlew checkBinaryLicense --configuration-cache --configure-on-demand stores a CC entry on first run and reuses it on the second, with no CC problems reported.
  • Cross-checked against the legacy src/check-binary-license.sh on the same tarballs — both agree (both pass; an early run flagged a missing LICENSE entry, identical to what the bash script would have flagged, confirming failure-mode parity).
  • ./gradlew spotlessCheck checkstyleMain checkstyleTest passes.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Move LICENSE/NOTICE coverage validation of the binary distributions
into a build-logic Gradle task so it integrates with the build
lifecycle, depends on the producing tarballs, and no longer requires
a separate bash/tar/grep/sed invocation outside of Gradle.
@lhotari lhotari added this to the 5.0.0-M1 milestone May 5, 2026
@merlimat merlimat merged commit a1b0a4f into apache:master May 5, 2026
45 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants