From 0f08e74051fc280790ea3e8960360299a3dd8be6 Mon Sep 17 00:00:00 2001 From: Xiang Fu Date: Sat, 25 Apr 2026 22:01:44 -0700 Subject: [PATCH 1/2] Bypass Dependabot for ClassLoaderTest's pinned commons-io 2.11.0 fixture ClassLoaderTest pins commons-io 2.11.0 as a test fixture (asserts the literal jar filename and contents to validate plugin realm isolation). The same coordinate is consumed at a newer version in the top-level pom.xml, so a repo-wide Dependabot ignore rule is not viable -- it would block legitimate commons-io upgrades. With the previous declaration, Dependabot kept opening PRs that bumped the pinned 2.11.0 reference (e.g. #18331), forcing manual reverts. Refactor the fixture fetch to a two-step pipeline that hides the coordinate from Dependabot's Maven updater while preserving Maven's normal resolution semantics: - maven-dependency-plugin:get with commons-io:commons-io:2.11.0 resolves the jar through the standard resolution chain (local cache, configured mirrors, authenticated repos, SHA-1 checksum validation). Dependabot scans structured groupId+artifactId+version triples in // blocks, not single-string values in plugin configuration. - maven-antrun-plugin:run copies the resolved jar from ${settings.localRepository} into target/test-classes, preserving the literal filename commons-io-2.11.0.jar that ClassLoaderTest.assemblyBasedRealm asserts on. Also update .github/dependabot.yml to drop the now-obsolete caveat about commons-io. Verified locally: ClassLoaderTest's 5 cases (assemblyBasedRealm, classRealms, classicPluginClassloader, limitedPluginRealm, unlimitedPluginRealm) all pass on a clean target dir with the local commons-io 2.11.0 cache deleted (forcing fresh resolution). Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/dependabot.yml | 17 ++++++---- pinot-spi/pom.xml | 74 ++++++++++++++++++++++++++++++++++++------ 2 files changed, 74 insertions(+), 17 deletions(-) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index cb571983d463..14f5c971e60e 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -30,14 +30,17 @@ updates: # These artifacts are pinned in pinot-spi/pom.xml (maven-dependency-plugin artifactItems) because # ClassLoaderTest asserts plugin realm isolation against known artifact contents. Dependabot's # `versions` filter is matched against the *target* version of an update, so we use range filters - # (">= ") to block every bump to a newer version for these coordinates. + # (">= ") to block every bump to a newer version for these coordinates. The rules apply + # repo-wide by coordinate, which is fine here because pinot-dropwizard / pinot-yammer / + # com.yammer.metrics:metrics-core are not referenced anywhere else. # - # Caveat: these rules apply repo-wide by coordinate. pinot-dropwizard / pinot-yammer / - # com.yammer.metrics:metrics-core are not referenced anywhere else, so locking them is safe. - # commons-io:commons-io is also consumed in the top-level pom.xml at a newer version, so it is - # intentionally NOT locked here; Dependabot may still open PRs that also modify the hardcoded - # 2.11.0 in pinot-spi/pom.xml — reviewers must revert that part manually (see the DO NOT BUMP - # comment next to the 2.11.0 entry in pinot-spi/pom.xml). + # The ClassLoaderTest also pins commons-io 2.11.0, but that coordinate is consumed at a newer + # version elsewhere (top-level pom.xml), so we cannot block it here. Instead, the fixture is + # resolved via maven-dependency-plugin's `dependency:get` goal in pinot-spi/pom.xml using a + # single-string commons-io:commons-io:2.11.0 parameter — Dependabot's + # Maven updater scans structured groupId+artifactId+version triples inside , + # , and blocks, not single-string values in plugin + # configuration, so no ignore rule for commons-io is needed. - dependency-name: "org.apache.pinot:pinot-dropwizard" versions: [">= 0.10.1"] - dependency-name: "org.apache.pinot:pinot-yammer" diff --git a/pinot-spi/pom.xml b/pinot-spi/pom.xml index fca66c6cc345..2d03ead2ac10 100644 --- a/pinot-spi/pom.xml +++ b/pinot-spi/pom.xml @@ -62,10 +62,10 @@ DO NOT BUMP the hardcoded versions below. They are intentionally pinned test fixtures used by ClassLoaderTest to validate plugin realm isolation against known artifact contents. Corresponding - ignore rules live in .github/dependabot.yml, but Dependabot cannot always distinguish these locked - references from other uses of the same coordinate elsewhere in the repo (e.g., commons-io is also - consumed at a newer version in the top-level pom.xml), so Dependabot may still open PRs that touch - this file. Reviewers: reject any change to the versions below. + ignore rules live in .github/dependabot.yml. (commons-io 2.11.0, which is also consumed at a newer + version in the top-level pom.xml, is fetched via the separate `fetch-classloadertest-commons-io-fixture` + / `copy-classloadertest-commons-io-fixture` executions below to keep its coordinate off Dependabot's + scanner without blocking legitimate commons-io upgrades.) --> generate-test-resources @@ -94,12 +94,6 @@ shaded ${project.build.testOutputDirectory}/plugins/pinot-shaded-yammer - - commons-io - commons-io - 2.11.0 - ${project.build.testOutputDirectory}/plugins/assemblybased-pinot-plugin - com.yammer.metrics metrics-core @@ -109,6 +103,36 @@ + + fetch-classloadertest-commons-io-fixture + + generate-test-resources + + get + + + commons-io:commons-io:2.11.0 + false + + unpack-pinot-plugins + generate-test-resources + + run + + + + + + + + + + org.apache.maven.plugins maven-surefire-plugin From 074e944f4a8b17ea8936e654a4c88c95523354a3 Mon Sep 17 00:00:00 2001 From: Xiang Fu Date: Sun, 26 Apr 2026 00:49:35 -0700 Subject: [PATCH 2/2] Apply the dependency:get bypass uniformly to all ClassLoaderTest fixtures Address review feedback on #18335: extend the dependency:get + antrun copy/unzip pattern to every pinned ClassLoaderTest fixture, not just commons-io. Before: pinot-dropwizard, pinot-yammer (x2 shaded), and metrics-core were still declared as blocks under copy-pinot-plugins and unpack-pinot-plugins, with `versions: [">= "]` ignore rules in .github/dependabot.yml suppressing Dependabot bumps. Those rules worked because none of those coordinates were used elsewhere -- but the asymmetric handling left a footgun: any future use of one of those coordinates at a different version elsewhere in the repo would silently get blocked from updates by the repo-wide ignore rule. After: all five fixtures (including the previously-handled commons-io) are routed through the same pipeline: - maven-dependency-plugin:get with single-string parameters resolves each unique GAV+classifier combination through Maven's normal chain (cache, mirrors, authenticated repos, SHA-1 validation). Dependabot's Maven updater does not scan single-string values, so the pinned coordinates are invisible to it. - A single maven-antrun-plugin:run execution stages the resolved jars from ${settings.localRepository} into target/test-classes/plugins/ with the literal filenames the test asserts on, and unzips the non-shaded pinot-yammer 0.10.0 into the assemblybased-pinot-plugin classes/ subdirectory. The .github/dependabot.yml fixture-specific ignore rules are removed since no blocks remain for these coordinates. Verified locally: ClassLoaderTest's 5 cases pass on a fully empty target dir AND empty local cache for all four fixture coordinates, confirming the resolve+stage pipeline is idempotent and self-bootstrapping. Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/dependabot.yml | 30 ++----- pinot-spi/pom.xml | 189 ++++++++++++++++++++--------------------- 2 files changed, 101 insertions(+), 118 deletions(-) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 14f5c971e60e..ab14ec838238 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -25,28 +25,14 @@ updates: schedule: interval: "daily" open-pull-requests-limit: 20 - ignore: - # Locked test resources for pinot-spi/src/test/java/org/apache/pinot/spi/plugin/ClassLoaderTest.java. - # These artifacts are pinned in pinot-spi/pom.xml (maven-dependency-plugin artifactItems) because - # ClassLoaderTest asserts plugin realm isolation against known artifact contents. Dependabot's - # `versions` filter is matched against the *target* version of an update, so we use range filters - # (">= ") to block every bump to a newer version for these coordinates. The rules apply - # repo-wide by coordinate, which is fine here because pinot-dropwizard / pinot-yammer / - # com.yammer.metrics:metrics-core are not referenced anywhere else. - # - # The ClassLoaderTest also pins commons-io 2.11.0, but that coordinate is consumed at a newer - # version elsewhere (top-level pom.xml), so we cannot block it here. Instead, the fixture is - # resolved via maven-dependency-plugin's `dependency:get` goal in pinot-spi/pom.xml using a - # single-string commons-io:commons-io:2.11.0 parameter — Dependabot's - # Maven updater scans structured groupId+artifactId+version triples inside , - # , and blocks, not single-string values in plugin - # configuration, so no ignore rule for commons-io is needed. - - dependency-name: "org.apache.pinot:pinot-dropwizard" - versions: [">= 0.10.1"] - - dependency-name: "org.apache.pinot:pinot-yammer" - versions: [">= 0.10.1"] - - dependency-name: "com.yammer.metrics:metrics-core" - versions: [">= 2.1.6"] + # No `ignore` rules needed for the ClassLoaderTest pinned test fixtures + # (pinot-dropwizard, pinot-yammer, commons-io, com.yammer.metrics:metrics-core). + # Those fixtures are resolved by pinot-spi/pom.xml via maven-dependency-plugin's + # `dependency:get` goal using single-string groupId:artifactId:version + # parameters — Dependabot's Maven updater scans structured groupId+artifactId+version + # triples inside , , and blocks, not single-string + # values in plugin configuration, so the pinned coordinates are invisible + # to Dependabot and no ignore rule is required. - package-ecosystem: "npm" directory: "/pinot-controller/src/main/resources" diff --git a/pinot-spi/pom.xml b/pinot-spi/pom.xml index 2d03ead2ac10..50cb2c700dd5 100644 --- a/pinot-spi/pom.xml +++ b/pinot-spi/pom.xml @@ -50,114 +50,92 @@ org.apache.maven.plugins maven-dependency-plugin - - - copy-pinot-plugins - + DO NOT replace these versions with ${project.version}, as the test depends on the presence + (and absence) of specific classes in those exact jars. DO NOT BUMP the hardcoded versions + below — they are deliberately frozen test fixtures. + + Why dependency:get with a single-string instead of blocks? + Dependabot's Maven updater scans , , and blocks for + structured groupId+artifactId+version triples; it does NOT parse single-string + ... values inside plugin configuration. Using here + gave Dependabot a coordinate to "upgrade" and produced repeated noise PRs (e.g. #18331). + For coordinates that are not consumed elsewhere in the repo, a `versions: [">= "]` + ignore rule in .github/dependabot.yml would suppress those PRs, but commons-io is also + consumed at a newer version in the top-level pom.xml, so a repo-wide ignore would block + legitimate upgrades. To keep the mechanism uniform across all the pinned fixtures (and + defensive against any future shared-coordinate situation), we route every fixture through + dependency:get + antrun copy/unzip from ${settings.localRepository}. + + dependency:get goes through Maven's normal resolution chain (local cache, configured + mirrors, authenticated repos, SHA-1 checksum validation), so offline/mirrored builds keep + working and artifact integrity is verified. + + The companion antrun execution `stage-classloadertest-fixtures` copies the resolved jars + from ${settings.localRepository} into target/test-classes/plugins/ with the literal + filenames ClassLoaderTest asserts on (see CodeSource.getLocation().getPath().endsWith(...) + checks in the test). + --> + + + fetch-classloadertest-fixture-pinot-dropwizard-shaded generate-test-resources - copy + get - - - org.apache.pinot - pinot-dropwizard - 0.10.0 - shaded - ${project.build.testOutputDirectory}/plugins/pinot-dropwizard - - - org.apache.pinot - pinot-yammer - 0.10.0 - shaded - ${project.build.testOutputDirectory}/plugins/pinot-yammer - - - org.apache.pinot - pinot-yammer - 0.10.0 - shaded - ${project.build.testOutputDirectory}/plugins/pinot-shaded-yammer - - - com.yammer.metrics - metrics-core - 2.1.5 - ${project.build.testOutputDirectory}/plugins/assemblybased-pinot-plugin - - + org.apache.pinot:pinot-dropwizard:0.10.0:jar:shaded + false - fetch-classloadertest-commons-io-fixture - + fetch-classloadertest-fixture-pinot-yammer-shaded generate-test-resources get - commons-io:commons-io:2.11.0 + org.apache.pinot:pinot-yammer:0.10.0:jar:shaded false - unpack-pinot-plugins - + fetch-classloadertest-fixture-pinot-yammer + generate-test-resources + + get + + + org.apache.pinot:pinot-yammer:0.10.0 + false + + + + fetch-classloadertest-fixture-metrics-core generate-test-resources - unpack + get - - - org.apache.pinot - pinot-yammer - 0.10.0 - ${project.build.testOutputDirectory}/plugins/assemblybased-pinot-plugin/classes - - + com.yammer.metrics:metrics-core:2.1.5 + false + + + + fetch-classloadertest-fixture-commons-io + generate-test-resources + + get + + + commons-io:commons-io:2.11.0 + false @@ -167,16 +145,15 @@ maven-antrun-plugin - copy-classloadertest-commons-io-fixture - generate-test-resources @@ -184,9 +161,29 @@ + + + + + + + + + + + todir="${project.build.testOutputDirectory}/plugins/assemblybased-pinot-plugin"/> + + +