[MINOR][BUILD] Drop retired oro dependency - #57492
Conversation
uros-b
left a comment
There was a problem hiding this comment.
The removal's stated rationale is empirically false for the pinned Ivy 2.5.3, whose GlobPatternMatcher hard-requires org.apache.oro, and Spark's --packages path always builds glob exclude rules; so this turns every Maven-coordinate resolution into a NoClassDefFoundError. To drop oro, Spark must first migrate MavenUtils off GlobPatternMatcher (e.g. RegexpPatternMatcher / ExactPatternMatcher) and prove resolution plus the existing MavenUtilsSuite pass without oro on the classpath.
2e89503 to
c6485ac
Compare
|
Thanks for the detailed review — you're right. I've updated the PR to first migrate
The substitution is safe because all patterns Spark constructs for exclusion rules are either exact strings (e.g. The five changed files are now in a single commit. Happy to add a |
c6485ac to
f78eebb
Compare
…expPatternMatcher oro (Apache ORO) has been unmaintained since 2010. Its sole consumer in Spark is MavenUtils.buildIvySettings, which registers GlobPatternMatcher with Ivy. GlobPatternMatcher delegates to oro's GlobCompiler to convert glob patterns to regex; removing oro without migrating away from it causes a NoClassDefFoundError on any --packages path. MavenUtils passes patterns like "spark-core_*" (glob suffix) to createExclusion, so ExactPatternMatcher is not a safe drop-in. RegexpPatternMatcher (ships with Ivy, no external deps) is: - Replace GlobPatternMatcher with RegexpPatternMatcher. - Add globToRegex helper that splits on "*" and rejoins with ".*", using Pattern.quote on each literal segment so "." in groupIds is not treated as a regex wildcard. - Update createExclusion to convert coordinates through globToRegex before storing them in ArtifactId / ModuleId, and look up "regexp" instead of "glob". With that in place, oro is removed from pom.xml, common/utils/pom.xml, the dependency lock file, and LICENSE-binary. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
f78eebb to
715b265
Compare
|
seems not lucky. @ybapat, are you still working on this? |
What changes were proposed in this pull request?
Remove the
oro(Jakarta ORO) dependency from Apache Spark.<oro.version>2.0.8</oro.version>property from the rootpom.xmlorodependency block fromcommon/utils/pom.xmloro/2.0.8//oro-2.0.8.jarentry fromdev/deps/spark-deps-hadoop-3-hive-2.3oro:oroentry fromLICENSE-binaryWhy are the changes needed?
The
oro(Jakarta ORO) project was officially retired approximately 16 years ago. It was added to Spark 12 years ago as an optional runtime dependency of Apache Ivy (see the inline comment: "oro is needed by ivy, but only listed as an optional dependency"). Modern Ivy (2.5.x, which Spark currently uses) falls back tojava.util.regexwhenorois absent, so the jar is no longer needed.No Spark source code directly imports
org.apache.oro.*; the jar served only as an optional Ivy runtime detail.Relates to GitHub issue #57209.
Does this PR introduce any user-facing change?
No. This is a build-level dependency removal with no runtime impact on Spark users.
How was this patch tested?
The dependency manifest (
dev/deps/spark-deps-hadoop-3-hive-2.3) andLICENSE-binarywere updated to remove all traces of theoroartifact. No Spark source files importorg.apache.oro, so no code changes or tests are required.