Skip to content

[MINOR][BUILD] Drop retired oro dependency - #57492

Open
ybapat wants to merge 1 commit into
apache:masterfrom
ybapat:issue-57209-drop-oro
Open

[MINOR][BUILD] Drop retired oro dependency#57492
ybapat wants to merge 1 commit into
apache:masterfrom
ybapat:issue-57209-drop-oro

Conversation

@ybapat

@ybapat ybapat commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Remove the oro (Jakarta ORO) dependency from Apache Spark.

  • Remove the <oro.version>2.0.8</oro.version> property from the root pom.xml
  • Remove the oro dependency block from common/utils/pom.xml
  • Remove the oro/2.0.8//oro-2.0.8.jar entry from dev/deps/spark-deps-hadoop-3-hive-2.3
  • Remove the oro:oro entry from LICENSE-binary

Why are the changes needed?

The oro (Jakarta ORO) project was officially retired approximately 16 years ago. It was added to Spark 12 years ago as an optional runtime dependency of Apache Ivy (see the inline comment: "oro is needed by ivy, but only listed as an optional dependency"). Modern Ivy (2.5.x, which Spark currently uses) falls back to java.util.regex when oro is absent, so the jar is no longer needed.

No Spark source code directly imports org.apache.oro.*; the jar served only as an optional Ivy runtime detail.

Relates to GitHub issue #57209.

Does this PR introduce any user-facing change?

No. This is a build-level dependency removal with no runtime impact on Spark users.

How was this patch tested?

The dependency manifest (dev/deps/spark-deps-hadoop-3-hive-2.3) and LICENSE-binary were updated to remove all traces of the oro artifact. No Spark source files import org.apache.oro, so no code changes or tests are required.

This PR was created with the assistance of Claude (AI). Disclosed per Apache Arrow/Spark contribution guidelines.

@uros-b uros-b left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The removal's stated rationale is empirically false for the pinned Ivy 2.5.3, whose GlobPatternMatcher hard-requires org.apache.oro, and Spark's --packages path always builds glob exclude rules; so this turns every Maven-coordinate resolution into a NoClassDefFoundError. To drop oro, Spark must first migrate MavenUtils off GlobPatternMatcher (e.g. RegexpPatternMatcher / ExactPatternMatcher) and prove resolution plus the existing MavenUtilsSuite pass without oro on the classpath.

@ybapat
ybapat force-pushed the issue-57209-drop-oro branch from 2e89503 to c6485ac Compare July 28, 2026 17:31
@ybapat

ybapat commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review — you're right. GlobPatternMatcher from Ivy 2.5.3 does hard-require org.apache.oro at runtime, so the removal alone would have caused a NoClassDefFoundError on any --packages path that builds glob exclude rules.

I've updated the PR to first migrate MavenUtils off GlobPatternMatcher:

  • import org.apache.ivy.plugins.matcher.GlobPatternMatcherExactPatternMatcher
  • ivySettings.addMatcher(new GlobPatternMatcher)new ExactPatternMatcher
  • ivySettings.getMatcher("glob")getMatcher("exact")

The substitution is safe because all patterns Spark constructs for exclusion rules are either exact strings (e.g. "scala-library") or "*". Ivy's AbstractPatternMatcher special-cases "*" as match-all regardless of the concrete matcher type (PatternMatcher.ANY_EXPRESSION), so ExactPatternMatcher is behaviourally identical for Spark's usage.

The five changed files are now in a single commit. Happy to add a MavenUtilsSuite test covering the --exclude-packages / createExclusion path without oro on the classpath if that would help confidence.

@ybapat
ybapat requested a review from uros-b July 28, 2026 18:19
@ybapat
ybapat force-pushed the issue-57209-drop-oro branch from c6485ac to f78eebb Compare July 29, 2026 00:00
…expPatternMatcher

oro (Apache ORO) has been unmaintained since 2010. Its sole consumer in
Spark is MavenUtils.buildIvySettings, which registers GlobPatternMatcher
with Ivy. GlobPatternMatcher delegates to oro's GlobCompiler to convert
glob patterns to regex; removing oro without migrating away from it causes
a NoClassDefFoundError on any --packages path.

MavenUtils passes patterns like "spark-core_*" (glob suffix) to
createExclusion, so ExactPatternMatcher is not a safe drop-in.
RegexpPatternMatcher (ships with Ivy, no external deps) is:

- Replace GlobPatternMatcher with RegexpPatternMatcher.
- Add globToRegex helper that splits on "*" and rejoins with ".*", using
  Pattern.quote on each literal segment so "." in groupIds is not treated
  as a regex wildcard.
- Update createExclusion to convert coordinates through globToRegex before
  storing them in ArtifactId / ModuleId, and look up "regexp" instead of "glob".

With that in place, oro is removed from pom.xml, common/utils/pom.xml,
the dependency lock file, and LICENSE-binary.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@ybapat
ybapat force-pushed the issue-57209-drop-oro branch from f78eebb to 715b265 Compare July 29, 2026 00:00
@pan3793

pan3793 commented Aug 6, 2026

Copy link
Copy Markdown
Member

seems not lucky. @ybapat, are you still working on this?

[info] MavenUtilsSuite:
...
[info] - excludes works correctly *** FAILED *** (16 milliseconds)
[info]   "[\Qa\E]" did not equal "[a]" (MavenUtilsSuite.scala:121)
[info]   Analysis:
[info]   "[\Qa\E]" -> "[a]"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants