Skip to content

Exclude httpcore5-h2, httpcore5-reactive, httpclient5 from SQL bundle to fix jar hell#5400

Merged
ahkcs merged 2 commits intoopensearch-project:feature/mustang-ppl-integrationfrom
ahkcs:fix/jarhell-httpcore5-h2
May 1, 2026
Merged

Exclude httpcore5-h2, httpcore5-reactive, httpclient5 from SQL bundle to fix jar hell#5400
ahkcs merged 2 commits intoopensearch-project:feature/mustang-ppl-integrationfrom
ahkcs:fix/jarhell-httpcore5-h2

Conversation

@ahkcs
Copy link
Copy Markdown
Collaborator

@ahkcs ahkcs commented May 1, 2026

Summary

Adds three missing jar exclusions to plugin/build.gradle's bundlePlugin block so the SQL plugin install no longer trips JarHell when analytics-engine (in versions that ship httpcore5-h2/httpcore5-reactive/httpclient5) is co-installed.

Failure being fixed

Reported by @peterzhuamazon against the latest feature-build-opensearch distribution:

java.lang.IllegalStateException: failed to load plugin opensearch-sql due to jar hell
Caused by: java.lang.IllegalStateException: jar hell!
    class: org.apache.hc.core5.http2.H2ConnectionException
    jar1: <sql>/httpcore5-h2-5.4.jar
    jar2: <analytics-engine>/httpcore5-h2-5.4.jar

Root cause

The existing exclusion list (added by #5302 to share JARs via the analytics-engine extendedPlugins classloader) covered httpcore5-5*.jar, but the httpcore5-* family ships in four variants — only the first was excluded:

JAR Previously excluded?
httpcore5-5.4.jar yes (httpcore5-5*.jar)
httpcore5-h2-5.4.jar no
httpcore5-reactive-5.4.jar no
httpclient5-5.6.jar no

The first uncovered variant trips install; the next two would trip immediately after.

Change

plugin/build.gradle — three new exclusions next to the existing httpcore5-5*.jar:

exclude 'httpcore5-h2-*.jar'
exclude 'httpcore5-reactive-*.jar'
exclude 'httpclient5-*.jar'

End-to-end verification

Built an OpenSearch 3.7 distro and ran bin/opensearch-plugin install for both bundle variants, with opensearch-job-scheduler + a representative analytics-engine (one that ships httpcore5-h2) pre-installed.

Without the fix:

-> Failed installing file:///tmp/.../sql-without-fix.zip
Caused by: java.lang.IllegalStateException: jar hell!
class: org.apache.hc.client5.http.AbstractClientContextBuilder
jar1: .../analytics-engine/httpclient5-5.6.jar
jar2: .../.installing-.../httpclient5-5.6.jar

↑ Reproduces Peter's stack (different class — JarHell catches whichever duplicate it iterates first; same root cause).

With the fix:

-> Installed opensearch-sql with folder name opensearch-sql

--- installed plugins:
analytics-engine
opensearch-job-scheduler
opensearch-sql

Static check against the currently vendored analytics-engine-3.7.0-SNAPSHOT.zip: zero overlapping JARs (by filename and by base name), confirming no class-level JarHell can fire for that pair.

Caveats

  • This PR is the minimum fix to clear Peter's reported failure. The analytics-engine build that's about to be vendored tomorrow may bundle additional JARs (e.g., avatica-metrics, antlr4-runtime, icu4j, error_prone_annotations) that overlap with SQL — those would require additional exclusions. We deliberately stop here rather than over-excluding, because excluding a JAR that the new vendored zip doesn't ship would break SQL at runtime (ClassNotFoundException).
  • After the libs update lands tomorrow, re-run the same install verification against the new vendored zip and add exclusions for any newly-overlapping JARs.
  • The longer-term fix is the AnalyticsFrontEndExtension SPI work in Add AnalyticsFrontEndExtension SPI + AnalyticsServices bundle for analytics-engine frontend integration OpenSearch#21449, which makes analytics-engine an optional dependency entirely.

The analytics-engine plugin bundles httpcore5-h2-5.4.jar,
httpcore5-reactive-5.4.jar, and httpclient5-5.6.jar in addition to
httpcore5-5.4.jar. The existing bundlePlugin exclusion list only covered
httpcore5-5*.jar, so the other three variants leaked into the SQL bundle
and tripped JarHell at install time when analytics-engine was already
installed:

  jar hell!
    class: org.apache.hc.core5.http2.H2ConnectionException
    jar1: <sql>/httpcore5-h2-5.4.jar
    jar2: <analytics-engine>/httpcore5-h2-5.4.jar

Adding the missing exclusions matches the existing pattern of stripping
shared jars provided by the analytics-engine extendedPlugins classloader.

Signed-off-by: Kai Huang <ahkcs@amazon.com>
Per Peter's review on opensearch-project#5400: flag the three new httpcore5/httpclient5
exclusions (and ideally the entire bundlePlugin exclusion block) for
removal once analytics-engine becomes an optional dependency via the
AnalyticsFrontEndExtension SPI in opensearch-project/OpenSearch#21449.

Signed-off-by: Kai Huang <ahkcs@amazon.com>
@ahkcs ahkcs merged commit e113b7f into opensearch-project:feature/mustang-ppl-integration May 1, 2026
36 checks passed
@ahkcs ahkcs deleted the fix/jarhell-httpcore5-h2 branch May 1, 2026 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintenance Improves code quality, but not the product

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants