Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PHOENIX-6766 Fix failure of sqlline due to conflicting jline dependency pulled from Hadoop 3.3. #1484

Closed
wants to merge 1 commit into from

Conversation

iwasakims
Copy link
Member

https://issues.apache.org/jira/browse/PHOENIX-6766

Dependency on org.jline:jline:jar:3.9.0 conflicts with org.jline:jline-*:jar:3.12.1. It causes following error on invoking sqlline. It should be excluded from transitive dependency of Hadoop (>= 3.3.0).

$ mvn clean install -DskipTests -Dhadoop.version=3.3.4 -Dhbase.version=2.4.13 -Dhbase.profile=2.4
$ bin/sqlline.py
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.phoenix.shaded.org.jline.reader.impl.completer.StringsCompleter.<init>([Lorg/apache/phoenix/shaded/org/jline/reader/Candidate;)V
        at sqlline.SqlLineOpts.setOptionCompleters(SqlLineOpts.java:160)
        at sqlline.Application.getCommandHandlers(Application.java:294)
        at sqlline.SqlLine$Config.<init>(SqlLine.java:1946)
        at sqlline.SqlLine.setAppConfig(SqlLine.java:1875)
        at sqlline.SqlLine.<init>(SqlLine.java:229)
        at sqlline.SqlLine.start(SqlLine.java:266)
        at sqlline.SqlLine.main(SqlLine.java:206)

@iwasakims
Copy link
Member Author

The stack trace implies that the constructor of StringsCompleter which is not exists in 3.9.0 was called.

compile scope dependency on org.jline:jline:jar:3.9.0 is pulled from Hadoop (after YARN-8778).

$ mvn dependency:tree -DskipTests -Dhadoop.version=3.3.4 -Dhbase.version=2.4.13 -Dhbase.profile=2.4 | grep jline
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:provided
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:provided
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:provided
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:provided
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:provided
[INFO] |  |  \- org.jline:jline:jar:3.9.0:compile
[INFO] |  \- jline:jline:jar:0.9.94:compile
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:compile
[INFO] |  |  \- jline:jline:jar:0.9.94:compile
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:compile
[INFO] |  |  \- jline:jline:jar:0.9.94:compile
[INFO]    +- org.jline:jline-terminal:jar:3.12.1:runtime
[INFO]    +- org.jline:jline-reader:jar:3.12.1:runtime
[INFO]    +- org.jline:jline-terminal-jansi:jar:3.12.1:runtime
[INFO]    +- org.jline:jline-terminal-jna:jar:3.12.1:runtime
[INFO]    \- org.jline:jline-builtins:jar:3.12.1:runtime
[INFO]       \- org.jline:jline-style:jar:3.12.1:runtime
[INFO] |  |  \- jline:jline:jar:0.9.94:compile
[INFO] |  |  \- org.jline:jline:jar:3.9.0:provided
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:provided
[INFO] |  |  \- jline:jline:jar:0.9.94:provided
[INFO] |  |  |  |  \- org.jline:jline:jar:3.9.0:compile
[INFO] |  |  |  \- jline:jline:jar:0.9.94:compile
[INFO] |     +- org.jline:jline-terminal:jar:3.12.1:runtime
[INFO] |     +- org.jline:jline-reader:jar:3.12.1:runtime
[INFO] |     +- org.jline:jline-terminal-jansi:jar:3.12.1:runtime
[INFO] |     +- org.jline:jline-terminal-jna:jar:3.12.1:runtime
[INFO] |     \- org.jline:jline-builtins:jar:3.12.1:runtime
[INFO] |        \- org.jline:jline-style:jar:3.12.1:runtime
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:compile
[INFO] |  |  \- jline:jline:jar:0.9.94:compile
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:compile
[INFO] |  |  \- jline:jline:jar:0.9.94:compile
[INFO]    +- org.jline:jline-terminal:jar:3.12.1:runtime
[INFO]    +- org.jline:jline-reader:jar:3.12.1:runtime
[INFO]    +- org.jline:jline-terminal-jansi:jar:3.12.1:runtime
[INFO]    +- org.jline:jline-terminal-jna:jar:3.12.1:runtime
[INFO]    \- org.jline:jline-builtins:jar:3.12.1:runtime
[INFO]       \- org.jline:jline-style:jar:3.12.1:runtime

org.jline:jline is bundled jar. It contains duplicate classes of org.jline:jline-*:3.12.1 (which are transitive dependencies of sqlline).

@iwasakims
Copy link
Member Author

While I saw this error on Phoenix 5.1.2, master branch looks affected too.

$ mvn dependency:tree -DskipTests -Dhadoop.version=3.3.4 -Dhbase.version=2.4.13 -Dhbase.profile=2.4 | grep jline
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:provided
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:provided
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:provided
[INFO] |  |  \- org.jline:jline:jar:3.9.0:compile
[INFO] |  \- jline:jline:jar:0.9.94:provided
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:compile
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:provided
[INFO] |  |  |  \- org.jline:jline:jar:3.9.0:compile
[INFO] |  |  \- org.jline:jline:jar:3.9.0:provided
[INFO] |  |  |  |  \- org.jline:jline:jar:3.9.0:compile
[INFO]    +- org.jline:jline-terminal:jar:3.12.1:compile
[INFO]    +- org.jline:jline-reader:jar:3.12.1:compile
[INFO]    +- org.jline:jline-terminal-jansi:jar:3.12.1:compile
[INFO]    +- org.jline:jline-terminal-jna:jar:3.12.1:compile
[INFO]    \- org.jline:jline-builtins:jar:3.12.1:compile
[INFO]       \- org.jline:jline-style:jar:3.12.1:compile

Copy link
Contributor

@stoty stoty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We certainly don't need jline from hadoop.
+1 LGTM

Copy link
Contributor

@stoty stoty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dependency is at one level lower, coming from
org.apache.hadoop:hadoop-yarn-client .

It would be more robust and readable to add the exclusion there.
(In case hadoop-yarn-client gets pulled via via another transitive dependency in the future.)

@stoty
Copy link
Contributor

stoty commented Aug 9, 2022

Also please remove the dot after the JIRA number in the commit message.

@iwasakims iwasakims changed the title PHOENIX-6766. Fix failure of sqlline due to conflicting jline dependency pulled from Hadoop 3.3. PHOENIX-6766 Fix failure of sqlline due to conflicting jline dependency pulled from Hadoop 3.3. Aug 10, 2022
@iwasakims
Copy link
Member Author

Thanks, @stoty. I updated the PR based on your comment.

Copy link
Contributor

@stoty stoty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you.
LGTM

@stoty stoty closed this Aug 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants