[FLINK-33358][sql] Fix Flink SQL Client fail to start in Flink on YARN #23629

PrabhuJoseph · 2023-10-31T07:57:25Z

What is the purpose of the change

Moves the flink sql client jar precedence before the Hadoop classpath to avoid conflicts with the older jline version from the Hadoop classpath.

Brief change log

Changed the order of sql client jar and hadoop classpath in sql-client.sh

Verifying this change

Launched the Flink SQL Session and verified with Flink sql examples.

This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no

flinkbot · 2023-10-31T08:02:40Z

CI report:

2e3b8fc Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

@flinkbot run azure re-run the last Azure build

Samrat002 · 2023-10-31T12:16:46Z

@flinkbot run azure

PrabhuJoseph · 2023-10-31T16:08:19Z

@Samrat002 @1996fanrui Could you review this patch when you have time? Thanks.

Samrat002

Thank you Prabhu for the patch.
Changes looks good to me 🚀

1996fanrui

In my humble opinion, this change is fine. But I don't know much about the details of flink sql, and I'm not sure whether this change introduces additional risks. It would be better if there are flink sql experts to help and confirm.

Hi @lsyldliu , would you mind helping take a look this PR in your free time? thanks~

lsyldliu · 2023-11-01T03:19:02Z

In my humble opinion, this change is fine. But I don't know much about the details of flink sql, and I'm not sure whether this change introduces additional risks. It would be better if there are flink sql experts to help and confirm.

Hi @lsyldliu , would you mind helping take a look this PR in your free time? thanks~

cc @fsk119

PrabhuJoseph · 2023-11-09T05:54:58Z

@fsk119 Please review this patch when you have time. Thanks.

ferenc-csaky

I think it makes sense for the SQL client JAR to take precedence over the Hadoop CP, so in general any deps that is shaded into the SQL client JAR will be picked up instead whatever is on the Hadoop CP.

Although these kind of errors are likely to depend on the environment, how the Hadoop CP is setup.

PrabhuJoseph · 2023-11-24T03:56:50Z

@1996fanrui Could you help in committing this change. Thanks.

rmoff · 2024-01-30T14:46:57Z

I am hitting the same problem, with Flink 1.18.1.

Setup

# Unpack clean 1.18.1 download
cd ~/flink
tar xf flink-1.18.1-bin-scala_2.12.tgz

# Unpack clean Hadoop download
cd ~/hadoop
tar xf hadoop-3.3.2.tar.gz

# Set `HADOOP_CLASSPATH` per instructions at 
# https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/connectors/table/hive/overview/#dependencies
export HADOOP_CLASSPATH=`~/hadoop/hadoop-3.3.2/bin/hadoop classpath`

Check env

❯ echo $HADOOP_CLASSPATH
/Users/rmoff/hadoop/hadoop-3.3.2/etc/hadoop:/Users/rmoff/hadoop/hadoop-3.3.2/share/hadoop/common/lib/*:/Users/rmoff/hadoop/hadoop-3.3.2/share/hadoop/common/*:/Users/rmoff/hadoop/hadoop-3.3.2/share/hadoop/hdfs:/Users/rmoff/hadoop/hadoop-3.3.2/share/hadoop/hdfs/lib/*:/Users/rmoff/hadoop/hadoop-3.3.2/share/hadoop/hdfs/*:/Users/rmoff/hadoop/hadoop-3.3.2/share/hadoop/mapreduce/*:/Users/rmoff/hadoop/hadoop-3.3.2/share/hadoop/yarn:/Users/rmoff/hadoop/hadoop-3.3.2/share/hadoop/yarn/lib/*:/Users/rmoff/hadoop/hadoop-3.3.2/share/hadoop/yarn/*

Run SQL Client

❯ ./bin/sql-client.sh
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/rmoff/flink/flink-1.18.1/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/rmoff/hadoop/hadoop-3.3.2/share/hadoop/common/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

                                   ▒▓██▓██▒
                               ▓████▒▒█▓▒▓███▓▒
                            ▓███▓░░        ▒▒▒▓██▒  ▒
                          ░██▒   ▒▒▓▓█▓▓▒░      ▒████
                          ██▒         ░▒▓███▒    ▒█▒█▒
                            ░▓█            ███   ▓░▒██
                              ▓█       ▒▒▒▒▒▓██▓░▒░▓▓█
                            █░ █   ▒▒░       ███▓▓█ ▒█▒▒▒
                            ████░   ▒▓█▓      ██▒▒▒ ▓███▒
                         ░▒█▓▓██       ▓█▒    ▓█▒▓██▓ ░█░
                   ▓░▒▓████▒ ██         ▒█    █▓░▒█▒░▒█▒
                  ███▓░██▓  ▓█           █   █▓ ▒▓█▓▓█▒
                ░██▓  ░█░            █  █▒ ▒█████▓▒ ██▓░▒
               ███░ ░ █░          ▓ ░█ █████▒░░    ░█░▓  ▓░
              ██▓█ ▒▒▓▒          ▓███████▓░       ▒█▒ ▒▓ ▓██▓
           ▒██▓ ▓█ █▓█       ░▒█████▓▓▒░         ██▒▒  █ ▒  ▓█▒
           ▓█▓  ▓█ ██▓ ░▓▓▓▓▓▓▓▒              ▒██▓           ░█▒
           ▓█    █ ▓███▓▒░              ░▓▓▓███▓          ░▒░ ▓█
           ██▓    ██▒    ░▒▓▓███▓▓▓▓▓██████▓▒            ▓███  █
          ▓███▒ ███   ░▓▓▒░░   ░▓████▓░                  ░▒▓▒  █▓
          █▓▒▒▓▓██  ░▒▒░░░▒▒▒▒▓██▓░                            █▓
          ██ ▓░▒█   ▓▓▓▓▒░░  ▒█▓       ▒▓▓██▓    ▓▒          ▒▒▓
          ▓█▓ ▓▒█  █▓░  ░▒▓▓██▒            ░▓█▒   ▒▒▒░▒▒▓█████▒
           ██░ ▓█▒█▒  ▒▓▓▒  ▓█                █░      ░░░░   ░█▒
           ▓█   ▒█▓   ░     █░                ▒█              █▓
            █▓   ██         █░                 ▓▓        ▒█▓▓▓▒█░
             █▓ ░▓██░       ▓▒                  ▓█▓▒░░░▒▓█░    ▒█
              ██   ▓█▓░      ▒                    ░▒█▒██▒      ▓▓
               ▓█▒   ▒█▓▒░                         ▒▒ █▒█▓▒▒░░▒██
                ░██▒    ▒▓▓▒                     ▓██▓▒█▒ ░▓▓▓▓▒█▓
                  ░▓██▒                          ▓░  ▒█▓█  ░░▒▒▒
                      ▒▓▓▓▓▓▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒░░▓▓  ▓░▒█░

    ______ _ _       _       _____  ____  _         _____ _ _            _  BETA
   |  ____| (_)     | |     / ____|/ __ \| |       / ____| (_)          | |
   | |__  | |_ _ __ | | __ | (___ | |  | | |      | |    | |_  ___ _ __ | |_
   |  __| | | | '_ \| |/ /  \___ \| |  | | |      | |    | | |/ _ \ '_ \| __|
   | |    | | | | | |   <   ____) | |__| | |____  | |____| | |  __/ | | | |_
   |_|    |_|_|_| |_|_|\_\ |_____/ \___\_\______|  \_____|_|_|\___|_| |_|\__|

        Welcome! Enter 'HELP;' to list all available commands. 'QUIT;' to exit.

Command history file path: /Users/rmoff/.flink-sql-history



Exception in thread "main" org.apache.flink.table.client.SqlClientException: Could not read from command line.
        at org.apache.flink.table.client.cli.CliClient.getAndExecuteStatements(CliClient.java:221)
        at org.apache.flink.table.client.cli.CliClient.executeInteractive(CliClient.java:179)
        at org.apache.flink.table.client.cli.CliClient.executeInInteractiveMode(CliClient.java:121)
        at org.apache.flink.table.client.cli.CliClient.executeInInteractiveMode(CliClient.java:114)
        at org.apache.flink.table.client.SqlClient.openCli(SqlClient.java:169)
        at org.apache.flink.table.client.SqlClient.start(SqlClient.java:118)
        at org.apache.flink.table.client.SqlClient.startClient(SqlClient.java:228)
        at org.apache.flink.table.client.SqlClient.main(SqlClient.java:179)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.table.client.config.SqlClientOptions
        at org.apache.flink.table.client.cli.parser.SqlClientSyntaxHighlighter.highlight(SqlClientSyntaxHighlighter.java:59)
        at org.jline.reader.impl.LineReaderImpl.getHighlightedBuffer(LineReaderImpl.java:3633)
        at org.jline.reader.impl.LineReaderImpl.getDisplayedBufferWithPrompts(LineReaderImpl.java:3615)
        at org.jline.reader.impl.LineReaderImpl.redisplay(LineReaderImpl.java:3554)
        at org.jline.reader.impl.LineReaderImpl.doCleanup(LineReaderImpl.java:2340)
        at org.jline.reader.impl.LineReaderImpl.cleanup(LineReaderImpl.java:2332)
        at org.jline.reader.impl.LineReaderImpl.readLine(LineReaderImpl.java:626)
        at org.apache.flink.table.client.cli.CliClient.getAndExecuteStatements(CliClient.java:194)
        ... 7 more

Shutting down the session...
done.

rmoff · 2024-01-30T14:58:17Z

I can confirm that using the fix in this PR resolves the issue for me.

PrabhuJoseph · 2024-01-30T17:00:04Z

Thanks @rmoff for the verification. @dannycranmer Could you please assist with reviewing this patch? Thank you.

fsk119

LGTM

PrabhuJoseph · 2024-02-01T04:35:36Z

Thanks @fsk119

tchivs · 2024-04-25T02:25:23Z

Sure enough it can be used

[FLINK-33358][sql] Fix Flink SQL Client fail to start in Flink on YARN

2e3b8fc

Samrat002 approved these changes Oct 31, 2023

View reviewed changes

1996fanrui reviewed Nov 1, 2023

View reviewed changes

ferenc-csaky approved these changes Nov 22, 2023

View reviewed changes

fsk119 approved these changes Feb 1, 2024

View reviewed changes

fsk119 merged commit fa364c7 into apache:master Feb 1, 2024

flinkbot added component=Deployment/YARN component=TableSQL/Client labels Apr 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FLINK-33358][sql] Fix Flink SQL Client fail to start in Flink on YARN #23629

[FLINK-33358][sql] Fix Flink SQL Client fail to start in Flink on YARN #23629

PrabhuJoseph commented Oct 31, 2023

flinkbot commented Oct 31, 2023 •

edited

Samrat002 commented Oct 31, 2023

PrabhuJoseph commented Oct 31, 2023

Samrat002 left a comment

1996fanrui left a comment

lsyldliu commented Nov 1, 2023

PrabhuJoseph commented Nov 9, 2023

ferenc-csaky left a comment

PrabhuJoseph commented Nov 24, 2023

rmoff commented Jan 30, 2024

rmoff commented Jan 30, 2024

PrabhuJoseph commented Jan 30, 2024

fsk119 left a comment

PrabhuJoseph commented Feb 1, 2024

tchivs commented Apr 25, 2024

[FLINK-33358][sql] Fix Flink SQL Client fail to start in Flink on YARN #23629

[FLINK-33358][sql] Fix Flink SQL Client fail to start in Flink on YARN #23629

Conversation

PrabhuJoseph commented Oct 31, 2023

What is the purpose of the change

Brief change log

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

flinkbot commented Oct 31, 2023 • edited

CI report:

Samrat002 commented Oct 31, 2023

PrabhuJoseph commented Oct 31, 2023

Samrat002 left a comment

Choose a reason for hiding this comment

1996fanrui left a comment

Choose a reason for hiding this comment

lsyldliu commented Nov 1, 2023

PrabhuJoseph commented Nov 9, 2023

ferenc-csaky left a comment

Choose a reason for hiding this comment

PrabhuJoseph commented Nov 24, 2023

rmoff commented Jan 30, 2024

Setup

Check env

Run SQL Client

rmoff commented Jan 30, 2024

PrabhuJoseph commented Jan 30, 2024

fsk119 left a comment

Choose a reason for hiding this comment

PrabhuJoseph commented Feb 1, 2024

tchivs commented Apr 25, 2024

flinkbot commented Oct 31, 2023 •

edited