Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEATURE] Support Flink 1.17 #4367

Closed
3 of 4 tasks
link3280 opened this issue Feb 19, 2023 · 0 comments
Closed
3 of 4 tasks

[FEATURE] Support Flink 1.17 #4367

link3280 opened this issue Feb 19, 2023 · 0 comments
Labels
kind:feature Feature request priority:major

Comments

@link3280
Copy link
Contributor

Code of Conduct

Search before asking

  • I have searched in the issues and found no similar issues.

Describe the feature

Flink 1.17 is in the feature-freeze phase and will be released soon, which Kyuubi should add support for it.

The SQL client module is refactored in Flink 1.17, and it isn't easy to keep the compatibility through reflection with Flink 1.16 or below. Thus I directly modified the codes to support only Flink 1.17, so that we can see the difference clearly. If we agree on the changes, we may add a new module for Flink 1.17 and keep the old one for some time.

Motivation

Adapt to the new Flink SQL client/gateway architecture.

Describe the solution

  • Merge the new Flink sessions into Kyuubi sessions.
  • Replace the LocalExecutor in the SQL client with OperationExecutor in the SQL gateway module.

Additional context

No response

Are you willing to submit PR?

  • Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
  • No. I cannot submit a PR at this time.
link3280 added a commit to link3280/kyuubi that referenced this issue Feb 19, 2023
@link3280
[KYUUBI apache#4367] Support Flink 1.17
679203a
@link3280 link3280 mentioned this issue Mar 12, 2023
Closed
4 tasks
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 10, 2023
@link3280
[KYUUBI apache#4367] Fix Flink SessionManager compatibility issue
e18b3c2
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 10, 2023
@link3280
[KYUUBI apache#4367] Remove support for Flink 1.15
00dd58b
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 10, 2023
@link3280
[KYUUBI apache#4367] Remove support for Flink 1.15
ef6f4d4
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 10, 2023
@link3280
[KYUUBI apache#4367] Flink FlinkSQLEngine capabilities with Flink 1.16
d289495
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 10, 2023
@link3280
[KYUUBI apache#4367] Fix NPE in dependencies
6a08d0b
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 11, 2023
@link3280
[KYUUBI apache#4367] Fix reflection failures
614ecc3
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 11, 2023
@link3280
[KYUUBI apache#4367] Fix NoSuchFieldException
799c938
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 11, 2023
@link3280
[KYUUBI apache#4367] Fix Flink conf dir not found
29728c0
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 11, 2023
@link3280
[KYUUBI apache#4367] Improve code style
b685ff1
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 12, 2023
@link3280
[KYUUBI apache#4367] Fix FLINK_CONF_DIR not set in ut
700ee04
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 12, 2023
@link3280
[KYUUBI apache#4367] Fix test failure in PlanOnlyStatementSuit
1a7833b
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 12, 2023
@link3280
[KYUUBI apache#4367] Fix udf not found in local flink engine tests
54cfe3b
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 13, 2023
@link3280
[KYUUBI apache#4367] Remove profile for flink-1.15
7291e27
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 13, 2023
@link3280
[KYUUBI apache#4367] Disable query id test for flink 1.16
e851d97
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 13, 2023
@link3280
[KYUUBI apache#4367] Fix flink sql gateway jar not included in local …
7e1a198 
…mode
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 13, 2023
@link3280
[KYUUBI apache#4367] Refactor Flink engine tests
47beb1a
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 14, 2023
@link3280
[KYUUBI apache#4367] Filter out non kyuubi prefixed conf in flink log…
b7d46f5 
…in engine
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 14, 2023
@link3280
[KYUUBI apache#4367] Fix local engine tagged as YARN app
f71c601
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 14, 2023
@link3280
Revert "[KYUUBI apache#4367] Filter out non kyuubi prefixed conf in f…
fd0c88d 
…link login engine"

This reverts commit b7d46f5.
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 14, 2023
@link3280
[KYUUBI apache#4367] Fix local engine tagged as YARN app
7ce1e9a
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 14, 2023
@link3280
[KYUUBI apache#4367] Improve flink version match
9b2072e
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 14, 2023
@link3280
[KYUUBI apache#4367] Use kyuubi refection tools
20e1a55
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 14, 2023
@link3280
[KYUUBI apache#4367] Improve code style
a02e01a
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 14, 2023
@link3280
[KYUUBI apache#4367] Fix FlinkProcessBuilder test error
20e9913
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 14, 2023
@link3280
[KYUUBI apache#4367] Replace vanilla reflection with kyuubi refection…
42cce7a 
… tools
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 14, 2023
@link3280
[KYUUBI apache#4367] Fix IncompatibleClassChangeError
3520a51
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 14, 2023
@link3280
[KYUUBI apache#4367] Fix IncompatibleClassChangeError
e1e5cd2
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 15, 2023
@link3280
[KYUUBI apache#4367] Remove unnecessary reflection code
33eeb37
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 15, 2023
@link3280
[KYUUBI apache#4367] Remove unnecessary variables
1c41e44
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 15, 2023
@link3280
[KYUUBI apache#4367] Remove unnecessary classes
8530aec
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 15, 2023
@link3280
[KYUUBI apache#4367] Improve code style for java.util.*
4ef8c52
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 15, 2023
@link3280
[KYUUBI apache#4367] Improve code style for java.lang.Boolean
7e36e70
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 15, 2023
@link3280
[KYUUBI apache#4367] Improve code style for error messages and iterat…
c2a23d5 
…ions
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 15, 2023
@link3280
[KYUUBI apache#4367] Improve code style for iterations
7b28eaf
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 15, 2023
@link3280
[KYUUBI apache#4367] Fix spotless
ed95060
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 15, 2023
@link3280
[KYUUBI apache#4367] Fix ambiguous reference
08f8999
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 16, 2023
@link3280
[KYUUBI apache#4367] Improve java.long.Long usage
56567fc
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 16, 2023
@link3280
[KYUUBI apache#4367] Improve code style
5c9d0aa
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 16, 2023
@link3280
[KYUUBI apache#4367] Rename delegation package to shim
23d87ba
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 16, 2023
@link3280
[KYUUBI apache#4367] Fix test failure
81a10f6
link3280 added a commit to link3280/kyuubi that referenced this issue Apr 16, 2023
@link3280
[KYUUBI apache#4367] Fix test failure
8eb4da6
pan3793 added a commit to pan3793/kyuubi that referenced this issue Apr 17, 2023
@pan3793
[KYUUBI apache#4367][DOCS][FOLLOWUP] Kyuubi requries Flink 1.16/1.17
1e64dcb
pan3793 added a commit that referenced this issue Apr 17, 2023
@pan3793
[KYUUBI #4367][DOCS][FOLLOWUP] Kyuubi requires Flink 1.16/1.17
a4cdb26 
### _Why are the changes needed?_

Kyuubi requires Flink 1.16/1.17 now

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4723 from pan3793/flink-1.17.

Closes #4367

1e64dcb [Cheng Pan] [KYUUBI #4367][DOCS][FOLLOWUP] Kyuubi requries Flink 1.16/1.17

Authored-by: Cheng Pan <chengpan@apache.org>
Signed-off-by: Cheng Pan <chengpan@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind:feature Feature request priority:major
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@link3280