Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BACKPORT] [KYUUBI #933] Enhance the detection mechanism for engine startup #1364

Closed
wants to merge 2 commits into from

Conversation

zhouyifan279
Copy link
Contributor

@zhouyifan279 zhouyifan279 commented Nov 12, 2021

Why are the changes needed?

Sub task of #1361.
#1176 is based on #933 . In order to backport #1176, #933 has to be backported first.

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before make a pull request

charlesy6 and others added 2 commits November 11, 2021 17:55
### 1. Description

EngineRef use exists znode to determine whether engine started successfully or not(use the last znode), there are inconsistencies in this. So, take an idea to improve the detection mechanism by adding the started sessionid to the znone. `EngineRef` will check the sessionid in a loop after a engine started.

### 2. Here are two options
**Option I, add sesssionid to the data of znode**
![image](https://user-images.githubusercontent.com/86483005/128994276-5c9c7839-0a21-4253-8989-110b88ac0e74.png)

**Option II, add sessionid to the path of znode**
![image](https://user-images.githubusercontent.com/86483005/129527282-46f7f726-e110-4d87-9272-dda9b9f325bf.png)

### 3. Solution

In order to be consistent with the design of server znode, prefer to choose _Option II_ .

**Demo**
```
/kyuubi
- serviceUri=bigdata:10009;version=1.3.0-SNAPSHOT;sequence=0000000000
/kyuubi_USER/test
- serviceUri=bigdata:39869;version=1.3.0-SNAPSHOT;session=8178069f-0b22-4d06-b1c0-908094769397;sequence=0000000000]
```

Closes apache#935 from timothy65535/ky-933.

Closes apache#933

ab5d6d8 [timothy65535] add issue id
d21cb99 [timothy65535] compatible with back
568acf2 [timothy65535] fix error
3db8ba7 [timothy65535] move EngineRef get to Discovery
827f378 [timothy65535] rename get to getEngineBySessionId
8822931 [timothy65535] update session id using internal
ab06adf [timothy65535] improve EngineRef get logic
6a1cc39 [timothy65535] update session version
c94b6e5 [timothy65535] rename sessionId to createSessionId
0cc6aae [timothy65535] update ha conf
d24a152 [timothy65535] update ha conf
1a67c86 [timothy65535] fix initialize sql suite
b3b502d [timothy65535] [KYUUBI apache#933] Enhance the detection mechanism for engine startup

Authored-by: timothy65535 <timothy65535@163.com>
Signed-off-by: Kent Yao <yao@apache.org>
(cherry picked from commit 32b18ba)
@zhouyifan279 zhouyifan279 changed the title [BACKPORT] [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on [WIP][BACKPORT] [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on Nov 12, 2021
@codecov-commenter
Copy link

Codecov Report

Merging #1364 (8cd735c) into branch-1.3 (89f1cfb) will increase coverage by 0.02%.
The diff coverage is 72.41%.

Impacted file tree graph

@@               Coverage Diff                @@
##             branch-1.3    #1364      +/-   ##
================================================
+ Coverage         80.53%   80.55%   +0.02%     
  Complexity           11       11              
================================================
  Files               146      146              
  Lines              5390     5396       +6     
  Branches            643      643              
================================================
+ Hits               4341     4347       +6     
- Misses              669      670       +1     
+ Partials            380      379       -1     
Impacted Files Coverage Δ
...org/apache/kyuubi/ha/client/ServiceDiscovery.scala 45.52% <57.89%> (+0.90%) ⬆️
...cala/org/apache/kyuubi/ctl/ServiceControlCli.scala 73.40% <100.00%> (ø)
...la/org/apache/kyuubi/ha/HighAvailabilityConf.scala 98.27% <100.00%> (+0.12%) ⬆️
...ain/scala/org/apache/kyuubi/engine/EngineRef.scala 76.11% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 89f1cfb...8cd735c. Read the comment docs.

@zhouyifan279 zhouyifan279 changed the title [WIP][BACKPORT] [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on [BACKPORT] [KYUUBI #933] Enhance the detection mechanism for engine startup Nov 12, 2021
@zhouyifan279
Copy link
Contributor Author

zhouyifan279 commented Nov 12, 2021

@ulysses-you @pan3793 @timothy65535 please help to do the review

@ulysses-you
Copy link
Contributor

thanks, merging to branch-1.3

ulysses-you added a commit that referenced this pull request Nov 12, 2021
…sm for engine startup

### _Why are the changes needed?_
Sub task of #1361.
#1176 is based on #933 . In order to backport #1176, #933 has to be backported first.

### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #1364 from zhouyifan279/1361.

Closes #1364

Closes #933

8cd735c [ulysses-you] [BACKPORT] [KYUUBI #1176] InvalidACL appears in the engine when zookeeper acl is turned on
42d0adc [timothy65535] [KYUUBI #933] Enhance the detection mechanism for engine startup

Lead-authored-by: timothy65535 <timothy65535@163.com>
Co-authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: ulysses-you <ulyssesyou@apache.org>
@ulysses-you ulysses-you added this to the v1.3.2 milestone Nov 12, 2021
@zhouyifan279 zhouyifan279 deleted the 1361 branch November 12, 2021 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants