New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEATURE] Kyuubi proxy spark on kubernetes driver pod ui #3420
Comments
we're planning similar feature for our product, but using svc object of spark-driver pod is our approach. we think this is more reliable than ip address in the zookeeper. |
Thanks for your reply. This issue aims to help Kyuubi to have the ability to expose Spark UI On Kubernetes, so as to more conveniently implement Spark On Kubernetes solutions without relying on external solutions. Of course, we also welcome other solutions that can help us to implement Access UI. |
### _Why are the changes needed?_ Kyuubi Server Proxy Engine UI ### _How was this patch tested?_ - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [x] Add screenshots for manual tests if appropriate ![截屏2023-06-06 10 35 54](https://github.com/apache/kyuubi/assets/52876270/ecbc33aa-11dd-418f-bfef-19aad9e7ea39) - [ ] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request Closes #4795 from zwangsheng/KYUUBI_3420. Closes #3420 079dc1c [zwangsheng] fix frontend unit test case 6e71b45 [Cheng Pan] fix cf7ca51 [Cheng Pan] Update kyuubi-server/src/main/scala/org/apache/kyuubi/server/api/v1/ApiRootResource.scala 9a91d62 [Cheng Pan] polish a5dcfae [zwangsheng] fix 5d4a8c2 [zwangsheng] Rebase 71d22fc [zwangsheng] fix 3b0152f [zwangsheng] [KYUUBI #3420][UI] Proxy Engnie UI Lead-authored-by: zwangsheng <2213335496@qq.com> Co-authored-by: Cheng Pan <chengpan@apache.org> Co-authored-by: Cheng Pan <pan3793@gmail.com> Signed-off-by: Cheng Pan <chengpan@apache.org>
Code of Conduct
Search before asking
Describe the feature
Help proxy spark on kubernetes cluster deploy-mode spark ui.
Motivation
When Spark On Kubernetes with cluster deploy-mode, spark driver runs in the pods, using the Kubernetes network. Users cannot directly access the spark ui.
Related to Spark On Kubernetes Doc[2], you need to
kubectl port-forward
for each spark application, which is very inconvenient.So related to Kyuubi Issue 2681[1], we design using kyuubi to help port forward spark ui.
Describe the solution
See more detail with Google Doc https://docs.google.com/document/d/1FOMELnyAN9JSKeYi3qSgcVukvj8IUzYk7KF0eW5CNc8/edit#
Additional context
#2681
SubTask
Are you willing to submit PR?
The text was updated successfully, but these errors were encountered: