Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support submission timeout for yarn application manager and get the applicationInfo in-memory #4719

Closed
wants to merge 1 commit into from

Conversation

turboFei
Copy link
Member

@turboFei turboFei commented Apr 17, 2023

Why are the changes needed?

To prevent the create batch operation stuck.

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before make a pull request

@turboFei turboFei changed the title Yarn app mgr Support submission timeout for yarn application manager and get the applicationInfo in-memory Apr 17, 2023
@codecov-commenter
Copy link

codecov-commenter commented Apr 17, 2023

Codecov Report

Merging #4719 (964a943) into master (17514a3) will increase coverage by 0.01%.
The diff coverage is 74.28%.

❗ Current head 964a943 differs from pull request most recent head 1b0da93. Consider uploading reports for the commit 1b0da93 to get more accurate results

@@             Coverage Diff              @@
##             master    #4719      +/-   ##
============================================
+ Coverage     58.03%   58.05%   +0.01%     
  Complexity       13       13              
============================================
  Files           580      580              
  Lines         32270    32241      -29     
  Branches       4307     4303       -4     
============================================
- Hits          18728    18717      -11     
+ Misses        11739    11729      -10     
+ Partials       1803     1795       -8     
Impacted Files Coverage Δ
...ala/org/apache/kyuubi/operation/LaunchEngine.scala 89.65% <ø> (ø)
...pache/kyuubi/engine/YarnApplicationOperation.scala 57.33% <53.84%> (-1.40%) ⬇️
...g/apache/kyuubi/operation/BatchJobSubmission.scala 78.57% <85.00%> (+2.91%) ⬆️
.../kyuubi/operation/KyuubiApplicationOperation.scala 43.47% <100.00%> (ø)
.../apache/kyuubi/server/api/v1/BatchesResource.scala 70.66% <100.00%> (+2.47%) ⬆️

... and 15 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@@ -104,7 +101,6 @@ class BatchJobSubmission(

override protected def currentApplicationInfo: Option[ApplicationInfo] = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this function has side-effects, should add ()

Copy link
Member

@pan3793 pan3793 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, please fill the PR description to mention the background

@turboFei
Copy link
Member Author

https://github.com/apache/kyuubi/actions/runs/4728413557/jobs/8389933543?pr=4719

BatchRestApiSuite:
- basic batch rest client
- basic batch rest client with uploading resource file
- basic batch rest client with invalid user
- spnego batch rest client *** FAILED ***
  0 did not equal 1 (BatchRestApiSuite.scala:166)

always update

do not fetch if not exists

comments

unknown

side effects

comments

fix app start time

refactor
@turboFei turboFei self-assigned this Apr 20, 2023
@turboFei turboFei added this to the v1.8.0 milestone Apr 20, 2023
@turboFei turboFei closed this in a59e27a Apr 20, 2023
@turboFei turboFei deleted the yarn_app_mgr branch April 20, 2023 12:14
pan3793 pushed a commit that referenced this pull request Apr 20, 2023
…r and get the applicationInfo in-memory

To prevent the create batch operation stuck.

- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4719 from turboFei/yarn_app_mgr.

Closes #4719

1b0da93 [fwang12] check timeout

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
@pan3793 pan3793 modified the milestones: v1.8.0, v1.7.1 Apr 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants