Skip to content

Conversation

@wForget
Copy link
Member

@wForget wForget commented Nov 2, 2023

Why are the changes needed?

When debugging SQL tasks, we may need to replace the output table with a test table to avoid overwriting online data.
So I implemented a MockOutputTable rule to replace the output table with a mock table which ends with _mock_${System.currentTimeMillis()}.

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before make a pull request

Was this patch authored or co-authored using generative AI tooling?

No

@wForget wForget changed the title Add mock output table rule [KYUUBI #5605] Add mock output table rule Nov 2, 2023
@codecov-commenter
Copy link

Codecov Report

Merging #5605 (7b7739f) into master (591250c) will decrease coverage by 0.28%.
Report is 11 commits behind head on master.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #5605      +/-   ##
============================================
- Coverage     61.69%   61.42%   -0.28%     
  Complexity       23       23              
============================================
  Files           603      603              
  Lines         35670    35669       -1     
  Branches       4869     4869              
============================================
- Hits          22005    21908      -97     
- Misses        11281    11381     +100     
+ Partials       2384     2380       -4     

see 29 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@wForget wForget requested a review from ulysses-you November 2, 2023 06:59
case i @ InsertIntoStatement(r: HiveTableRelation, _, _, _, _, _, _)
if !isMockTable(r.tableMeta) =>
val newTable = mockTable(r.tableMeta, suffix, true)
i.copy(table = DDLUtils.readHiveTable(newTable))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should think about more. For example, the mocked table is not existed, how can we do insert into ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mockTable method will create mock table if it does not exist.

@wForget wForget changed the title [KYUUBI #5605] Add mock output table rule [WIP][KYUUBI #5605] Add mock output table rule Nov 3, 2023
@wForget wForget marked this pull request as draft November 3, 2023 06:39
@github-actions
Copy link

Thanks for the PR! This PR is being closed due to inactivity. This isn't a judgement on the merit of the PR in any way. If this is still an issue with the latest version of Kyuubi, please reopen it and ask a committer to remove the Stale tag!

Thank you for using Kyuubi!

@github-actions github-actions bot added the Stale label Feb 12, 2024
@github-actions github-actions bot closed this Feb 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants