Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(storage): simply support trigger manual compaction and risectl #3218

Merged
merged 7 commits into from
Jun 15, 2022

Conversation

Li0k
Copy link
Contributor

@Li0k Li0k commented Jun 14, 2022

What's changed and what's your intention?

simply support trigger manual compaction

Please explain IN DETAIL what the changes are in this PR and why they are needed:

  • simply support trigger manual compaction in hummock_manager
  • add trigger manual compaction rpc
  • support trigger manual compaction for risectl

Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests

Refer to a related PR or issue link (optional)

@Li0k Li0k requested a review from hzxa21 June 14, 2022 12:19
@Li0k Li0k changed the title [WIP] feat(storage): simply support trigger manual compaction and risectl feat(storage): simply support trigger manual compaction and risectl Jun 14, 2022
@Li0k Li0k force-pushed the li0k/trigger_manual_compaction branch from b74a99b to 2a0ebe7 Compare June 14, 2022 14:44
@Li0k
Copy link
Contributor Author

Li0k commented Jun 14, 2022

I found a bug that:
After pick_compaction, compaction_status will hold the pending_task in memory. so if send_task rpc fail, the state of pending_task remains in memory and prevents subsequent compaction.

how to resolve this ?
I think that we need to cancel the pending_task status when send_task rpc fail to avoid the above , and i add some unit test for this case.

cc @skyzh @hzxa21

@codecov
Copy link

codecov bot commented Jun 14, 2022

Codecov Report

Merging #3218 (ad278e9) into main (47fb009) will decrease coverage by 0.01%.
The diff coverage is 63.80%.

@@            Coverage Diff             @@
##             main    #3218      +/-   ##
==========================================
- Coverage   73.41%   73.39%   -0.02%     
==========================================
  Files         744      745       +1     
  Lines      101665   101873     +208     
==========================================
+ Hits        74634    74769     +135     
- Misses      27031    27104      +73     
Flag Coverage Δ
rust 73.39% <63.80%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
.../src/cmd_impl/hummock/trigger_manual_compaction.rs 0.00% <0.00%> (ø)
src/ctl/src/lib.rs 4.34% <0.00%> (-1.91%) ⬇️
src/meta/src/hummock/level_handler.rs 96.15% <ø> (ø)
src/meta/src/hummock/mock_hummock_meta_client.rs 41.48% <0.00%> (-2.47%) ⬇️
src/meta/src/rpc/server.rs 0.00% <0.00%> (ø)
src/meta/src/rpc/service/hummock_service.rs 0.00% <0.00%> (ø)
src/rpc_client/src/meta_client.rs 0.00% <0.00%> (ø)
src/storage/src/hummock/hummock_meta_client.rs 0.00% <0.00%> (ø)
src/meta/src/hummock/hummock_manager.rs 86.74% <79.48%> (-0.54%) ⬇️
src/meta/src/hummock/hummock_manager_tests.rs 89.14% <85.29%> (-0.38%) ⬇️
... and 10 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@Li0k Li0k requested a review from Little-Wallace June 15, 2022 04:11
Copy link
Contributor

@skyzh skyzh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

src/meta/src/hummock/hummock_manager_tests.rs Outdated Show resolved Hide resolved
}
}

tracing::trace!(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe change the log level to info, otherwise in release mode this won't be printed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix it

@Li0k Li0k force-pushed the li0k/trigger_manual_compaction branch from 2a0ebe7 to a1582ac Compare June 15, 2022 05:44
@skyzh skyzh enabled auto-merge (squash) June 15, 2022 05:53
@skyzh skyzh merged commit 4eb6fb5 into main Jun 15, 2022
@skyzh skyzh deleted the li0k/trigger_manual_compaction branch June 15, 2022 06:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants