-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
endpoints to get and delete minion task metadata #8551
endpoints to get and delete minion task metadata #8551
Conversation
5645ce9
to
8ca9729
Compare
8ca9729
to
89bf429
Compare
Codecov Report
@@ Coverage Diff @@
## master #8551 +/- ##
=============================================
- Coverage 70.77% 25.71% -45.07%
=============================================
Files 1681 1675 -6
Lines 87957 87894 -63
Branches 13320 13314 -6
=============================================
- Hits 62252 22599 -39653
- Misses 21354 63135 +41781
+ Partials 4351 2160 -2191
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
...c/main/java/org/apache/pinot/controller/helix/core/minion/PinotHelixTaskResourceManager.java
Outdated
Show resolved
Hide resolved
...c/main/java/org/apache/pinot/controller/helix/core/minion/PinotHelixTaskResourceManager.java
Outdated
Show resolved
Hide resolved
return _pinotHelixTaskResourceManager.getTaskMetadataByTable(taskType, tableNameWithType); | ||
} | ||
|
||
@DELETE |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
helix deletes task metadata after sometime, right ? Why provide an endpoint to delete it? Have you verified with helix that this ok? and there is no race condition?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Helix task framework is not aware of the task metadata here, e.g. RealtimToOfflineTask keeps a timestamp as watermark in a ZNode under /Propertystore/MINION_TASK_METADATA/
to manage its progress.
0d6edf1
to
dd2d22c
Compare
dd2d22c
to
8d43d8b
Compare
add two minion tasks related endpoints to allow GET and DELETE /tasks/{taskType}/{tableNameWithType}/metadata
Release Notes
Added 2 http endpoints for getting and deleting the minion task metadata, like the watermark of RealtimeToOfflineTask.