Rest API for Ledger Offloading #1639
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Implemented for both V1 and V2 topic name formats. API takes a message ID, up to which the broker will try to offload messages. It returns the message ID of the first message in the topic which has not been offloaded. This patch also adds basic support for setting the Offloader implementation in the broker (needed for testing). Subsequent patches will make this configurable through ServiceConfiguration.
|
retest this please // integration test timeout |
sijie
reviewed
Apr 25, 2018
There was a problem hiding this comment.
@ivankelly : what would be the expected behavior when triggering offloading?
if you are issuing an admin operation, are you blocking on waiting the result? from the implementation, it seems to be that behavior.
I think the behavior is not that useful. I would expect users would like a behavior like:
- users issue trigger an offload rest request. if broker is already is offloading, return "offloading inprogress"; if broker is not offloading, scheduling an offload, return "offloading scheduled/inprogress" and return that last message id as some sort of offload-task-id. so later on that the user can use
offload-task-id(message id) to query if the offload is completed.
|
@sijie ya, I thought about doing it like that when I was finishing this version. Compaction is done like that. Will change this. |
|
cool thanks @ivankelly |
One for triggering and one for getting the current status.
|
retest this please // PersistentFailoverE2ETest.testSimpleConsumerEventsWithoutPartition |
merlimat
added
the
type/feature
|
@sijie pinging. This is ready for review again. |
|
@ivankelly yeah. will review. I was holding it after Matteo cut the 2.0 branch. |
sijie
approved these changes
May 2, 2018
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implemented for both V1 and V2 topic name formats. Each persistent
topic has a /offload endpoint. A PUT to this endpoint will trigger offloading.
This put takes a message ID as parameter, which is the highest message
ID that will be offloaded. If there is already an offload in progress, the put
will return a HTTP 409 response, signifying that there is already an offload
in progress. A GET on the endpoint will return the status of a previous
triggered offload, if any.
This patch also adds basic support for setting the Offloader
implementation in the broker (needed for testing). Subsequent patches
will make this configurable through ServiceConfiguration.
Master Issue: #1511