Support for callback when records in hybrid log become read-only #133
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR allows users to register a user-defined callback, which is invoked when records in the hybrid log become read-only. It can be used for various reasons:
(1) to perform data write-through to an external store
(2) to perform stream processing on the change log created by the key-value store
How it Works
The FASTER log accessor now supports the
IObservable<IFasterScanIterator<Key, Value>>
interface. This means you canSubscribe
to FASTER log, in order to receive (as push calls) updates whenever records enter the read-only region of the hybrid log:The parameter
LogObserver
is an instance of typeIObserver<IFasterScanIterator<Key, Value>>
. User gets push-based callbacks via theOnNext
method of the observer. Each callback provides an iterator for a batch of records that have just become read-only, in bulk. The user can then perform operations such as stream processing or write-through to the remote database.Here is an example of the callback: