Data aggregation C++ API #248
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## develop #248 +/- ##
===========================================
- Coverage 80.85% 80.15% -0.70%
===========================================
Files 51 54 +3
Lines 3008 3326 +318
===========================================
+ Hits 2432 2666 +234
- Misses 576 660 +84
|
get_list_length() returns type int (not size_t). This seems more consistent.
mellis13
force-pushed
the
data_aggregation_pipeline
branch
from
c29b2f8
to
ac17d2c
Compare
retrieval from aggregation lists. Tests now also check that dataset names match.
error handling is easier to understand. This required some changes to PipelineReply ojbect and an additional vector of command pointers in the unordered pipeline execution.
ashao
requested changes
Apr 26, 2022
There was a problem hiding this comment.
Really nice work here @mellis13. You and @billschereriii already anticipated/addressed the things that I thought might have been issues. The main change that I'm suggesting is to add/expose pop functionality to the list in the event that you have multiple consumers of the aggregated list.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a draft PR for the data aggregation API in C++. A draft PR has been put up so parallel work can be done multi-threading the dataset retrieval from cluster shards. CI tests will likely fail until all functionality is done.
The following items still need to be addressed before final review and merge of this PR:
rename_list()rename_list()copy_list()copy_list()Redis::run_via_unordered_pipelines(). This was implemented inRedisClusterand theRedisversion of this function will be extremely easy since all keys are colocated on one shard.Client::_get_dataset_list_rangeto functionalize the code for reuse in other areas (e.g.Client.get_dataset()RedisCluster::run_via_unordered_pipelines()). This will be evaluated in a separate ticket, and this PR should not be merged before performance data is available from that.