Release 0.1#111
Merged
GoldenBadger merged 169 commits intomasterfrom Oct 22, 2017
Merged
Conversation
Merge (#9) into develop branch
* Added base workspace * Add missing closing bracket in email * Fix version and maintainers email
* Add first draft of contributing file, a lot to be discussed * Update branching policy * Add git-flow branching policy link * Update branching policy * Add squash and automated testing to contributing.md * Add commenting guidlines to Contributing.md * Add style guide to contributing.md * Code review comments
- Add info about building - Add basic MapReduce proto
Includes error_chain setup code and a very basic test + function example.
Initial commit of libcerberus.
V1 Master -> Worker protobuf
Remove information about squashing on github from the contributing.md
Since Map and Reduce may have multiple outputs per input, we cannot return from the functions. Instead we have to use an Emit trait. Structs implementing this trait contain a reference to in-memory data structure where key-value pairs created by the map-reduce are stored. Emit::emit takes ownership of this key-value pair and moves it to said data structure.
Define Emit trait and basic MultiMapEmitter implementation.
Initial commit of master
Add documentation for the Emit trait and MultiMapEmitter.
VecEmitter is an emitter which emits key-value pairs directly into a Vec as tuples. This operates differently to MultiMapEmitter on the inside, as it does not automatically group together duplicate keys.
Add VecEmitter.
Mark Cargo.lock as autogenerated to hide it when diffing.
Change proto build process to use cargo
Change Rust GRPC implementation
Document protoc dependency.
When writing the `Map` trait I realised that `Emit` hides the types it emits as part of the implementation. Really, they should be public so that users writing `Map` and `Reduce` implementations have control over the types the output.
cli: Namespacing of protobufs (#103)
master: Namespacing of protobufs
cli: Added short flags
Add a polling loop for workers
Fix clippy lint errors in worker poller
Added doc describing the requirements of the sprint 1 payload.
GoldenBadger
requested review from
ConorGriffin37,
DarraghGriffin,
RyanConnell and
VoyTechnology
Change demo to only use String.
Change workers to have a unique output directory in /tmp Change reduce file names to be the intermediate key for the reduce
Fix getting count of workers in cluster Fix queue size when a map reduce is in progress Fix build Fix not checking operation flag Fix not setting total map tasks Remove unnecessary scheduling in progress logic Fix wrong error Up the frequency of polling. Update master version number Split output directory by map and reduce Only attempt to process the next map reduce if the queue is not empty.
GoldenBadger
force-pushed
the
release-0.1
branch
from
729761b to
d55c283
Compare
ConorGriffin37
approved these changes
Oct 22, 2017
RyanConnell
approved these changes
Oct 22, 2017
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.