IterativeReduce

jpatanooga edited this page Oct 8, 2012 · 2 revisions

Overview

  • A simple abstraction in the same concept as MapReduce but for parallel iterative algorithms
  • similar in nature to the AllReduce primitive or some of Spark's concepts
  • Involves writing a ComputableMaster and ComputableWorker
  • Work is done in the worker and then aggregated every so often in the master

ComputableWorker

  • does the work in parallel on batches of data

ComputableMaster

  • Aggregates partial results from workers, computes a bit, and then returns an aggregation to the workers
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.