Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
D5.1: Turn the Python prototypes for tree exploration into production code, integrate to Sage . #107
MapReduce is a classical programming model for distributed computations where one maps a function on a large data set and use a reduce function to summarize all the produced information. A use case that occurs often e.g. in combinatorics is to have a data sets that is described by a recursion tree, and is too big to be expanded in memory. Instances include counting the number of elements in the data set, or collecting some statistics on them.
A prototype distributed implementation of this programming model had been written in 2010-2014 for SageMath, using multiple processes on a single machine and work-stealing for load balancing. In this deliverable, we have turned this prototype into production code and integrated it into the SageMath distribution.
This work was presented at the journée du groupe de travail LaMHA at Université Pierre et Marie Curie on November the 26th of 2016. The slides give an overview of the motivations, algorithm, and implementation.