Compmake is an unobstrusive Python library that provides Make--like facilities to a Python application, including:
- Minimal effort job management and parallelization (multiple CPU on a single host, cluster computing using SGE, and experimental support for cloud computing using Multyvac).
- Caching of temporary results: you can interrupt your program and restart it without losing what was already computed.
- Failure tolerance: if a job fails, other jobs that do not depend on it continue to be executed.
- A console for inspecting failures and partial completion,
with familiar commands such as
Please see the manual at:
Please report any problem using Github's issue tracker at
Your tax dollars at work! Compmake's development was supported by the National Science Foundation in the National Robotics Initiative program under grant #1405259.
Screencast 1: Compmake overview
Screencast 2: Compmake basic usage
Screencast 3: Compmake + Multyvac
Screencast 4: Compmake + SGE
Some animations of the job graph
Robustness to job failure
Simple dynamic jobs
Dynamic jobs and recursive parallel executions
- v3.5 - Added
whycommand --- compact error visualization. Removed color effects which might not be suitable for all console types.
- v3.4.1 - Bug fix; experimental multyvac_sync
- v3.4.0 - Changed the way that dynamic job IDs are generated to avoid a race condition in a corner case.
- v3.3.7 - Bug fix.
- v3.3.6 - Experimental Multyvac backend.