Skip to content
Pydra Dataflow Engine
Python Jupyter Notebook Other
Branch: master
Clone or download
satra Merge pull request #208 from djarecka/fix/usecache_nodelevel
[fix] passing wf.cache_locations and submitter.rerun
Latest commit 73ac982 Feb 24, 2020


Type Name Latest commit message Commit time
Failed to load latest commit information.
.binder make binder a hidden folder Aug 18, 2019
.circleci fix: add correct ssh-key fingerprint Jan 4, 2020
.github adding contributing and zenodo files; small changes to the issue temp… Jan 31, 2020
ci adding a new travis run with singularity Nov 22, 2019
docs fix: update requirements Jan 17, 2020
pydra removing asserts for time of execution, for slurm it doesn't work ver… Feb 24, 2020
tools MAINT: Add script to keep min-requirements synced with setup.cfg Jul 29, 2019
tutorial fixing tutorial tests Jan 30, 2020
.coveragerc CI: Add coverage/codecov Oct 9, 2018
.et add .et file Nov 23, 2019
.gitattributes MAINT: Versioneer Oct 9, 2018
.gitignore enh: add etelemetry support Nov 23, 2019
.pre-commit-config.yaml update pre-commit Nov 9, 2019
.travis.yml adding a new travis run with singularity Nov 22, 2019
.zenodo.json Update .zenodo.json Feb 3, 2020 Update Feb 3, 2020
LICENSE maint: license + pr template Jul 19, 2019 fix: add schema to setup process Dec 9, 2018
Makefile FIX: Specify black options in pyproject.toml Jul 17, 2019 update README [skip ci] Jan 17, 2020
codecov.yml changing codecov, adding setups for project and patch Nov 22, 2019
long_description.rst MAINT: Move to setup.cfg + pyproject.toml Jul 17, 2019
min-requirements.txt fix: update requirements Jan 17, 2020
pyproject.toml FIX: Specify black options in pyproject.toml Jul 17, 2019
setup.cfg trying to fix binder/jupyter notebook Jan 31, 2020 STY: black Jul 17, 2019 sty: black; remove workingdir Mar 14, 2019


A simple dataflow engine with scalable semantics.

Build Status CircleCI codecov

The goal of pydra is to provide a lightweight Python dataflow engine for DAG construction, manipulation, and distributed execution.

Feature list:

  1. Python 3.7+ using type annotation and attrs
  2. Composable dataflows with simple node semantics. A dataflow can be a node of another dataflow.
  3. splitter and combiner provides many ways of compressing complex loop semantics
  4. Cached execution with support for a global cache across dataflows and users
  5. Distributed execution (presently via ConcurrentFutures and SLURM, more on their way)

[API Documentation]


This tutorial will walk you through the main concepts of Pydra! Binder

Please note that mybinder times out after an hour.


pip install pydra

Developer installation

Pydra requires Python 3.7+. To install in developer mode:

git clone
cd pydra
pip install -e .[dev]

It is also useful to install pre-commit:

pip install pre-commit
You can’t perform that action at this time.