Skip to content
Pydra Dataflow Engine
Python Jupyter Notebook Other
Branch: master
Clone or download
satra Merge pull request #208 from djarecka/fix/usecache_nodelevel
[fix] passing wf.cache_locations and submitter.rerun
Latest commit 73ac982 Feb 24, 2020

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.binder make binder a hidden folder Aug 18, 2019
.circleci fix: add correct ssh-key fingerprint Jan 4, 2020
.github adding contributing and zenodo files; small changes to the issue temp… Jan 31, 2020
ci adding a new travis run with singularity Nov 22, 2019
docs fix: update requirements Jan 17, 2020
pydra removing asserts for time of execution, for slurm it doesn't work ver… Feb 24, 2020
tools MAINT: Add script to keep min-requirements synced with setup.cfg Jul 29, 2019
tutorial fixing tutorial tests Jan 30, 2020
.coveragerc CI: Add coverage/codecov Oct 9, 2018
.et add .et file Nov 23, 2019
.gitattributes MAINT: Versioneer Oct 9, 2018
.gitignore enh: add etelemetry support Nov 23, 2019
.pre-commit-config.yaml update pre-commit Nov 9, 2019
.travis.yml adding a new travis run with singularity Nov 22, 2019
.zenodo.json Update .zenodo.json Feb 3, 2020
CONTRIBUTING.md Update CONTRIBUTING.md Feb 3, 2020
LICENSE maint: license + pr template Jul 19, 2019
MANIFEST.in fix: add schema to setup process Dec 9, 2018
Makefile FIX: Specify black options in pyproject.toml Jul 17, 2019
README.md update README [skip ci] Jan 17, 2020
codecov.yml changing codecov, adding setups for project and patch Nov 22, 2019
long_description.rst MAINT: Move to setup.cfg + pyproject.toml Jul 17, 2019
min-requirements.txt fix: update requirements Jan 17, 2020
pydra_logo.png
pyproject.toml FIX: Specify black options in pyproject.toml Jul 17, 2019
setup.cfg trying to fix binder/jupyter notebook Jan 31, 2020
setup.py STY: black setup.py Jul 17, 2019
versioneer.py sty: black; remove workingdir Mar 14, 2019

README.md

Pydra

A simple dataflow engine with scalable semantics.

Build Status CircleCI codecov

The goal of pydra is to provide a lightweight Python dataflow engine for DAG construction, manipulation, and distributed execution.

Feature list:

  1. Python 3.7+ using type annotation and attrs
  2. Composable dataflows with simple node semantics. A dataflow can be a node of another dataflow.
  3. splitter and combiner provides many ways of compressing complex loop semantics
  4. Cached execution with support for a global cache across dataflows and users
  5. Distributed execution (presently via ConcurrentFutures and SLURM, more on their way)

[API Documentation]

Tutorial

This tutorial will walk you through the main concepts of Pydra! Binder

Please note that mybinder times out after an hour.

Installation

pip install pydra

Developer installation

Pydra requires Python 3.7+. To install in developer mode:

git clone git@github.com:nipype/pydra.git
cd pydra
pip install -e .[dev]

It is also useful to install pre-commit:

pip install pre-commit
pre-commit
You can’t perform that action at this time.