dagrun.py

This is a lightweight task/DAG runner that can be added as a standalone script, using only the Python standard library.

Requires Python 3.9+.

Installation

You can install this as a standalone script within your project by either copying dagrun.py from this repository, or by installing with pip install and using the dagrun-install script to export dagrun.py as an executable with a shebang, which can be invoked as python ./dagrun.py, or as ./dagrun.py on Unix-like systems.

Dev setup

Requires GNU Make.

git clone https://git.sr.ht/~wintershadows/dagrun.py
cd dagrun.py
make setup

For push access with make git-push:

git remote set-url origin 'git@git.sr.ht:~wintershadows/dagrun.py'
git remote add github-mirror 'git@github.com:gwerbin/dagrun.py.git'

To-do / roadmap

• Tests.

• PyPI upload.

• Better docs/instructions.

• Think more about ExecTask interface. Too clunky?

• Add PythonTask for running Python code

• Add other specialized tasks for Python projects?

• Add support for outputs and dependencies as well as other ways to check for success/failure beyond exit code.

• Support multiple DAGs in the same file.

• Confirm that dagrun-install actually works right on Windows.

• Coherent versioning system.

• YAML-based file format in addition to Python scripts.

• Some kind of logging and nicer-looking output.

• Set README in Sourcehut: https://man.sr.ht/git.sr.ht/#setting-a-custom-readme

• Error or warning if adding a duplicate graph node.