YAML based adage
Switch branches/tags
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
docs
examples
tests
yadage
.bumpversion.cfg
.gitignore
.travis.yml
CODE_OF_CONDUCT.md
CONTRIBUTING.md
Dockerfile
LICENSE
README.md
pytest.ini
setup.py
yadagedocker.sh
yadagelocaldocker.sh

README.md

yadage - yaml based adage

arXiv DOI PyPI version Build Status Code Health Coverage Status Documentation Status

This package reads and executes workflows adhering to the workflow JSON schemas defined at https://github.com/diana-hep/cap-schemas such as the onces stored in the community repository https://github.com/lukasheinrich/yadage-workflows. For executing the individual steps it mainly uses the packtivity python bindings provided by https://github.com/diana-hep/packtivity.

Example Workflow

cat << 'EOF' > wflow.yml
stages:
- name: hello_world
  dependencies: [init]
  scheduler:
    scheduler_type: singlestep-stage
    parameters:
      name: {step: init, output: name}
      outputfile: '{workdir}/hello_world.txt'
    step:
      process:
        process_type: 'string-interpolated-cmd'
        cmd: 'echo Hello my Name is {name} | tee {outputfile}'
      publisher:
        publisher_type: 'frompar-pub'
        outputmap:
          outputfile: outputfile
      environment:
        environment_type: 'docker-encapsulated'
        image: busybox
EOF

You can try this workflow via

yadage-run workdir wflow.yml -p name="John Doe"

For more thorough examples, please see the documentation

Possible Backends:

Yadage can run on various backends such as multiprocessing pools, ipython clusters, or celery clusters. If human intervention is needed for certain steps, it can also be run interactively.

Published versions of related packages (main dependencies of yadage)

package version
packtivity PyPI version
yadage-schemas PyPI version
adage PyPI version