Please cite this package as follows if you use it in your work:
@misc{DistributedWorkflow,
author = {Dastur, Firoozeh and Zeyen, Max and Rahn, Mirko},
title = {DistributedWorkflow.jl - A Julia interface to a distributed
task-based workflow management system},
year = {2024},
month = {January},
howpublished = {\url{https://github.com/FiroozehDastur/DistributedWorkflow.jl}}
}
> **Note**
Detailed instructions on the installation will follow soon.
- Spack - Getting Started
- Spack - Basic Usage
- add instructions to install this package
> **Note**
Detailed instructions on the usage, as well as more examples, will follow soon.
Before starting a Julia session, please set the following environment variable:
export GSPC_APPLICATION_SEARCH_PATH=$HOME/.distributedworkflow/workflows
Next, when testing the application locally set the hostname as follows:
hostname > <path-to-a-nodefile>
Start a Julia session and load the package with using DistributedWorkflow and initiate the connection by using the following function:
initiate_connection()
Assuming that you have a workflow Petri net stored in an accessible location, use compile_workflow(<workflow-name>.xpnet, "/path/to/build/location") function to compile the workflow.
Next, start the client using client(<number-of-workers>, "/path/to/nodefile", "rif-strategy")
As a next step, it is advisible to create a script with your workflow configuration (see the executor files in the examples folder), and submit your application using the submit_workflow() function.
Once your application runs through, the output files will be stored in your desired output location.
- A small example to create a Petri net, compile it, start agent and run the application locally.
- A list of all the api functions plus the description in a table.
- Serializer agnostic, for details see (add the file in the docs that explains the serialiser part).
- Reduced complexity in deploying your parallel application.
- Localised testing of workflow, before launching expensive cluster resources.
- Write your own
xpnetfile and compile your workflow using thecompile_workflow()function. - You could also, generate a Petri net in Julia using
PetriNet()and generate thexpnetfile from the Petri net usingworkflow_generator()before compiling it. - Visualise Petri net within the Julia REPL.
- At the moment, this package is only efficient and recommended for long running processes.
- Due to the underlying workflow manager, this package only supports the following operating systems:
* Centos 7
* Oracle Linux 8
* Ubuntu 20.04 LTS
* Ubuntu 22.04 LTS
- Add examples for a cluster run.
- Improve UI by adding documentation and examples.
- Extend the interface to add more features.
The underlying workflow management system is called GPI-Space which is a task-based workflow management system for parallel applications developed at Fraunhofer ITWM by the CC-HPC group.