Lightweight function pipeline (DAG) creation in pure Python for scientific workflows 🕸️🧪

Run many functions (adaptively) on many cores (>10k-100k) using mpi4py.futures, ipyparallel, loky, or dask-mpi. 🎉

Machine Learning Engineering Open Book

A python package to interact with slurm

AWS Slurm Cluster for EDA Workloads

pip install the deep learning & HPC starter pack to begin your project.

Some examples for various purposes

Best practices & guides on how to write distributed pytorch training code

A scalable, efficient, cross-platform (Linux/macOS) and easy-to-use workflow engine in pure Python.

Simple HPC queuing system adapter for Python on based jinja templates to automate the submission script creation.

Dotfiles that I like to use

UAB Research Computing Documentation

Carbon-aware batch job scheduler extension for Slurm.

A modern Python interface for interacting with OpenFOAM

naive python slurm control

SlurmGen - Simple Slurm Manager

🦠📇 Microbial genomes-to-report pipeline

qtop (pronounced queue-top) is a tool written in order to monitor the state of Queueing Systems, along with related information relevant on HPC & grid clusters. At present it supports **PBS, SGE & OAR** families. There is a historic reference for the prior shell version of the tool, at former CERN source:

generate HPC scheduler systems jobs input scripts and submit these scripts to HPC systems and poke until they finish

Slurm-Mail is a drop in replacement for Slurm's e-mails to give users much more information about their jobs compared to the standard Slurm e-mails.