2019-09-23-batch_notebook_execution.md

layout	title	date	categories	slug
post	Execute Jupyter Notebooks not interactively	2019-09-23 12:00	jupyter notebook condor	batch-notebook-execution

Over the years, I have explored how to scale up easily computation through Jupyter Notebooks by executing them not-interactively, possibily parametrized and remotely. This is mostly for reference.

• nbsubmit is a Python package which has Python API to send a local notebook for execution on a remote SLURM cluster, for example Comet, see an example. This project is not maintained right now.
• Back in 2017 I tested submitting notebooks to Open Science Grid, see the batch-notebooks-condor repository
• Back in 2016 I created scripts to template a Jupyter Notebook and launch SLURM jobs, see slurm.shared.template and runipyloop.sh