course-files/python/06-slurm-script-generator.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "601e8617",
   "metadata": {},
   "source": [
    "# Automating Slurm Script Generation with Python\n",
    "\n",
    "In this session, we'll explore how to streamline the process of generating Slurm scripts tailored to specific commands or tasks. Slurm serves as a robust workload manager utilized in high-performance computing (HPC) environments to efficiently schedule and manage computational tasks across clusters of machines.\n",
    "\n",
    "As you delve into complex bioinformatic projects, you'll find yourself frequently interacting with the Slurm scheduler, often writing repetitive scripts to execute various jobs. By leveraging Python, we aim to automate the generation of Slurm scripts, ultimately enhancing your productivity and efficiency.\n",
    "\n",
    "Note that there is an online SLURM script generator for Nova available at [https://www.hpc.iastate.edu](https://www.hpc.iastate.edu/guides/nova/slurm-script-generator-for-nova). However, this session will focus on creating a custom Python script to generate Slurm scripts directly on your terminal.\n",
    "\n",
    "## Learning Objectives\n",
    "\n",
    "- Understand the role of Slurm in HPC environments.\n",
    "- Learn how to automate the generation of Slurm scripts using Python.\n",
    "- Customize Slurm job parameters and resource allocations.\n",
    "- Basic error handling and debugging techniques.\n",
    "\n",
    "\n",
    "\n",
    "## Before we begin\n",
    "\n",
    "### Update your course repository\n",
    "\n",
    "You need to clone the course repository to Nova. You probably already have cloned this, so you can skip this step.\n",
    "\n",
    "```bash\n",
    "git clone git@github.com:EEOB-BioData/BCB546_Spring2024.git\n",
    "```\n",
    "\n",
    "You will still need to pull new changes to this repository at the beginning of class. This will enable you to access new data files and scripts needed for in-class activities.\n",
    "\n",
    "```bash\n",
    "cd BCB546_Spring2024\n",
    "git pull\n",
    "```\n",
    "\n",
    "Note that if you have modified any files in the repository, you will need to commit those changes before you can pull new changes. If you don't care about the changes, just delete and re-clone the repository.\n",
    "\n",
    "### Start Jupyter notebook on Nova on demand.\n",
    "\n",
    "You can start Jupyter notebook on Nova on demand. This will allow you to run Jupyter notebook on the server and access it from your local machine.\n",
    "\n",
    "1. Go to the [Nova OnDemand](https://nova-ondemand.its.iastate.edu/) and login\n",
    "2. Under the \"Interactive Apps\" tab, click on \"Jupyter Notebook\", request desired resources and click \"Launch\"\n",
    "3. Wait for the job to start and click on the \"Connect to Jupyter\" button\n",
    "\n",
    "\n",
    "## Slurm Overview\n",
    "\n",
    "Slurm (Simple Linux Utility for Resource Management) is an open-source, fault-tolerant, and highly scalable workload manager designed for Linux clusters. It provides a robust framework for job scheduling, resource management, and job monitoring across a cluster of machines. Slurm is widely used in high-performance computing (HPC) environments to efficiently manage computational tasks and optimize resource utilization.\n",
    "\n",
    "On Nova, we use Slurm scheduler for managing the jobs on the cluster. You can submit jobs to the Slurm scheduler using the `sbatch` command. You ran a job on the cluster in the previous session and a typical job script looks like this:\n",
    "\n",
    "|  Slurm script                                          |   Description                               |\n",
    "|--------------------------------------------------------|---------------------------------------------|\n",
    "| `#!/bin/bash`                                          |  shebang line                               |\n",
    "| `#SBATCH --nodes=1`                                    |  number of nodes                            |\n",
    "| `#SBATCH --ntasks-per-node=4`                          |  processor core(s) per node                 |\n",
    "| `#SBATCH --partition=nova`                             |  partition name                             |\n",
    "| `#SBATCH --mem=24GB`                                   |  memory per node                            |\n",
    "| `#SBATCH --time=24:00:00`                              |  walltime limit (HH:MM:SS or   DD-HH:MM:SS) |\n",
    "| `#SBATCH --account=mhufford-lab`                       |  account name                               |\n",
    "| `#SBATCH --job-name=JOBNAME`                           |  job name                                   |\n",
    "| `#SBATCH --output=sbatch_stdout.txt`                   |  output file                                |\n",
    "| `#SBATCH --error=sbatch_stdout.txt`                    |  error file                                 |\n",
    "| `#SBATCH   --mail-user=username@domain.com`            |  email address                              |\n",
    "| `#SBATCH --mail-type=begin`                            |  email at the start of the job              |\n",
    "| `#SBATCH --mail-type=end`                              |  email at the end of the job                |\n",
    "| `#SBATCH --mail-type=fail`                             |  email on job failure                       |\n",
    "| `module load modulename1`                              |  load the required module                   |\n",
    "| `samtools view -bS --threads=4 file.sam   > file.bam`  |  command to run                             |\n",
    "\n",
    "\n",
    "\n",
    "If you have a lot of jobs to run, you might find yourself writing a lot of these scripts. In this session, we will learn how to automate the generation of these scripts using Python.\n",
    "\n",
    "You can also simplify the process by submitting array jobs, but most often, for heterogeneous jobs, you will need to write individual scripts.\n",
    "\n",
    "We will write both a simple script and a more complex script that can be used to generate Slurm scripts for different types of jobs.\n",
    "\n",
    "\n",
    "## Automating Slurm Script Generation\n",
    "\n",
    "We will use python to write various scripts that can generate Slurm scripts for different types of jobs.\n",
    "\n",
    "### Simple Interactive Script Generator\n",
    "\n",
    "We will start by writing a simple Python script that generates a Slurm script for a given command. It will prompt for the options and once collected it will write a slurm job.\n",
    "\n",
    "\n",
    "`input` function is used to get the input from the user. It takes a string as an argument and returns the user input as a string.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "53678508",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter the number of nodes: 1\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'1'"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input(\"Enter the number of nodes: \")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3889d35",
   "metadata": {},
   "source": [
    "We can use this to collect all the required options for the Slurm script."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "ac6d3e35",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter the number of nodes: 1\n",
      "Enter the number of tasks per node: 4\n",
      "Enter the partition name: nova\n",
      "Enter the memory per node: 24GB\n",
      "Enter the walltime limit (HH:MM:SS or DD-HH:MM:SS): 1:00:00\n",
      "Enter the account name: mhufford-lab\n",
      "Enter the job name: \n",
      "Enter the output file: stdout\n",
      "Enter the error file: stderr\n",
      "Enter the email address: username@domain.edu\n",
      "Enter the email type (begin, end, fail): begin\n",
      "Enter the module name: samtools\n",
      "Enter the command to run: samtools index genome.fasta\n"
     ]
    }
   ],
   "source": [
    "nodes = input(\"Enter the number of nodes: \")\n",
    "ntasks_per_node = input(\"Enter the number of tasks per node: \")\n",
    "partition = input(\"Enter the partition name: \")\n",
    "mem = input(\"Enter the memory per node: \")\n",
    "time = input(\"Enter the walltime limit (HH:MM:SS or DD-HH:MM:SS): \")\n",
    "account = input(\"Enter the account name: \")\n",
    "job_name = input(\"Enter the job name: \")\n",
    "output = input(\"Enter the output file: \")\n",
    "error = input(\"Enter the error file: \")\n",
    "mail_user = input(\"Enter the email address: \")\n",
    "mail_type = input(\"Enter the email type (begin, end, fail): \")\n",
    "module = input(\"Enter the module name: \")\n",
    "command = input(\"Enter the command to run: \")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3eea06a9",
   "metadata": {},
   "source": [
    "And once collected, you can use the print function to output the script."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "b23a2dcb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "#!/bin/bash\n",
      "#SBATCH --nodes=1\n",
      "#SBATCH --ntasks-per-node=4\n",
      "#SBATCH --partition=nova\n",
      "#SBATCH --mem=24GB\n",
      "#SBATCH --time=1:00:00\n",
      "#SBATCH --account=mhufford-lab\n",
      "#SBATCH --job-name=\n",
      "#SBATCH --output=stdout\n",
      "#SBATCH --error=stderr\n",
      "#SBATCH --mail-user=username@domain.edu\n",
      "#SBATCH --mail-type=begin\n",
      "module load samtools\n",
      "samtools index genome.fasta\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(f\"\"\"#!/bin/bash\n",
    "#SBATCH --nodes={nodes}\n",
    "#SBATCH --ntasks-per-node={ntasks_per_node}\n",
    "#SBATCH --partition={partition}\n",
    "#SBATCH --mem={mem}\n",
    "#SBATCH --time={time}\n",
    "#SBATCH --account={account}\n",
    "#SBATCH --job-name={job_name}\n",
    "#SBATCH --output={output}\n",
    "#SBATCH --error={error}\n",
    "#SBATCH --mail-user={mail_user}\n",
    "#SBATCH --mail-type={mail_type}\n",
    "module load {module}\n",
    "{command}\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56c56dd4",
   "metadata": {},
   "source": [
    "You can save this script as `simple_slurm_script.py` and run it using the following command.\n",
    "\n",
    "```python\n",
    "python simple_slurm_script_v1.py\n",
    "```\n",
    "\n",
    "and the resulting job script will be printed to the terminal. You can then copy and paste it to file to generate the job.\n",
    "\n",
    "Few things that can make this better\n",
    "\n",
    "1. providing default values for the options\n",
    "2. error handling for the inputs (eg. checking if the input is a number)\n",
    "3. writing the script to a file directly\n",
    "\n",
    "For the first point, you can provide default values for the options by using the `or` operator.\n",
    "\n",
    "```python\n",
    "nodes = input(\"Enter the number of nodes: \") or \"1\"\n",
    "```\n",
    "So when the user just presses enter, it will default to 1.\n",
    "\n",
    "For the second point, you can use a `while` loop to keep asking for the input until the user provides a valid input.\n",
    "\n",
    "```python\n",
    "while True:\n",
    "    try:\n",
    "        nodes = int(input(\"Enter the number of nodes: \") or \"1\")\n",
    "        break\n",
    "    except ValueError:\n",
    "        print(\"Please enter a number\")\n",
    "```\n",
    "\n",
    "For the third point, you can use the `open` function to write the script to a file.\n",
    "\n",
    "```python\n",
    "with open(\"job.sh\", \"w\") as f:\n",
    "    f.write(f\"\"\"#!/bin/bash\n",
    "#SBATCH --nodes={nodes}\n",
    "#SBATCH --ntasks-per-node={ntasks_per_node}\n",
    "#SBATCH --partition={partition}\n",
    "#SBATCH --mem={mem}\n",
    "#SBATCH --time={time}\n",
    "#SBATCH --account={account}\n",
    "#SBATCH --job-name={job_name}\n",
    "#SBATCH --output={output}\n",
    "#SBATCH --error={error}\n",
    "#SBATCH --mail-user={mail_user}\n",
    "#SBATCH --mail-type={mail_type}\n",
    "module load {module}\n",
    "{command}\n",
    "\"\"\")\n",
    "```\n",
    "\n",
    "\n",
    "Let's write a script that does all of this.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "090a3386",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter the number of nodes: 1\n",
      "Enter the number of tasks per node: 4\n",
      "Enter the partition name: nova\n",
      "Enter the memory per node: \n",
      "Enter the walltime limit (HH:MM:SS or DD-HH:MM:SS): \n",
      "Enter the account name: \n",
      "Enter the job name: index\n",
      "Enter the output file: index_job\n",
      "Enter the error file: \n",
      "Enter the email address: \n",
      "Enter the email type (begin, end, fail): \n",
      "Enter the module name: \n",
      "Enter the command to run: samtools index genome.fasta\n",
      "Enter the filename for the Slurm script (e.g., job.sh): \n"
     ]
    }
   ],
   "source": [
    "while True:\n",
    "    try:\n",
    "        nodes = int(input(\"Enter the number of nodes: \") or \"1\")\n",
    "        ntasks_per_node = int(input(\"Enter the number of tasks per node: \") or \"4\")\n",
    "        partition = input(\"Enter the partition name: \") or \"nova\"\n",
    "        mem = input(\"Enter the memory per node: \") or \"24GB\"\n",
    "        time = input(\"Enter the walltime limit (HH:MM:SS or DD-HH:MM:SS): \") or \"24:00:00\"\n",
    "        account = input(\"Enter the account name: \") or \"mhufford-lab\"\n",
    "        job_name = input(\"Enter the job name: \") or \"my-cool-job\"\n",
    "        output = input(\"Enter the output file: \") or \"sbatch_stdout.txt\"\n",
    "        error = input(\"Enter the error file: \") or \"sbatch_stderr.txt\"\n",
    "        mail_user = input(\"Enter the email address: \") or \"username@domain.com\"\n",
    "        mail_type = input(\"Enter the email type (begin, end, fail): \") or \"begin\"\n",
    "        module = input(\"Enter the module name: \") or \"samtools\"\n",
    "        command = input(\"Enter the command to run: \")\n",
    "        break\n",
    "    except ValueError:\n",
    "        print(\"Please enter a valid positive integer for numeric inputs\")\n",
    "\n",
    "with open(input(\"Enter the filename for the Slurm script (e.g., job.sh): \") or 'job.sh', \"w\") as f:\n",
    "    f.write(f\"\"\"#!/bin/bash\n",
    "#SBATCH --nodes={nodes}\n",
    "#SBATCH --ntasks-per-node={ntasks_per_node}\n",
    "#SBATCH --partition={partition}\n",
    "#SBATCH --mem={mem}\n",
    "#SBATCH --time={time}\n",
    "#SBATCH --account={account}\n",
    "#SBATCH --job-name={job_name}\n",
    "#SBATCH --output={output}\n",
    "#SBATCH --error={error}\n",
    "#SBATCH --mail-user={mail_user}\n",
    "#SBATCH --mail-type={mail_type}\n",
    "\n",
    "module load {module}\n",
    "{command}\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28f5b894",
   "metadata": {},
   "source": [
    "You won't see any output, but you can check the `job.sh` file to see the generated script.\n",
    "\n",
    "\n",
    "You can save this script as `simple_slurm_script.py` and run it using the following command.\n",
    "\n",
    "```bash\n",
    "python simple_slurm_script_v2.py\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ec344ff",
   "metadata": {},
   "source": [
    "Now, what if we have multi-line command that we need to provide as input?\n",
    "\n",
    "\n",
    "Option 1.  Keep asking for the input until the user provides an empty line.\n",
    "\n",
    "```python\n",
    "command = []\n",
    "while True:\n",
    "    line = input(\"Enter the command to run (press enter to finish): \")\n",
    "    if not line:\n",
    "        break\n",
    "    command.append(line)\n",
    "```\n",
    "\n",
    "Option 2. Take input as a file and read the file.\n",
    "\n",
    "```python\n",
    "filename = input(\"Enter the filename containing the command to run: \")\n",
    "with open(filename) as f:\n",
    "    command = f.readlines()\n",
    "```\n",
    "\n",
    "You can also ask the user (you!) if you want to provide a file or type the command.\n",
    "\n",
    "```python\n",
    "command = input(\"Do you want to provide the command as a file? (y/n): \")\n",
    "if command.lower() == \"y\":\n",
    "    filename = input(\"Enter the filename containing the command to run: \")\n",
    "    with\n",
    "    open(filename) as f:\n",
    "        command = f.readlines()\n",
    "else:\n",
    "    command = []\n",
    "    while True:\n",
    "        line = input(\"Enter the command to run (press enter to finish): \")\n",
    "        if not line:\n",
    "            break\n",
    "        command.append(line)\n",
    "```\n",
    "\n",
    "I will leave it up to you to implement this in the script."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6c19c1a",
   "metadata": {},
   "source": [
    "### Advanced Script Generator\n",
    "\n",
    "We will now write a more advanced script that can generate Slurm scripts for different types of jobs. We will use the `argparse` module to parse the command-line arguments and generate the Slurm script. We will also implement error handling and write the script to a file directly.\n",
    "\n",
    "`argparse` module provides a mechanism to parse the command-line arguments and generate help messages. It is a standard module in Python and you can use it to write user-friendly command-line interfaces.\n",
    "\n",
    "You can start by importing the `argparse` module and creating a parser object.\n",
    "\n",
    "Main parts of the script:\n",
    "\n",
    "```python\n",
    "import argparse # import the argparse module\n",
    "parser = argparse.ArgumentParser(description=\"Generate Slurm scripts for different types of jobs\") # create a parser object\n",
    "```\n",
    "\n",
    "You can add arguments to the parser object using the `add_argument` method.\n",
    "\n",
    "```python\n",
    "parser.add_argument(\"--nodes\", type=int, default=1, help=\"number of nodes\")\n",
    "```\n",
    "\n",
    "You can then parse the arguments using the `parse_args` method.\n",
    "\n",
    "```python\n",
    "args = parser.parse_args()\n",
    "```\n",
    "\n",
    "You can access the arguments using the dot notation.\n",
    "\n",
    "```python\n",
    "print(args.nodes)\n",
    "```\n",
    "\n",
    "\n",
    "`os` module provides a way to interact with the operating system. We will use it to check if the file exists before reading it.\n",
    "\n",
    "```python\n",
    "import os\n",
    "if os.path.exists(args.command):\n",
    "    with open(args.command) as f:\n",
    "        command = f.readlines()\n",
    "else:\n",
    "    command = args.command.split(\"\\n\")\n",
    "```\n",
    "\n",
    "Similar to `os.path.exists`, the `os.path.isfile` function checks if the file exists and `os.path.getsize` checks if the file is empty.\n",
    "\n",
    "\n",
    "Now let's put this all together.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5e503a87",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import argparse\n",
    "import os\n",
    "\n",
    "def read_command_from_file(command_file):\n",
    "    with open(command_file, 'r') as file:\n",
    "        return file.read().strip()\n",
    "\n",
    "def generate_slurm_script(args):\n",
    "    with open(args.output_file, \"w\") as f:\n",
    "        f.write(f\"\"\"#!/bin/bash\n",
    "#SBATCH --nodes={args.nodes}\n",
    "#SBATCH --ntasks-per-node={args.ntasks_per_node}\n",
    "#SBATCH --partition={args.partition}\n",
    "#SBATCH --mem={args.mem}\n",
    "#SBATCH --time={args.time}\n",
    "#SBATCH --account={args.account}\n",
    "#SBATCH --job-name={args.job_name}\n",
    "#SBATCH --output={args.output}\n",
    "#SBATCH --error={args.error}\n",
    "#SBATCH --mail-user={args.mail_user}\n",
    "#SBATCH --mail-type={args.mail_type}\n",
    "\n",
    "module load {args.module}\n",
    "{args.command}\n",
    "\"\"\")\n",
    "\n",
    "def main():\n",
    "    parser = argparse.ArgumentParser(description=\"Generate Slurm script for job submission\")\n",
    "    parser.add_argument(\"--nodes\", type=int, default=1, help=\"Number of nodes\")\n",
    "    parser.add_argument(\"--ntasks-per-node\", type=int, default=4, help=\"Number of tasks per node\")\n",
    "    parser.add_argument(\"--partition\", default=\"nova\", help=\"Partition name\")\n",
    "    parser.add_argument(\"--mem\", default=\"24GB\", help=\"Memory per node\")\n",
    "    parser.add_argument(\"--time\", default=\"24:00:00\", help=\"Walltime limit (HH:MM:SS or DD-HH:MM:SS)\")\n",
    "    parser.add_argument(\"--account\", default=\"mhufford-lab\", help=\"Account name\")\n",
    "    parser.add_argument(\"--job-name\", default=\"my-cool-job\", help=\"Job name\")\n",
    "    parser.add_argument(\"--output\", default=\"sbatch_stdout.txt\", help=\"Output file\")\n",
    "    parser.add_argument(\"--error\", default=\"sbatch_stderr.txt\", help=\"Error file\")\n",
    "    parser.add_argument(\"--mail-user\", default=\"username@domain.com\", help=\"Email address\")\n",
    "    parser.add_argument(\"--mail-type\", default=\"begin\", help=\"Email type (begin, end, fail)\")\n",
    "    parser.add_argument(\"--module\", default=\"samtools\", help=\"Module name\")\n",
    "    parser.add_argument(\"--output-file\", help=\"Filename for the generated Slurm script\")\n",
    "    parser.add_argument(\"--command-file\", help=\"Filename containing the command to run\")\n",
    "    args = parser.parse_args()\n",
    "\n",
    "    if args.command_file:\n",
    "        if os.path.isfile(args.command_file) and os.path.getsize(args.command_file) > 0:\n",
    "            # Read command from file\n",
    "            args.command = read_command_from_file(args.command_file)\n",
    "        else:\n",
    "            print(\"Error: Command file is empty or does not exist.\")\n",
    "            return\n",
    "    else:\n",
    "        print(\"Error: Please provide a command file.\")\n",
    "        return\n",
    "\n",
    "    if args.output_file is None:\n",
    "        # Generate output file name based on input command filename with job.sh suffix\n",
    "        input_file_name = os.path.basename(args.command_file).replace('.txt', '')\n",
    "        args.output_file = f\"{input_file_name}_job.sh\"\n",
    "\n",
    "    generate_slurm_script(args)\n",
    "\n",
    "if __name__ == \"__main__\": # this line is required to run the script from the command line\n",
    "    main()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f616b6c8",
   "metadata": {},
   "source": [
    "There are 3 main parts to this script:\n",
    "\n",
    "1. `read_command_from_file` function reads the command from the file.\n",
    "2. `generate_slurm_script` function generates the Slurm script.\n",
    "3. `main` function parses the command-line arguments and generates the Slurm script.\n",
    "\n",
    "You can save this script as `advanced_slurm_script.py` and you will have a more advanced script generator that can handle multi-line commands and generate the Slurm script for you.\n",
    "\n",
    "Examples:\n",
    "\n",
    "```bash\n",
    "python advanced_slurm_script.py  --help\n",
    "```\n",
    "\n",
    "The output:\n",
    "```\n",
    "usage: myjob.py [-h] [--nodes NODES] [--ntasks-per-node NTASKS_PER_NODE] \n",
    "[--partition PARTITION] [--mem MEM] [--time TIME] [--account ACCOUNT] \n",
    "[--job-name JOB_NAME] [--output OUTPUT] [--error ERROR] [--mail-user MAIL_USER]\n",
    "[--mail-type MAIL_TYPE] [--module MODULE] [--output-file OUTPUT_FILE] \n",
    "[--command-file COMMAND_FILE]\n",
    "\n",
    "Generate Slurm script for job submission\n",
    "\n",
    "options:\n",
    "  -h, --help            show this help message and exit\n",
    "  --nodes NODES         Number of nodes\n",
    "  --ntasks-per-node NTASKS_PER_NODE\n",
    "                        Number of tasks per node\n",
    "  --partition PARTITION\n",
    "                        Partition name\n",
    "  --mem MEM             Memory per node\n",
    "  --time TIME           Walltime limit (HH:MM:SS or DD-HH:MM:SS)\n",
    "  --account ACCOUNT     Account name\n",
    "  --job-name JOB_NAME   Job name\n",
    "  --output OUTPUT       Output file\n",
    "  --error ERROR         Error file\n",
    "  --mail-user MAIL_USER\n",
    "                        Email address\n",
    "  --mail-type MAIL_TYPE\n",
    "                        Email type (begin, end, fail)\n",
    "  --module MODULE       Module name\n",
    "  --output-file OUTPUT_FILE\n",
    "                        Filename for the generated Slurm script\n",
    "  --command-file COMMAND_FILE\n",
    "                        Filename containing the command to run\n",
    "```\n",
    "\n",
    "Usage example:\n",
    "\n",
    "My commands in the file `gzip.txt` contains following lines:\n",
    "\n",
    "```bash\n",
    "gzip Zm-B73-REFERENCE-NAM-5.0_Zm00001eb.1.gff3\n",
    "gzip Zm-B97-REFERENCE-NAM-1.0_Zm00018ab.1.gff3\n",
    "gzip Zm-CML322-REFERENCE-NAM-1.0_Zm00025ab.1.gff3\n",
    "gzip Zm-CML333-REFERENCE-NAM-1.0_Zm00026ab.1.gff3\n",
    "gzip Zm-CML52-REFERENCE-NAM-1.0_Zm00019ab.1.gff3\n",
    "gzip Zm-HP301-REFERENCE-NAM-1.0_Zm00027ab.1.gff3\n",
    "gzip Zm-M37W-REFERENCE-NAM-1.0_Zm00032ab.1.gff3\n",
    "gzip Zm-NC350-REFERENCE-NAM-1.0_Zm00036ab.1.gff3\n",
    "gzip Zm-Oh43-REFERENCE-NAM-1.0_Zm00039ab.1.gff3\n",
    "gzip Zm-P39-REFERENCE-NAM-1.0_Zm00040ab.1.gff3\n",
    "gzip Zm-Tzi8-REFERENCE-NAM-1.0_Zm00042ab.1.gff3\n",
    "```\n",
    "\n",
    "To generate the Slurm script for this command, you can run the following command:\n",
    "\n",
    "```bash\n",
    "python advanced_slurm_script.py --command-file gzip.txt\n",
    "```\n",
    "\n",
    "This will generate a Slurm script named `gzip_job.sh` with the following content:\n",
    "\n",
    "```bash\n",
    "#!/bin/bash\n",
    "#SBATCH --nodes=1\n",
    "#SBATCH --ntasks-per-node=4\n",
    "#SBATCH --partition=nova\n",
    "#SBATCH --mem=24GB\n",
    "#SBATCH --time=24:00:00\n",
    "#SBATCH --account=mhufford-lab\n",
    "#SBATCH --job-name=my-cool-job\n",
    "#SBATCH --output=sbatch_stdout.txt\n",
    "#SBATCH --error=sbatch_stderr.txt\n",
    "#SBATCH --mail-user=username@domain.com\n",
    "#SBATCH --mail-type=begin\n",
    "\n",
    "module load samtools\n",
    "gzip Zm-B73-REFERENCE-NAM-5.0_Zm00001eb.1.gff3\n",
    "gzip Zm-B97-REFERENCE-NAM-1.0_Zm00018ab.1.gff3\n",
    "gzip Zm-CML322-REFERENCE-NAM-1.0_Zm00025ab.1.gff3\n",
    "gzip Zm-CML333-REFERENCE-NAM-1.0_Zm00026ab.1.gff3\n",
    "gzip Zm-CML52-REFERENCE-NAM-1.0_Zm00019ab.1.gff3\n",
    "gzip Zm-HP301-REFERENCE-NAM-1.0_Zm00027ab.1.gff3\n",
    "gzip Zm-M37W-REFERENCE-NAM-1.0_Zm00032ab.1.gff3\n",
    "gzip Zm-NC350-REFERENCE-NAM-1.0_Zm00036ab.1.gff3\n",
    "gzip Zm-Oh43-REFERENCE-NAM-1.0_Zm00039ab.1.gff3\n",
    "gzip Zm-P39-REFERENCE-NAM-1.0_Zm00040ab.1.gff3\n",
    "gzip Zm-Tzi8-REFERENCE-NAM-1.0_Zm00042ab.1.gff3\n",
    "```\n",
    "\n",
    "I can further customize it for my needs and submit it to the Slurm scheduler.\n",
    "\n",
    "eg:\n",
    "\n",
    "```bash\n",
    "python advanced_slurm_script.py --command-file gzip.txt --job-name gzip-job --output gzip_stdout.txt --error gzip_stderr.txt\n",
    "```\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python3 (BCB/EEOB 546)",
   "language": "python",
   "name": "notebook"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}