5. Scripting.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Scripting Exercise: Creating an automatic report\n",
    "In the course up to now, we've learned about how we can use native python techniques (functions, if/else statements, conditionals, etc) to read interact with python data types.\n",
    "\n",
    "In this section, we practice using these tools and see how they can be used to improve processes by automating tasks.\n",
    "\n",
    "## Scripting in python\n",
    "Remember that we can run python in two ways: in *scripting mode*, or in *interactive mode*.\n",
    "\n",
    "When we run code in a notebook cell, we are running code in interactive mode. This is great when we want an immediate output, and when we want to see how our code works and check that it is working properly. \n",
    "\n",
    "However, when we want to create a program that we can use to repeatedly complete repetitive tasks, it's best to use python in script mode.\n",
    "\n",
    "This means we package our code up into a .py file, and run it from the command line. When we do so, our entire file will run, not just individual lines. This is very powerful as it makes executing code for repetitive tasks very easy.\n",
    "\n",
    "### Moving code into a script\n",
    "When developing a script for the first time, it can be helpful to first write and check your code in a Jupyter notebook.\n",
    "\n",
    "Start by moving more and more of your code into one cell. Check where your code is running in the notebook with the command `!pwd`. Think about if this is also where you would want your script to run from.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Our scripting task\n",
    "Our task is to **create a script that takes any string, and counts the number of characters in it.** It should just return the number of characters in the string.\n",
    "\n",
    "First, we will hard-code the string that we want to count into the script. Then, we will add an argument parser to be able to pass in any string that we want. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Framing the task\n",
    "Remember that in script mode, we will run our script from the command line. We need to think about where will run our script from, meaning which directory in our file system. We can check this even within the jupyter system by simply putting `!` in front of the command. This exclamation point tells jupyter to execute the command as a bash command on the underlying server/computer system. It will be helpful for us if we start writing our script in the same place where we will run it, as where we run our script will influence while higher-level directories we have to include in our imports. \n",
    "\n",
    "We also have to consider where we want our output to be saved, and what the output format should be. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# seeing where we are in the file system \n",
    "!pwd"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next part of framing the task is writing what's called *pseudo code*. Pseudo code creates an outline of the code we want to write in the future, written in easy to follow steps in plain English. They should outline the steps our code would follow, or the order of the code that we will write into our program. \n",
    "\n",
    "#### Pseudo Code Steps\n",
    "1. Read in the string\n",
    "2. Count number of characters\n",
    "3. Return the number of characters (as a print-out to the command line)\n",
    "\n",
    "Now that we have our pseudo code steps, we can start coding, working through them. \n",
    "\n",
    "This will mainly involve writing the function to count characters in the string.\n",
    "\n",
    "We will start by writing our code here in jupyter to de-bug, and then move it into a script later."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1. Write the function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# remember that when running python in script mode, \n",
    "# we have to print anything we want to appear.\n",
    "# this is why we have a print statement before the return statement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "def string_length(input_string):\n",
    "    count = 0\n",
    "    for char in input_string:\n",
    "        count += 1\n",
    "    print(count)\n",
    "    return count"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "string_length(\"Data Science Retreat\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Converting into a Script\n",
    "Now we've seen that our code works. It's able to save the required data that we want to have in the report. \n",
    "\n",
    "Our next step is to combine the code we've written so far, and move it into a .py file that can be run in script mode from the command line. \n",
    "\n",
    "Try doing just that and running it!"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Finalizing the conversion to script mode\n",
    "For our reporting script to be really effective, we want the user to be able to do everything from the command line - we don't want them to have to go into the script to edit anything. This is especially true concerning the variables in the script that need to be changed or updated every time - the year and month variables for the timeframe when we want to run the script. \n",
    "\n",
    "We can use another python library called [argparse](https://docs.python.org/3/library/argparse.html) for this. Argparse lets us add arguments or inputs to our script right from the command line.\n",
    "\n",
    "After adding argparse to our script, the next important thing we need to do is add a signal to python about how the script should run. We do this by adding an `if __name__ == '__main__'` block at the bottom of the script, that calls the code in the rest of the script to run. You can learn more about why this block is necessary [here](https://www.geeksforgeeks.org/what-does-the-if-__name__-__main__-do/) and [here](https://stackoverflow.com/questions/419163/what-does-if-name-main-do).\n",
    "\n",
    "This means our script now looks like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import argparse\n",
    "\n",
    "\n",
    "def string_length(string):\n",
    "    count = 0\n",
    "    for char in string:\n",
    "        count += 1\n",
    "    print(count)\n",
    "    return count\n",
    "\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    parser = argparse.ArgumentParser(description='Character Counter')\n",
    "    parser.add_argument('--string', action='store', type=str)\n",
    "    results = parser.parse_args()\n",
    "    string = results.string\n",
    "\n",
    "    string_length(string)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running the Script\n",
    "\n",
    "Now that we have transferred the main code to a python script, it's time to run it! \n",
    "\n",
    "Remember that when we run Python in script mode, we do so from the command line. \n",
    "\n",
    "Every time we run a script from the command line in Python, we start by invoking the computer to run the code in python by simply writing `python`. After the `python` command comes the file path. **If we run the script in the same directory where it is located, we don't need to specify the file path**. However, if we run the code in a higher or lower level of the directory, we will need to specify the correct path to the file. Keep this in mind, as it is the cause of many 'File doesn't exist' errors that you may come across when running python scripts. \n",
    "\n",
    "Here, since our script is in the `/scripts` folder, we need to make sure we indicate this at the beginning of the path when running the script. \n",
    "\n",
    "After specifying the file path, we then include the arguments we added above using argparse. we indicate these by simply writing the argument keyword (here, `--string`). Then we just write the string we want to count after these keywords, with a space in between.\n",
    "\n",
    "Below we see a screenshot of how the code works and how exactly the command line command should look.\n",
    "\n",
    "### Command to run\n",
    "\n",
    "If you are in the top level of the repository, then the command to run the script should be:\n",
    "`python scrips/character_count_script.py --string [STRING]` , with `[STRING]` being filled in with the values you would like to use to generate your script. "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![running_script](./pics/running_script.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise\n",
    "Write a script that generates a report about each of the poems in the `text_files` directory. Three should be one report for each poem. The report for each poem should contain a **word count** for each of the words used in the poem, in descending order from most to least common. \n",
    "\n",
    "The report should be saved under a new folder called `/reports` in a file called `poem_report.txt`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tips: Making a directory\n",
    "\n",
    "For saving the report, we have to create a dedicated directory in which to save the new files. \n",
    "We will first check if a directory already exists, and if it doesn't then we will create it. \n",
    "\n",
    "We will use the [os package](https://docs.python.org/3/library/os.html) from python to navigate the file system. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [],
   "source": [
    "if not os.path.exists('folder'):\n",
    "    os.mkdir('folder')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tip: Writing to the file\n",
    "The following bit of code is to get you started writing a file to disk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [],
   "source": [
    "# opening the new file that we will write to \n",
    "file = open(\"file_name.txt\",\"w\") \n",
    "file.writelines(all_lines) \n",
    "file.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Further Exercises\n",
    "\n",
    "Now that you've seen how to build a script from start to finish, you can go further and make updates to the script, and make additional scripts of your own. Some of these questions require knowledge beyond what we've covered so far in the training. These will require some resarch on your part to complete. \n",
    "\n",
    "Test your knowledge by completing the following exercises:\n",
    "1. Update the script so that it doesn't count `the` and `The` separately\n",
    "2. Update the script to include an argument where the user can enter which poems to report on\n",
    "3. Follow the notebook on webscraping fundamentals in `./bonus/`. Complete the exercise at the end to scrape even more poems, from which to generate reports. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python-classes",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  },
  "vscode": {
   "interpreter": {
    "hash": "33e49de180f7b68adbe0b8c3da4cce2849e5189a09c8fdb059671b22301aa1fe"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}