I. Files.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"atom.png\" alt=\"Atom\" style=\"width:60px\" align=\"left\" vertical-align=\"middle\">\n",
    "\n",
    "## 1. Reading a File\n",
    "*in Python 3 (Updated: 26 June 2022)*\n",
    "\n",
    "----\n",
    "Computers use file systems to store and retrieve data. Each file is an individual container of related information. If you’ve ever saved a document, downloaded a song, or even sent an email you’ve created a file on some computer somewhere. Even the document you're reading now is a file.\n",
    "\n",
    "<br/>So, how do we interact with files using Python? We’re going to learn how to read and write different kinds of files using code. Let’s say we had a file called `real_cool_document.txt` with these contents:\n",
    "    \n",
    "    `Wowsers!`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wowsers!\n"
     ]
    }
   ],
   "source": [
    "with open('Examples/real_cool_document.txt') as cool_doc:\n",
    "    cool_contents = cool_doc.read()\n",
    "print(cool_contents)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This opens a file object called `cool_doc` and creates a new indented block where you can read the contents of the opened file. We then read the contents of the file `cool_doc` using `cool_doc.read()` and save the resulting string into the variable `cool_contents`. Then we print `cool_contents`, which outputs the statement `Wowsers!`. \n",
    "\n",
    "<br/>*Exercise:*\n",
    "<br/>Use `with` to open the file `welcome.tx`t. Save the file object as `text_file`. Read the contents of `text_file` and save the results in `text_data` then print it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello world!\n"
     ]
    }
   ],
   "source": [
    "with open('Examples/welcome.txt') as text_file:\n",
    "    text_data = text_file.read()\n",
    "print(text_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"atom.png\" alt=\"Atom\" style=\"width:60px\" align=\"left\" vertical-align=\"middle\">\n",
    "\n",
    "## 2. Iterating Through Lines\n",
    "*in Python 3*\n",
    "\n",
    "----\n",
    "When we read a file, we might want to grab the whole document in a single string, like `.read()` would return. But what if we wanted to store each line in a variable? We can use the `.readlines()` function to read a text file line by line instead of having the whole thing. Suppose we have a file: `keats_sonnet.txt`\n",
    "\n",
    "<br/>`To one who has been long in city pent,\n",
    "’Tis very sweet to look into the fair\n",
    "And open face of heaven,—to breathe a prayer\n",
    "Full in the smile of the blue firmament.`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "To one who has been long in city pent,\n",
      "\n",
      "’Tis very sweet to look into the fair\n",
      "\n",
      "And open face of heaven,—to breathe a prayer\n",
      "\n",
      "Full in the smile of the blue firmament.\n"
     ]
    }
   ],
   "source": [
    "with open('Examples/keats_sonnet.txt') as keats_sonnet:\n",
    "    for line in keats_sonnet.readlines():\n",
    "        print(line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above script creates a temporary file object called `keats_sonnet` that points to the file `keats_sonnet.txt`. It then iterates over each line in the document and prints the entire file out. \n",
    "\n",
    "<br/>Indeed, when opening a file, a list object is created in memory and holds the complete contents of the file. Hence, `.readlines()` is not required *in most instances,* as shown in this example:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1. How many lines do we write on the daily,\n",
      "\n",
      "\n",
      "\n",
      "2. Many money, we write many many many\n",
      "\n",
      "\n",
      "\n",
      "3. How many lines do you write on the daily,\n",
      "\n",
      "\n",
      "\n",
      "4. Say you say many money, you write many many many\n"
     ]
    }
   ],
   "source": [
    "with open('Examples/how_many_lines.txt') as lines_doc:\n",
    "    for line in lines_doc:\n",
    "        print(line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"atom.png\" alt=\"Atom\" style=\"width:60px\" align=\"left\" vertical-align=\"middle\">\n",
    "\n",
    "## 3. Reading a Line\n",
    "*in Python 3*\n",
    "\n",
    "----\n",
    "Sometimes you don’t want to iterate through a whole file. As mentioned above, there’s a different file method, `.readline()`, which will only read a single line at a time. If the entire document is read line by line in this way subsequent calls to `.readline()` will not throw an error but will start returning an empty string (\"\"). Suppose we had this file: `millay_sonnet.txt`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First line: I shall forget you presently, my dear,\n",
      "\n",
      "Second line: So make the most of this, your little day,\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('Examples/millay_sonnet.txt') as sonnet_doc:\n",
    "    first_line = sonnet_doc.readline()\n",
    "    second_line = sonnet_doc.readline()\n",
    "    print(\"First line:\", first_line)\n",
    "    print(\"Second line:\", second_line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This script also creates a file object called `sonnet_doc` that points to the file `millay_sonnet.txt`. It then reads in the first line using `sonnet_doc.readline()` and saves that to the variable `first_line`. It then saves the second line (`\"So make the most of this, your little day,\"`) into the variable `second_line` and then prints it out.\n",
    "\n",
    "<br/>*Exercise:*\n",
    "<br/>Using a `with` statement, create a file object pointing to the file `just_the_first.txt`. Store that file object in the variable `first_line_doc`. Save the first line of `just_the_first.txt` into the variable `first_line` and print it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "You do look, my son, in a moved sort,\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open('Examples/just_the_first.txt') as first_line_doc:\n",
    "    first_line = first_line_doc.readline()\n",
    "print(first_line)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"atom.png\" alt=\"Atom\" style=\"width:60px\" align=\"left\" vertical-align=\"middle\">\n",
    "\n",
    "## 4. Writing a File\n",
    "*in Python 3*\n",
    "\n",
    "----\n",
    "Reading a file is all well and good, but what if we want to create a file of our own? With Python we can do just that. It turns out that our `open()` function that we’re using to open a file to read needs another argument to open a file to write to."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('Examples/generated_file.txt', 'w') as gen_file:\n",
    "    gen_file.write(\"What an incredible file!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we pass the argument `'w'` to `open()` in order to indicate to open the file in write-mode. The default argument is `'r'` and passing `'r'` to `open()` opens the file in read-mode as we’ve been doing.\n",
    "\n",
    "<br/>This code creates a new file in the `Examples` folder and gives it the text `What an incredible file!`. It’s important to note that if there is already a file called `generated_file.txt` it will completely overwrite that file, erasing whatever its contents were before.\n",
    "\n",
    "<br/>*Exercise:*\n",
    "<br/>Create a file object for the file `bad_bands.txt` using the `open()` function with the `w` argument. Assign this object to the temporary variable `bad_bands_doc`. Use the `bad_bands_doc.write()` method to add the name of a musical group you dislike to the document `bad_bands`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('Examples/bad_bands.txt', 'w') as bad_bands_doc:\n",
    "    bad_bands_doc.write(\"ACDC\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"atom.png\" alt=\"Atom\" style=\"width:60px\" align=\"left\" vertical-align=\"middle\">\n",
    "\n",
    "## 5. Appending to a File\n",
    "*in Python 3*\n",
    "\n",
    "----\n",
    "So maybe completely deleting and overwriting existing files is something that bothers you. Isn’t there a way to just add a line to a file without completely deleting it? Of course there is! Instead of opening the file using the argument `'w'` for write-mode, we open it with `'a'` for append-mode. If we have a generated file `generated_file.txt` with the following contents: `\"This was a popular file...\"`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('Examples/generated_file.txt', 'a') as gen_file:\n",
    "    gen_file.write(\"This was a popular file...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we can add another line to that file with the following code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('Examples/generated_file.txt', 'a') as gen_file:\n",
    "    gen_file.write(\"\\n... and it still is\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the code above we open a file object in the temporary variable `gen_file`. This variable points to the file `generated_file.txt` and, since it’s open in append-mode, adds the line `\"... and it still is\"` as a new line to the file. If you were to open the file after running the script it would look like this:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "What an incredible file!This was a popular file...\n",
      "... and it still is\n"
     ]
    }
   ],
   "source": [
    "with open('Examples/generated_file.txt', 'r') as output:\n",
    "    print(output.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that opening the file in append-mode, with `'a'` as an argument to `open()`, means that using the file object’s `.write()` method *appends* whatever is passed to the end of the file in a new line. If we were to run the script again, this would be what `generated_file.txt` looks like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "What an incredible file!This was a popular file...\n",
      "... and it still is\n",
      "... and it still is\n"
     ]
    }
   ],
   "source": [
    "with open('Examples/generated_file.txt', 'a') as gen_file:\n",
    "    gen_file.write(\"\\n... and it still is\")\n",
    "\n",
    "# Now read the file\n",
    "with open('Examples/generated_file.txt', 'r') as output:\n",
    "    print(output.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that we’ve appended `\"... and it still is\"` to the file a second time (or more, depending on how many times you ran the script)! This is because we opened `generated_file.txt` in append-mode.\n",
    "\n",
    "<br/>*Exercise:*\n",
    "<br/>We’ve got a file, `cool_dogs.txt`, filled with all the cool dogs we know. Somehow while compiling this list we forgot about one very cool dog. Let’s fix that problem by adding him to our `cool_dogs.txt`. Open up the file in append-mode and assign it to the file object `cool_dogs_file`. Inside your with block, add `“Air Buddy”` to `cool_dogs.txt`. Air Buddy is a Golden Retriever that plays basketball, which more than qualifies him for this list!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('Examples/cool_dogs.txt','a') as cool_dogs_file:\n",
    "    cool_dogs_file.write(\"Air Buddy\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"atom.png\" alt=\"Atom\" style=\"width:60px\" align=\"left\" vertical-align=\"middle\">\n",
    "\n",
    "## 6. What's With \"with\"?\n",
    "*in Python 3*\n",
    "\n",
    "----\n",
    "We’ve been opening these files with this `with` block so far, but it seems a little weird that we can only use our file variable in the indented block. Why is that? The `with` keyword invokes something called a *context manager* for the file that we’re calling `open()` on. This context manager takes care of opening the file when we call `open()` and then closing the file after we leave the indented block.\n",
    "\n",
    "<br/>Why is closing the file so complicated? Well, most other aspects of our code deal with things that Python itself controls. All the variables you create: integers, lists, dictionaries — these are all Python objects, and Python knows how to clean them up when it’s done with them. Since your files exist *outside* your Python script, we need to tell Python when we’re done with them so that it can close the connection to that file. Leaving a file connection open unnecessarily can affect performance or impact other programs on your computer that might be trying to access that file.\n",
    "\n",
    "<br/>The `with` syntax replaces older ways to access files where you need to call `.close()` on the file object manually. We can still open up a file and append to it with the old syntax, as long as we remember to close the file connection afterwards."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "fun_cities_file = open('Examples/fun_cities.txt', 'a')\n",
    "\n",
    "# We can now append a line to \"fun_cities\".\n",
    "fun_cities_file.write(\"Montréal\")\n",
    "\n",
    "# But we need to remember to close the file\n",
    "fun_cities_file.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above script we added “Montréal” as a new line in our file `fun_cities.txt`. However, since we used the older-style syntax, we had to remember to close the file afterwards. Since this is necessarily more verbose (requires at least one more line of code) without being any more expressive, using `with` is preferred."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"atom.png\" alt=\"Atom\" style=\"width:60px\" align=\"left\" vertical-align=\"middle\">\n",
    "\n",
    "## 7. What Is a CSV File?\n",
    "*in Python 3*\n",
    "\n",
    "----\n",
    "Text files aren’t the only thing that Python can read, but they’re the only thing that we don’t need any additional parsing library to understand. CSV files are an example of a text file that impose a structure to their data. CSV stands for *Comma-Separated Values* and CSV files are usually the way that data from spreadsheet software (like Microsoft Excel or Google Sheets) is exported into a portable format. A spreadsheet that looks like the following:\n",
    "\n",
    "    Roger Smith | rsmith | wigginsryan@yahoo.com\n",
    "    Michelle Beck | mlbeck | hcosta@hotmail.com\n",
    "    Ashley Barker | a_bark_x | a_bark_x@turner.com\n",
    "    Lynn Gonzales | goodmanjames | lynniegonz@hotmail.com\n",
    "    Jennifer Chase | chasej | jchase@ramirez.com\n",
    "    Charles Hoover | choover | choover89@yahoo.com\n",
    "    Adrian Evans | adevans | adevans98@yahoo.com\n",
    "    Susan Walter | susan82 | swilliams@yahoo.com\n",
    "    Stephanie King | stephanieking | sking@morris-tyler.com\n",
    "    Erika Miller | jessica32 | ejmiller79@yahoo.com\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In a CSV file that same exact data would be rendered like this:\n",
    "\n",
    "<br/>Name,Username,Email\n",
    "<br/>Roger Smith,rsmith,wigginsryan@yahoo.com\n",
    "<br/>Michelle Beck,mlbeck,hcosta@hotmail.com\n",
    "<br/>Ashley Barker,a_bark_x,a_bark_x@turner.com\n",
    "<br/>Lynn Gonzales,goodmanjames,lynniegonz@hotmail.com\n",
    "<br/>Jennifer Chase,chasej,jchase@ramirez.com\n",
    "<br/>Charles Hoover,choover,choover89@yahoo.com\n",
    "<br/>Adrian Evans,adevans,adevans98@yahoo.com\n",
    "<br/>Susan Walter,susan82,swilliams@yahoo.com\n",
    "<br/>Stephanie King,stephanieking,sking@morris-tyler.com\n",
    "<br/>Erika Miller,jessica32,ejmiller79@yahoo.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that the first row of the CSV file doesn’t actually represent any data, just the labels of the data that’s present in the rest of the file. The rest of the rows of the file are the same as the rows in the spreadsheet software, just instead of being separated into different cells they’re separated by… well I suppose it’s fair to say they’re separated by commas.\n",
    "\n",
    "<br/>*Exercise:*\n",
    "<br/>CSV files are just plain text files! Open `logger.csv` using our standard with syntax, saving the file object in the temporary variable `log_csv_file`. Print out its contents by calling `.read()` on the file. Notice that it is parsed as a string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name,Age,ID\n",
      "Richard Andrews,43,0de2ecf31df2386377b1d2dc4fae8b16fad05ad0         \n",
      "Hailey Sellers,24,3d9b8a95458c1df2687191e8146a97ca4afb28aa          \n",
      "Jessica Pace,39,a5daa63ef893cb86bc8df1110cc9a5f8e1d0c563            \n",
      "Jasmine Escobar,42,9844e403841ec84b9a2fb3caf9d2a1c9ee042d31         \n",
      "Karen Kelly,26,8338f76ac0e9a76d73d57790f1d9843b185b5428             \n",
      "Patricia Christensen,70,23099bb630c1c64989458393045f08de3bac0eb9    \n",
      "Jessica Hansen,24,a8c035ccd099ef909a46e0d96b76c0f132c9c562          \n",
      "Raymond Adams,53,a051901830ff6c2095524ef92b1541eef9f8c64d           \n",
      "Stephanie Morrow,53,3bad04a5fc0a7ec33735ae45535f354887988f35        \n",
      "Timothy Ramos,34,b4930920b5256c4e592541297e43a556c8fe33a8 \n"
     ]
    }
   ],
   "source": [
    "with open('Examples/logger.csv') as log_csv_file:\n",
    "    print(log_csv_file.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"atom.png\" alt=\"Atom\" style=\"width:60px\" align=\"left\" vertical-align=\"middle\">\n",
    "\n",
    "## 8. Reading a CSV File\n",
    "*in Python 3*\n",
    "\n",
    "----\n",
    "Recall our CSV file from our last exercise `users.csv` that contained the following:\n",
    "\n",
    "<br/>Name,Username,Email\n",
    "<br/>Roger Smith,rsmith,wigginsryan@yahoo.com\n",
    "<br/>Michelle Beck,mlbeck,hcosta@hotmail.com\n",
    "<br/>Ashley Barker,a_bark_x,a_bark_x@turner.com\n",
    "<br/>Lynn Gonzales,goodmanjames,lynniegonz@hotmail.com\n",
    "\n",
    "<br/>Even though we can read these lines as text without a problem, there are ways to access the data in a format better suited for programming purposes. In Python we can convert that data into a dictionary using the `csv` library’s `DictReader` object. Here’s how we’d create a list of the email addresses of all of the users in the above table:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import csv\n",
    "\n",
    "list_of_email_addresses = []\n",
    "with open('Examples/users.csv', newline='') as users_csv:\n",
    "    user_reader = csv.DictReader(users_csv)\n",
    "    for row in user_reader:\n",
    "        list_of_email_addresses.append(row['Email'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above code we first import our `csv` library, which gives us the tools to parse our CSV file. We then create the empty list `list_of_email_addresses` which we’ll later populate with the email addresses from our CSV. Then we open the `users.csv` file with the temporary variable `users_csv`.\n",
    "\n",
    "<br/>We pass the additional keyword argument `newline=''` to the file opening `open()` function so that we don’t accidentally mistake a line break in one of our data fields as a new row in our CSV (read more about this in the Python documentation).\n",
    "\n",
    "<br/>After opening our new CSV file we use `csv.DictReader(users_csv)` which converts the lines of our CSV file to Python dictionaries which we can use access methods for. The keys of the dictionary are, by default, the entries in the first line of our CSV file. Since our CSV’s first line calls the third field in our CSV `“Email“`, we can use that as the key in each row of our `DictReader`.\n",
    "\n",
    "<br/>When we iterate through the rows of our `user_reader` object, we access all of the rows in our CSV as dictionaries (except for the first row, which we used to label the keys of our dictionary). By accessing the `'Email'` key of each of these rows we can grab the email address in that row and append it to our `list_of_email_addresses`.\n",
    "\n",
    "<br/>*Exercise:*\n",
    "<br/>Import the `csv` module. Open up the file `cool_csv.csv` in the temporary variable `cool_csv_file` then using `csv.DictReader` read the contents of `cool_csv_file` into a new variable called `cool_csv_dict`. For each row in `cool_csv_dict` print out that row’s `\"Cool Fact\"`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Has never been out of the country.\n",
      "Published a small biography on a local legend.\n",
      "Happened across a major movie star while biking once.\n",
      "Once ate three packages of cookies in one sitting.\n",
      "Has been to over fifteen different forests.\n",
      "Old job was across the street from their new job.\n",
      "Has a dog named Peanut.\n",
      "While working a phone bank accidentally called their mother.\n",
      "Can whistle the national anthem of twelve different nations.\n",
      "Is triple-jointed.\n"
     ]
    }
   ],
   "source": [
    "import csv\n",
    "with open('Examples/cool_csv.csv') as cool_csv_file:\n",
    "    cool_csv_dict = csv.DictReader(cool_csv_file)\n",
    "    for row in cool_csv_dict:\n",
    "        print(row[\"Cool Fact\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"atom.png\" alt=\"Atom\" style=\"width:60px\" align=\"left\" vertical-align=\"middle\">\n",
    "\n",
    "## 9. Reading Different Types of CSV Files\n",
    "*in Python 3*\n",
    "\n",
    "----\n",
    "I need to level with you, I’ve been lying to you for the past two exercises. Well, kind of. We’ve been acting like CSV files are Comma-Separated Values files. It’s true that CSV stands for that, but it’s also true that other ways of separating values are valid CSV files these days.\n",
    "\n",
    "<br/>People used to call Tab-Separated Values files TSV files, but as other separators grew in popularity everyone realized that creating a new `.[a-z]sv` file format for every value-separating character used is not sustainable.\n",
    "\n",
    "<br/>So we call all files with a list of different values a CSV file and then use different *delimiters* (like a comma or tab) to indicate where the different values start and stop.\n",
    "\n",
    "<br/>Let’s say we had an address book, `addresses.csv`. Since addresses usually use commas in them, we’ll need to use a different delimiter for our information. Since none of our data has semicolons (`;`) in them, we can use those.\n",
    "\n",
    "<br/>Name;Address;Telephone\n",
    "<br/>Donna Smith;126 Orr Corner Suite 857\\nEast Michael, LA 54411;906-918-6560\n",
    "<br/>Aaron Osborn;6965 Miller Station Suite 485\\nNorth Michelle, KS 64364;815.039.3661x42816\n",
    "<br/>Jennifer Barnett;8749 Alicia Vista Apt. 288\\nLake Victoriaberg, TN 51094;397-796-4842x451\n",
    "<br/>Joshua Bryan;20116 Stephanie Stravenue\\nWhitneytown, IA 87358;(380)074-6173\n",
    "<br/>Andrea Jones;558 Melissa Keys Apt. 588\\nNorth Teresahaven, WA 63411;+57(8)7795396386\n",
    "<br/>Victor Williams;725 Gloria Views Suite 628\\nEast Scott, IN 38095;768.708.3411x954\n",
    "\n",
    "<br/>Notice the `\\n` character, this is the escape sequence for a new line. The possibility of a new line escaped by a `\\n` character in our data is why we pass the `newline=''` keyword argument to the `open()` function.\n",
    "\n",
    "<br/>Also notice that many of these addresses have commas in them! This is okay, we’ll still be able to read it. If we wanted to, say, print out all the addresses in this CSV file we could do the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "126 Orr Corner Suite 857\\nEast Michael, LA 54411\n",
      "6965 Miller Station Suite 485\\nNorth Michelle, KS 64364\n",
      "8749 Alicia Vista Apt. 288\\nLake Victoriaberg, TN 51094\n",
      "20116 Stephanie Stravenue\\nWhitneytown, IA 87358\n",
      "558 Melissa Keys Apt. 588\\nNorth Teresahaven, WA 63411\n",
      "725 Gloria Views Suite 628\\nEast Scott, IN 38095\n"
     ]
    }
   ],
   "source": [
    "import csv\n",
    "\n",
    "with open('Examples/addresses.csv', newline='') as addresses_csv:\n",
    "    address_reader = csv.DictReader(addresses_csv, delimiter=';')\n",
    "    for row in address_reader:\n",
    "        print(row['Address'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that when we call `csv.DictReader` we pass in the `delimiter` parameter, which is the string that’s used to delineate separate fields in the CSV. We then iterate through the CSV and print out each of the addresses. \n",
    "\n",
    "<br/>*Exercise:*\n",
    "<br/>Import the `csv` module. Open up the file `books.csv` in the variable `books_csv`. Create a `DictReader` instance that uses the `@` symbol as a delimiter to read `books_csv`. Save the result in a variable called `books_reader` then create a list called `isbn_list`, iterate through `books_reade`r to get the ISBN number of every book in the CSV file. Use the `['ISBN']` key for the dictionary objects passed to it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Lauren Murray 978-0-12-995015-8 Enviornment Call, Amount Later Page Country\n",
      "Micheal Jones 978-1-78110-100-1 Rate Security Full\n",
      "Alexander Carr 978-0-315-25137-3 Still Response Size\n",
      "Michael Williams 978-0-388-70665-7 Position Result Five\n",
      "Kathleen Ferguson 978-1-75098-721-6 Country Week Receive And Sign\n",
      "Sarah Dorsey 978-1-06-483628-6 Audience Truth Small\n",
      "Mary Middleton 978-0-7419-8114-1 Travel: Special Offer Near Allow Goal\n",
      "William Todd 978-1-4457-0480-7 Money Exactly Drop Teach\n",
      "Joan Martin 978-0-657-61030-2 Theory Do Half Change\n",
      "Gary Roman 978-1-5039-7539-2 Bill Serve Pull Industry South Job\n"
     ]
    }
   ],
   "source": [
    "import csv\n",
    "isbn_list = []\n",
    "with open('Examples/books.csv') as books_csv:\n",
    "    books_reader = csv.DictReader(books_csv, delimiter='@')\n",
    "    for row in books_reader:\n",
    "        isbn_list.append(row['ISBN'])\n",
    "        print(row['Author'], row['ISBN'], row['Title'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"atom.png\" alt=\"Atom\" style=\"width:60px\" align=\"left\" vertical-align=\"middle\">\n",
    "\n",
    "## 10. Writing a CSV File\n",
    "*in Python 3*\n",
    "\n",
    "----\n",
    "Naturally if we have the ability to read different CSV files we might want to be able to programmatically create CSV files that save output and data that someone could load into their spreadsheet software. Let’s say we have a big list of data that we want to save into a CSV file. We could do the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "big_list = [{'name': 'Fredrick Stein', 'userid': 6712359021, 'is_admin': False}, {'name': 'Wiltmore Denis', 'userid': 2525942, 'is_admin': False}, {'name': 'Greely Plonk', 'userid': 15890235, 'is_admin': False}, {'name': 'Dendris Stulo', 'userid': 572189563, 'is_admin': True}] \n",
    "\n",
    "import csv\n",
    "with open('Examples/output.csv', 'w') as output_csv:\n",
    "    fields = ['name', 'userid', 'is_admin']\n",
    "    output_writer = csv.DictWriter(output_csv, fieldnames=fields)\n",
    "\n",
    "    output_writer.writeheader()\n",
    "    for item in big_list:\n",
    "        output_writer.writerow(item)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In our code above we had a set of dictionaries with the same keys for each, a prime candidate for a CSV. We import the `csv` library, and then open a new CSV file in write-mode by passing the `'w'` argument to the `open()` function.\n",
    "\n",
    "<br/>We define the fields we’re going to be using into a variable called `fields`. We then instantiate our CSV writer object and pass two arguments. The first is `output_csv`, the file handler object. The second is our list of fields fields which we pass to the keyword parameter `fieldnames`.\n",
    "\n",
    "<br/>Now that we’ve instantiated our CSV file writer, we can start adding lines to the file itself! First we want the headers, so we call `.writeheader()` on the writer object. This writes all the `fields` passed to `fieldnames` as the first row in our file. Then we iterate through our `big_list` of data. Each `item` in `big_list` is a dictionary with each field in fields as the keys. We call `output_writer.writerow()` with the `item` dictionaries which writes each line to the CSV file.\n",
    "\n",
    "<br/>*Exercise:*\n",
    "<br/>We have a list in the workspace `access_log` which is a list of dictionaries we want to write out to a CSV file. Let’s start by importing the `csv` module then open up the file `logger.csv` in the temporary variable `logger_csv`. Don’t forget to open the file in write-mode. Create a `csv.DictWriter` instance called `log_writer`. Pass `logger_csv` as the first argument and then `fields` as a keyword argument to the keyword `fieldnames`. Write the header to `log_writer` using the `.writeheader()` method. Finally, iterate through the `access_log` list and add each element to the CSV using `log_writer.writerow()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "access_log = [{'time': '08:39:37', 'limit': 844404, 'address': '1.227.124.181'}, {'time': '13:13:35', 'limit': 543871, 'address': '198.51.139.193'}, {'time': '19:40:45', 'limit': 3021, 'address': '172.1.254.208'}, {'time': '18:57:16', 'limit': 67031769, 'address': '172.58.247.219'}, {'time': '21:17:13', 'limit': 9083, 'address': '124.144.20.113'}, {'time': '23:34:17', 'limit': 65913, 'address': '203.236.149.220'}, {'time': '13:58:05', 'limit': 1541474, 'address': '192.52.206.76'}, {'time': '10:52:00', 'limit': 11465607, 'address': '104.47.149.93'}, {'time': '14:56:12', 'limit': 109, 'address': '192.31.185.7'}, {'time': '18:56:35', 'limit': 6207, 'address': '2.228.164.197'}]\n",
    "fields = ['time', 'address', 'limit']\n",
    "\n",
    "import csv\n",
    "with open('Examples/logger.csv', 'w') as logger_csv:\n",
    "    log_writer = csv.DictWriter(logger_csv, fieldnames=fields)\n",
    "    log_writer.writeheader()\n",
    "    for item in access_log:\n",
    "        log_writer.writerow(item)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"atom.png\" alt=\"Atom\" style=\"width:60px\" align=\"left\" vertical-align=\"middle\">\n",
    "\n",
    "## 11. Reading a JSON File\n",
    "*in Python 3*\n",
    "\n",
    "----\n",
    "CSV isn’t the only file format that Python has a built-in library for. We can also use Python’s file tools to read and write JSON. JSON, an abbreviation of *JavaScript Object Notation,* is a file format inspired by the programming language JavaScript. The name, like CSV is a bit of a misnomer — some JSON is not valid JavaScript (and plenty of JavaScript is not valid JSON).\n",
    "\n",
    "<br/>JSON’s format is endearingly similar to Python dictionary syntax, and so JSON files might be easy to read from a Python developer standpoint. Nonetheless, Python comes with a json package that will help us parse JSON files into actual Python dictionaries. Suppose we have a JSON file, `purchase_14781239.json`, like the following:\n",
    "\n",
    "<br/>`{`\n",
    "<br/>  `'user': 'ellen_greg',`\n",
    "<br/>  `'action': 'purchase',`\n",
    "<br/>  `'item_id': '14781239',`\n",
    "<br/>`}`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We would be able to read that in as a Python dictionary with the following code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ellen_greg\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "with open('Examples/purchase_14781239.json') as purchase_json:\n",
    "    purchase_data = json.load(purchase_json)\n",
    "    print(purchase_data['user'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we import the `json` package. We opened the file using our trusty `open()` command. Since we’re opening it in read-mode we just need to pass the file name. We save the file in the temporary variable `purchase_json`.\n",
    "\n",
    "<br/>We continue by parsing `purchase_json` using `json.load()`, creating a Python dictionary out of the file. Saving the results into `purchase_data` means we can interact with it. We print out one of the values of the JSON file by keying into the `purchase_data` object.\n",
    "\n",
    "<br/>*Exercise:*\n",
    "<br/>Let’s read a JSON file! Start by importing the `json` module. Open up the file `message.json`, saving the file object to the variable `message_json`. Open the file in read-mode, without passing any additional arguments to `open()`. Pass the JSON file object as an argument to `json.load()` and save the resulting Python dictionary as `message`, then print `message['text']`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "That's a cool JSON!\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "with open('Examples/message.json') as message_json:\n",
    "    message = json.load(message_json)\n",
    "    print(message['text'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"atom.png\" alt=\"Atom\" style=\"width:60px\" align=\"left\" vertical-align=\"middle\">\n",
    "\n",
    "## 12. Writing a JSON File\n",
    "*in Python 3*\n",
    "\n",
    "----\n",
    "Naturally we can use the `json` library to translate Python objects to JSON as well. This is especially useful in instances where you’re using a Python library to serve web pages, you would also be able to serve JSON. Let’s say we had a Python dictionary we wanted to save as a JSON file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "turn_to_json = {\n",
    "  'eventId': 674189,\n",
    "  'dateTime': '2015-02-12T09:23:17.511Z',\n",
    "  'chocolate': 'Semi-sweet Dark',\n",
    "  'isTomatoAFruit': True\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We’d be able to create a JSON file with that information by doing the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "with open('Examples/output.json', 'w') as json_file:\n",
    "    json.dump(turn_to_json, json_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We import the `json` module, open up a write-mode file under the variable `json_file`, and then use the `json.dump()` method to write to the file. `json.dump()` takes two arguments: first the `data` object, then the `file` object you want to save.\n",
    "\n",
    "<br/>*Exercise:*\n",
    "<br/>We want to save `data_payload` to a file called `data.json`. Let’s start by importing the `json` library, then open a new file object in the variable `data_json`. The filename should be `'data.json'` and the file should be opened in write-mode. Finally, call `json.dump()` with `data_payload` and `data_json` to convert our data to JSON and then save it to the file `data.json`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_payload = [\n",
    "  {'interesting message': 'What is JSON? A web application\\'s little pile of secrets.',\n",
    "   'follow up': 'But enough talk!'}\n",
    "]\n",
    "import json\n",
    "with open('Examples/data.json', 'w') as data_json:\n",
    "    json.dump(data_payload, data_json)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"atom.png\" alt=\"Atom\" style=\"width:60px\" align=\"left\" vertical-align=\"middle\">\n",
    "\n",
    "## 13. Review\n",
    "*Python 3*\n",
    "\n",
    "----\n",
    "Now you know all about files! You were able to:\n",
    "\n",
    "    1. Open up file objects using open() and with.\n",
    "    2. Read a file’s full contents using Python’s .read() method.\n",
    "    3. Read a file line-by-line using .readline() and .readlines()\n",
    "    4. Create new files by opening them in write-mode.\n",
    "    5. Append to a file by opening a file in append-mode.\n",
    "    6. Apply all of the above to different types of data-carrying files including CSV and JSON.\n",
    "\n",
    "\n",
    "<br/>You have all the skills necessary to read, write, and update files programmatically, a very useful skill in the Python universe! Here's a final example to illustrate the concepts learned:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Thank you for learning about files in Python with us!\n"
     ]
    }
   ],
   "source": [
    "with open('Examples/file.txt') as file_object:\n",
    "    print(file_object.read())"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}