Type_The_Bible_v11.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Type Through The Bible\n",
    "\n",
    "By Kenneth Burchfiel\n",
    "\n",
    "Code is released under the MIT license; Bible verses are from the Web English Bible (Catholic Edition)* and are in the public domain.\n",
    "\n",
    "\\* Genesis was not found within the original WEB Catholic Edition folder, so I copied in files from another Web English Bible translation instead. I imagine, but am not certain, that these files are the same as the actual Catholic Edition Genesis files."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Instructions for getting started:\n",
    "\n",
    "If you have just downloaded this game, you'll want to create new copies of the **WEB_Catholic_Version_for_game_updated.csv**, **results.csv**, and **word_stats.csv** files. That way, the files will show your results and progress, not mine. You can do so using the following steps:\n",
    "\n",
    "1. Rename the existing versions of these files as **WEB_Catholic_Version_for_game_updated_sample.csv**, **results_sample.csv**, **character_stats_sample.csv**, and **word_stats_sample.csv**\n",
    "\n",
    "2. Make a copy of **WEB_Catholic_Version_for_game.csv** and rename it **WEB_Catholic_Version_for_game_updated.csv**.\n",
    "\n",
    "3. Make a copy of **blank_results_file.csv** and rename it **results.csv**.\n",
    "\n",
    "5. Make a copy of **blank_word_stats_file.csv** and rename it **word_stats.csv**.\n",
    "\n",
    "\n",
    "You're now ready to play!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## More documentation to come!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next steps: (Not necessarily in order of importance)\n",
    "\n",
    "* Finish documenting your v2 print code and the variables/ANSI escape codes that it uses\n",
    "* Improve chart formatting (e.g. add titles, legend names, etc.)\n",
    "* Add documentation to other parts of the code as well\n",
    "* Revise file names or create a renaming script so that users won't need to manually rename them.\n",
    "* Explain reasoning for removing character stats (essentially, (1) I didn't think these would be that interesting now that I had added in word stats; (2) there were accuracy concerns; and (3) their presence might slow down V2 of the typing test and thus skew the word-level statistics being calculated\n",
    "* Create a regression analysis that attempts to use data about the verse (length, number of capital letters, etc.)\n",
    "* Add in a help/disclaimer/credits section"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "pd.set_option('display.max_columns', 1000)\n",
    "import time\n",
    "import plotly.express as px\n",
    "from getch import getch # Installed this library using pip install py-getch, not\n",
    "# pip install getch. See https://github.com/joeyespo/py-getch\n",
    "import numpy as np\n",
    "from datetime import datetime, date, timezone # Based on \n",
    "# https://docs.python.org/3/library/datetime.html\n",
    "import os\n",
    "from colorama import just_fix_windows_console, Fore, Back, Style\n",
    "# From https://github.com/tartley/colorama/blob/master/demos/demo01.py\n",
    "just_fix_windows_console() # From https://github.com/tartley/colorama/blob/master/demos/demo01.py\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_response = 'Here is a tesst'\n",
    "' '.join(test_response.rstrip().split(' ')[:-1]) + ' '"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "extra_analyses = False"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Checking whether the program is currently running on a Jupyter notebook:\n",
    "\n",
    "(The program normally uses getch() to begin typing tests; however, I wasn't able to enter input after getch() got called within a Jupyter notebook and thus couldn't begin a typing test in that situation. Therefore, the program will use input() instead of getch() to start tests when running within a notebook.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The following method of determining whether the code is running\n",
    "# within a Jupyter notebook is based on Gustavo Bezerra's response\n",
    "# at https://stackoverflow.com/a/39662359/13097194 . I found that\n",
    "# just calling get_ipython() was sufficient, at least on Windows and within\n",
    "# Visual Studio Code.\n",
    "\n",
    "try: \n",
    "    get_ipython()\n",
    "    run_on_notebook = True\n",
    "except:\n",
    "    run_on_notebook = False\n",
    "\n",
    "# print(run_on_notebook)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_Bible = pd.read_csv('WEB_Catholic_Version_for_game_updated.csv', index_col = 'Verse_Order')\n",
    "df_Bible.sort_index(inplace = True) # Helps ensure that\n",
    "# any code that relies on df_Bible's being sorted from the first to the last\n",
    "# verse will run correctly\n",
    "\n",
    "df_Bible"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_results = pd.read_csv('results.csv', index_col='Test_Number')\n",
    "df_results.sort_index(inplace = True)\n",
    "df_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_word_stats = pd.read_csv('word_stats.csv')\n",
    "df_word_stats"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for column in ['Local_Start_Time', 'UTC_Start_Time']:\n",
    "    df_results[column] = pd.to_datetime(df_results[column])\n",
    "df_results['WPM'] = df_results['WPM'].astype('float') # Prevents a glitch\n",
    "# that can be caused when this column is stored as an object. The WPM\n",
    "# column should only be in object format when the results table is blank.\n",
    "df_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # If you accidentally overwrite your Unix_Start_Time values with something else,\n",
    "# # you can recreate them using UTC_Start_Time values as follows:\n",
    "# # (This code is based on that shown in\n",
    "# # https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#from-timestamps-to-epoch )\n",
    "# df_results['Unix_Start_Time'] = ((df_results['UTC_Start_Time'] - pd.Timestamp(\n",
    "# \"1970-01-01\", tz = 'utc')) // pd.Timedelta(\"1ns\") / 1000000000)\n",
    "# df_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# If you ever need to drop a particular result,\n",
    "# you can do so as follows:\n",
    "# df_results.drop(17, inplace = True)\n",
    "# df_results.to_csv('results.csv') # We want to preserve the index so as not\n",
    "# to lose our 'Test_Number' values\n",
    "# df_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating an RNG seed:\n",
    "# In order to make the RNG values a bit more random, the following code will\n",
    "# derive the RNG seed from the decimal component of the current timestamp.\n",
    "# This seed will change 1 million times each second.\n",
    "\n",
    "# Using the decimal component of time.time() to select an RNG seed:\n",
    "current_time = time.time()\n",
    "decimal_component = current_time - int(current_time) # This \n",
    "# line retrieves the decimal component of current_time. int() is used instead\n",
    "# of np.round() so that the code won't ever round current_time up prior\n",
    "# to the subtraction operation, which would return a different value.\n",
    "# I don't think that converting current_time to an integer (e.g. via\n",
    "# np.int64(current_time)) is necessary, as int() appears to handle at least \n",
    "# some integers larger than 32 bits in size just fine.\n",
    "decimal_component\n",
    "random_seed = round(decimal_component * 1000000)\n",
    "decimal_component, random_seed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rng = np.random.default_rng(random_seed) # Based on\n",
    "# https://numpy.org/doc/stable/reference/random/index.html?highlight=random#module-numpy.random"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[This fantastic answer](https://stackoverflow.com/a/23294659/13097194) by Kevin at Stack Overflow proved helpful in implementing user validation code within this program. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def select_verse():\n",
    "    print(\"Select a verse to type! Enter 0 to receive a random verse\\n\\\n",
    "or enter a verse number (see 'Verse_Order column of\\n\\\n",
    "the WEB_Catholic_Version.csv spreadsheet for a list of numbers to enter)\\n\\\n",
    "to select a specific verse.\\n\\\n",
    "You can also enter -2 to receive a random verse that you haven't yet typed\\n\\\n",
    "or -3 to choose the first Bible verse that hasn't yet been typed.\")\n",
    "    verses_not_yet_typed = df_Bible.query(\n",
    "                \"Typed == 0\").copy().index.to_list()\n",
    "    while True:\n",
    "        try:\n",
    "            response = int(input())\n",
    "        except:\n",
    "            print(\"Please enter an integer corresponding to a particular Bible \\\n",
    "verse or 0 for a randomly selected verse.\")\n",
    "            continue # Allows the user to retry entering a number\n",
    "\n",
    "        if response == 0:\n",
    "            return rng.integers(1, 35380) # Selects any verse within the Bible.\n",
    "            # there are 35,379 verses present, so we'll pass 1 (the first verse)\n",
    "            # and 35,380 (1 more than the last verse, as rng.integers won't \n",
    "            # include the final number within the range) to rng.integers().\n",
    "        elif response == -2:\n",
    "            if len(verses_not_yet_typed) == 0:\n",
    "                print(\"Congratulations! You have typed all verses from \\\n",
    "the Bible, so there are no new verses to type! Try selecting another option \\\n",
    "instead.\")\n",
    "                continue\n",
    "            print(f\"{len(verses_not_yet_typed)} verses have not yet \\\n",
    "been typed.\")\n",
    "            return rng.choice(verses_not_yet_typed) # Chooses one of these\n",
    "            # untyped verses at random\n",
    "        elif response == -3:\n",
    "            if len(verses_not_yet_typed) == 0:\n",
    "                print(\"Congratulations! You have typed all verses from \\\n",
    "the Bible, so there are no new verses to type! Try selecting another option \\\n",
    "instead.\")\n",
    "                continue\n",
    "            print(f\"{len(verses_not_yet_typed)} verses have not yet \\\n",
    "been typed.\")\n",
    "            verses_not_yet_typed.sort() # Probably not necessary, as df_Bible\n",
    "            # was already sorted from the first to the last verse.\n",
    "            return verses_not_yet_typed[0]\n",
    "        \n",
    "        else:\n",
    "            if ((response >= 1) \n",
    "            & (response <= 35379)): # Making sure that the response is \n",
    "                # an integer between 1 and 35,379 (inclusive) so that it \n",
    "                # matches one of the Bible verse numbers present:                    \n",
    "                return response\n",
    "            else: # Will be called if a non-integer number was passed\n",
    "                    # or if the integer didn't correspond to a Bible verse\n",
    "                    # number. \n",
    "                print(\"Please enter an integer between 1 and 35,379.\") # Since\n",
    "                # we're still within a While loop, the user will be returned\n",
    "                # to the initial try/except block.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "column_width = 120\n",
    "column_width - (55 % column_width)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_word_table(verse):\n",
    "    '''Creating a list of words within the verse that we can use for\n",
    "    word-based typing test analyses:\n",
    "    We could use verse.split(' ') to create a list of words. However,\n",
    "    since our goal is to build a list that can then serve as the basis\n",
    "    of analyses, we'll want to create a list that includes:\n",
    "    1. The starting and finishing index of each word within the verse. This will\n",
    "    allow us to determine when the player has arrived at and completed\n",
    "    that word.\n",
    "    2. Words without any spacing or punctuation attached. (Learning how fast\n",
    "    you wrote 'they' is more interesting than is learning how fast \n",
    "    you wrote '\"that' or 'that,'.)\n",
    "    Therefore, the following code is more complex than a simple split()\n",
    "    operation.\n",
    "    This function also counts contractions ('don't', 'can't,\n",
    "    etc.) as individual words. Some care is needed to distinguish between\n",
    "    apostrophes/single quotes that begin contractions and those that start\n",
    "    or end single quotes.'''\n",
    "\n",
    "    first_letter_index_list = []\n",
    "    # Determining the indices of the verse that contain starting letters of\n",
    "    # words: \n",
    "    # (We can find these indices by searching for characters that are preceded\n",
    "    # by a non-alphabetic character *or* an alphabetic character that is also\n",
    "    # the first character in the verse.\n",
    "    for i in range(1, len(verse)): # Starting at 1 rather than 0 because\n",
    "        # we'll sometimes need to reference the preceding character\n",
    "        # (which wouldn't be possible if i were 0)\n",
    "\n",
    "        if i == 1:\n",
    "            if ((verse[i].isalpha()) & (verse[i-1].isalpha() == False)): \n",
    "                # This if statement addresses a special case\n",
    "                # in which a verse begins with ' and a letter. In this \n",
    "                # case, because there are no letters preceding the ' \n",
    "                # (since i is 1), we'll assume that the ' is the start of \n",
    "                # a single quote rather than the beginning of a contraction\n",
    "                # and will thus add the letter to our starting letters list.\n",
    "                # print(\"Found first letter of word (Part A)\")\n",
    "                first_letter_index_list.append(i)\n",
    "                continue # Now that we've added i to the list, we can \n",
    "                # return to the top of the for loop. (This will prevent the\n",
    "                # code from adding duplicate copies of a starting letter\n",
    "                # to first_letter_index_list.\n",
    "\n",
    "            if (verse[i-1].isalpha()):\n",
    "                # print(\"Found first letter of word (Part B)\")\n",
    "                first_letter_index_list.append(i-1) # In this case, the\n",
    "            # character preceding i is the starting letter \n",
    "            # rather than i itself (which may or may not be a letter).\n",
    "                continue\n",
    "\n",
    "        if ((verse[i].isalpha()) & \n",
    "        ((verse[i-1].isalpha() == False) & (verse[i-1] != \"'\"))): # If \n",
    "            # the current character is a letter (e.g. isalpha() is True)\n",
    "            # and the character behind it isn't a letter *and* also isn't\n",
    "            # \"'\" (which is found in contractions), we'll assume that we\n",
    "            # have found the start of a word.\n",
    "            # This if statement will fail to detect the first word in a verse\n",
    "            # if it gets preceded by a single quote (e.g. 'Hello' in \n",
    "            # 'Hello there'), but a previous if statement\n",
    "            # will have already detected it.\n",
    "            # print(\"Found first letter of word (Part C)\")\n",
    "            first_letter_index_list.append(i)\n",
    "            continue\n",
    "    \n",
    "        if i >= 2:\n",
    "            if ((verse[i].isalpha()) & \n",
    "            ((verse[i-1].isalpha() == False)) &\n",
    "            (verse[i-2].isalpha() == False)): # In this case,\n",
    "                # if both of the characters preceding a letter\n",
    "                # aren't letters, then we'll assume that\n",
    "                # this letter begins a word. (This was added in\n",
    "                # to account for words that are preceded by a single\n",
    "                # quote. Checking the character before that single quote\n",
    "                # helps us distinguish apostrophes that begin quotes (which\n",
    "                # won't be preceded by a letter)\n",
    "                # from apostrophes that form part of contractions (which\n",
    "                # will be).\n",
    "                # print(\"Found first letter of word (Part D)\")\n",
    "                first_letter_index_list.append(i)\n",
    "                continue\n",
    "\n",
    "\n",
    "    # print(first_letter_index_list)\n",
    "\n",
    "\n",
    "    # Now that we've determined the index of each word's starting letter,\n",
    "    # we will go through the verse again to locate the final letter\n",
    "    # within each word.\n",
    "    df_word_index_list = []\n",
    "\n",
    "    for index in first_letter_index_list: # This code iterates through\n",
    "        # each word's starting character.\n",
    "        first_letter = verse[index]\n",
    "        # Initializing the word started by this character with \n",
    "        # the starting character:\n",
    "        word = first_letter\n",
    "        i = 0\n",
    "        while True:\n",
    "            i += 1\n",
    "            position_within_verse = index + i\n",
    "            # print(\"i is now\", i)\n",
    "            if position_within_verse < (len(verse) -1): # If this is False,\n",
    "                # then there are at least two characters left in the verse.\n",
    "                # We need to perform this check so that an upcoming if\n",
    "                # statement that reads two characters ahead won't go\n",
    "                # out of bounds.\n",
    "                next_character = verse[position_within_verse]\n",
    "                # print(f\"Next character: '{next_character}'\")\n",
    "                if (next_character.isalpha() == True) | (\n",
    "                    (next_character == \"'\") & \n",
    "                    (verse[position_within_verse + 1].isalpha() == True)): \n",
    "                    # Here, the next character will be added to the word if\n",
    "                    # (1) it's a letter or (2) it's an apostrophe followed\n",
    "                    # by a letter (which indicates that the apostrophe\n",
    "                    # is part of a contraction).\n",
    "                    word += next_character\n",
    "                else:\n",
    "                    last_letter_index = position_within_verse -1\n",
    "                    break # Now that we've found the last letter, we can\n",
    "                    # exit out of the while loop and begin to evaluate\n",
    "                    # the next first index in first_letter_index_list.\n",
    "\n",
    "            elif position_within_verse < len(verse): # In this case, \n",
    "                # we haven't yet made it to the very end of the verse.\n",
    "                # We need to use elif here rather than 'if' so that\n",
    "                # the preceding block of if statements and this one\n",
    "                # will not both return True.\n",
    "                next_character = verse[position_within_verse]\n",
    "                if next_character.isalpha() == True: # \n",
    "                    word += next_character\n",
    "                else:\n",
    "                    last_letter_index = position_within_verse -1\n",
    "                    break\n",
    "            else: # In this case, we're already at the end of the verse,\n",
    "                # so we can instead store the previous index position\n",
    "                # within last_letter_index.\n",
    "                last_letter_index = position_within_verse -1\n",
    "                break\n",
    "\n",
    "        df_word_index_list.append({'first_letter_index':index, \n",
    "            'last_letter_index': last_letter_index,\n",
    "            'word':word})\n",
    "        # print(index, last_letter_index, word)\n",
    "\n",
    "    df_word_index_list = pd.DataFrame(df_word_index_list)\n",
    "    df_word_index_list['previous_character_index'] = np.where(\n",
    "        df_word_index_list['first_letter_index'] != 0,  \n",
    "        df_word_index_list['first_letter_index'] - 1, np.NaN)\n",
    "    df_word_index_list\n",
    "    # print(\"Words identified by create_word_table():\",df_word_index_list['word'].to_list())\n",
    "    return df_word_index_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def run_typing_test(verse_order, results_table, \n",
    "    word_stats_table, test_type = 'v2'):\n",
    "    '''This function calculates how quickly the user types the characters\n",
    "    passed to the Bible verse represented by verse_order, then saves those \n",
    "    results to the DataFrame passed to results_table.'''\n",
    "\n",
    "    # Retrieving the verse to be typed:\n",
    "    # df_Bible uses verse order values for its index, so we can access\n",
    "    # verses within df_Bible by passing verse_order to .at[].\n",
    "    verse = df_Bible.at[verse_order, 'Verse']\n",
    "    book = df_Bible.at[verse_order, 'Book_Name']\n",
    "    chapter = df_Bible.at[verse_order, 'Chapter_Name']\n",
    "    verse_number_within_chapter = df_Bible.at[verse_order, 'Verse_#']\n",
    "    \n",
    "    df_word_index_list = create_word_table(verse)\n",
    "    # print(df_word_index_list)\n",
    "\n",
    "    # I moved these introductory comments out of the following while loop\n",
    "    # in order to simplify the dialogue presented to users during retest\n",
    "    # attempts.\n",
    "    print(f\"Welcome to the typing test! Your verse to type is {book} \\\n",
    "{chapter}:{verse_number_within_chapter} (verse {verse_order} \\\n",
    "within the Bible .csv file).\\n\")\n",
    "    if run_on_notebook == False:\n",
    "        print(\"Press any key to begin typing!\")\n",
    "    else:\n",
    "        print(\"Press Enter to begin the test!\")\n",
    "    \n",
    "    complete_flag = 0\n",
    "    while complete_flag == 0:\n",
    "        print(f\"Here is the verse:\\n\\n{verse}\") \n",
    "\n",
    "        if run_on_notebook == False: # In this case, we can use getch()\n",
    "            # to begin the test.\n",
    "        # time.sleep(3) # I realized that players could actually begin typing\n",
    "        # during this sleep period, thus allowing them to complete the test\n",
    "        # faster than intended. Therefore, I'm now having the test start\n",
    "        # after the player hits a character of his/her choice. getch()\n",
    "        # accomplishes this task well.\n",
    "        # A simpler approach would be to add in an additional input block\n",
    "        # and have the player begin after he/she presses Enter, but that would\n",
    "        # cause the player's right hand to leave the default home row position,\n",
    "        # which could end up slowing him/her down. getch() allows any character\n",
    "        # to be pressed (such as the space bar) and thus avoids this issue.\n",
    "\n",
    "            start_character = getch() # See https://github.com/joeyespo/py-getch\n",
    "        \n",
    "        else: # When running the program within a Jupyter notebook, I wasn't\n",
    "            # able to enter input after getch() was called, so I created\n",
    "            # an alternative start method below that simply uses input().\n",
    "            input()\n",
    "\n",
    "        # The following line determines the width of the terminal just before\n",
    "        # the beginning of the typing test. This width will help determine\n",
    "        # when a line has been completed, which will in turn inform\n",
    "        # when to move the cursor up and how many lines to fill with\n",
    "        # blank spaces.\n",
    "\n",
    "        if run_on_notebook == False: # The following line crashed for me\n",
    "            # when running the program within a notebook.\n",
    "            column_width = os.get_terminal_size().columns\n",
    "        # get_terminal_size() is cross-platform. See\n",
    "        # https://docs.python.org/3.8/library/os.html?highlight=get_terminal_size#os.get_terminal_size\n",
    "        else:\n",
    "            column_width = 120 # The default column width for my\n",
    "            # terminal\n",
    "\n",
    "        # print(f\"Column width is {column_width}\")\n",
    "        print(\"Start!\")      \n",
    "        if test_type == 'v1': \n",
    "            # This is a simple typing test setup that receives input from\n",
    "            # the user when 'Enter' is pressed, then checks that input\n",
    "            # against the verse. Because it doesn't check the response\n",
    "            # for accuracy as the player types, the player might not realize\n",
    "            # a character was mistyped until the very end, which can get\n",
    "            # frustrating. Therefore, I've now added in a new version\n",
    "            # of the test (called 'v2') that can be used instead. \n",
    "            no_mistakes = np.NaN\n",
    "            backspaces_as_pct_of_length = np.NaN\n",
    "            incorrect_characters_as_pct_of_length = np.NaN\n",
    "            unix_start_time = time.time()\n",
    "            local_start_time = pd.Timestamp.now()\n",
    "            utc_start_time = pd.Timestamp.now(timezone.utc)\n",
    "            # I used to use ISO8601-compatible timestamps via the following\n",
    "            # lines, but decided to switch to a value that Pandas would \n",
    "            # immediately recognize as a datetime.\n",
    "            # local_start_time = datetime.now().isoformat()\n",
    "            # utc_start_time = datetime.now(timezone.utc).isoformat()\n",
    "\n",
    "            typing_start_time = time.perf_counter_ns()\n",
    "            verse_response = input() \n",
    "            # The following code will execute once the player finishes typing and\n",
    "            # hits Enter. (Having the program evaluate the player's entry only after\n",
    "            # 'Enter' is pressed isn't the best option, as the time required to\n",
    "            # hit Enter will reduce the player's reported WPM. Version v2, \n",
    "            # shown below, stops the test right when the final correct\n",
    "            # character is typed, which will make the final WPM slightly faster.\n",
    "            typing_end_time = time.perf_counter_ns()\n",
    "            typing_time = (typing_end_time - typing_start_time) / 1000000000\n",
    "\n",
    "        elif test_type == 'v2':\n",
    "            # This version of the test checks the player's input after\n",
    "            # each character is typed. If the player's input is correct\n",
    "            # so far, the text will be highlighted green; otherwise,\n",
    "            # it will be highlighted red. (This allows the player to be \n",
    "            # notified of an error without the need for a line break\n",
    "            # in the console, which could prove distracting.)\n",
    "            # This function has been tested on Windows, but not yet \n",
    "            # on Mac or Linux. The use of the Colorama library should\n",
    "            # make it cross-platform, however.\n",
    "            verse_response = '' # This string will store the player's \n",
    "            # response.\n",
    "            no_mistakes = 1 # This flag will get set to 0 if the player makes\n",
    "            # a mistake. If it remains at 1 throughout the race, then\n",
    "            # a mistake-free race will get logged in results_table.\n",
    "            previous_line_count = 1\n",
    "            backspace_count = 0\n",
    "            incorrect_character_count = 0\n",
    "            word_stats_list = []\n",
    "            # code_execution_time_list = []\n",
    "\n",
    "            # The following two lists will keep track of first- and last-letter\n",
    "            # indices that have already been reached by the player. These lists\n",
    "            # will be incorporated into if statements so that (1) timing for a\n",
    "            # word will begin only if that word's first letter index is not in \n",
    "            # first_letter_indices_reached and (2) timing for a word will end\n",
    "            # only if that word's last letter index is not in\n",
    "            # last_letter_indices_reached. This will prevent duplicate \n",
    "            # timing entries for the same word from getting added to the word\n",
    "            # stats list; in addition, it will prevent the player from\n",
    "            # 'resetting' the timing for a given word by returning to its index.\n",
    "            first_letter_indices_reached = []\n",
    "            last_letter_indices_reached = []\n",
    "\n",
    "            first_keypress = 1 # Designates whether or not the player is \n",
    "            # currently making his or her first keypress. This flag, which\n",
    "            # will be set to 0 after the first keypress is made,\n",
    "            # will help determine whether or not to begin timing the player's\n",
    "            # first word.\n",
    "            last_letter_index = -1 # Initializing this variable with a number\n",
    "            # that will never occur within the game so that this value won't\n",
    "            # get interpreted as an actual word starting point.\n",
    "\n",
    "            unix_start_time = time.time()\n",
    "            local_start_time = pd.Timestamp.now()\n",
    "            utc_start_time = pd.Timestamp.now(timezone.utc)\n",
    "            typing_start_time = time.perf_counter_ns()\n",
    "            # Using perf_counter_ns() allows for more accurate time duration\n",
    "            # calculations than does time.time(). See\n",
    "            # https://docs.python.org/3/library/time.html#time.perf_counter_ns\n",
    "\n",
    "\n",
    "            while True: # This while loop allows the player to \n",
    "                # enter multiple characters.\n",
    "                # The following if statement determines whether to \n",
    "                # begin timing the player's first word. Timing will only\n",
    "                # begin\n",
    "                if (len(verse_response) == 0) & (\n",
    "                    0 in df_word_index_list['first_letter_index'].to_list()\n",
    "                    ) & (first_keypress == 1):\n",
    "                    word_start_time = typing_start_time\n",
    "                    last_letter_index = df_word_index_list.query(\n",
    "                        'first_letter_index == 0').copy().iloc[0][\n",
    "                            'last_letter_index']\n",
    "                    word = df_word_index_list.query(\n",
    "                        'first_letter_index == 0').copy().iloc[0][\n",
    "                            'word']\n",
    "                    first_letter_indices_reached.append(0)\n",
    "                    # print(f\" Started typing {word}.\")\n",
    "                    typed_word_without_mistakes = 1\n",
    "                    # print(f\" {last_letter_index}\")\n",
    "                # code_execution_end_time = time.perf_counter_ns()\n",
    "                # if first_keypress == 0: \n",
    "                #     code_execution_time_list.append((code_execution_end_time - character_press_time) / 1000000)\n",
    "                    # We divide by 1,000,000 to convert from nanoseconds (the output\n",
    "                    # of perf_counter_ns() to milliseconds.\n",
    "                character = getch() # getch() allows each character to be \n",
    "                # checked, making it easier to identify mistyped words.\n",
    "                character_press_time = time.perf_counter_ns()\n",
    "                first_keypress = 0\n",
    "\n",
    "                if character == b'\\x08': \n",
    "                    # This will return True if the user hits backspace.\n",
    "                    # In this case, we'll want to remove the latest character\n",
    "                    # from verse_response in order to keep that value\n",
    "                    # in sync with what the player sees on the screen.\n",
    "                    # Calling print(character) after\n",
    "                    # hitting backspace revealed that b'\\x08' was the code\n",
    "                    # associated with the backspace key. \n",
    "                    backspace_count += 1\n",
    "                    verse_response = verse_response[:-1] # Trims the last\n",
    "                    # value off verse_response.\n",
    "                    typed_word_without_mistakes = 0\n",
    "                elif character == b'\\x7f': # This is the output of Ctrl + Backspace\n",
    "                    # on Windows; I haven't yet tested its output on Mac or Linux.\n",
    "                    # The following code allows this sequence to remove the last\n",
    "                    # word typed, which is often a faster deletion method than\n",
    "                    # pressing the backspace key repeatedly.\n",
    "                    verse_response = ' '.join(verse_response.rstrip().split(' ')[:-1])\n",
    "                    # The above line first removes all spaces from the right of the last\n",
    "                    # word typed so that they won't interfere with the split() call.\n",
    "                    # It then splits the response into individual words; deselects\n",
    "                    # the last word; and joins the response back together.\n",
    "                    if len(verse_response) != 0: # As long as we're not at the start\n",
    "                        # of the console, it will be useful to add the space after\n",
    "                        # the last word back in so that the player doesn't need\n",
    "                        # to retype it.\n",
    "                        verse_response += ' '\n",
    "                    backspace_count += 1 # I was considering increasing backspace_count\n",
    "                    # here by the number of characters removed through Ctrl + Backspace,\n",
    "                    # but since the player still only has to press Backspace once\n",
    "                    # in this operation, I decided to still increment it by one.\n",
    "                    # This means that backspace_count may diverge significantly\n",
    "                    # from incorrect_character_count and thus be a poorer measure\n",
    "                    # of accuracy.\n",
    "                    typed_word_without_mistakes = 0\n",
    "                elif character == b'`':\n",
    "                    print(Style.RESET_ALL) # Resets the color of the text.\n",
    "                    verse_response += character.decode('ascii') # The presence\n",
    "                    # of this character within verse_response will instruct\n",
    "                    # the program to exit the user out of this test later on.\n",
    "                    # See https://pypi.org/project/colorama/\n",
    "                    break\n",
    "                else: \n",
    "                    # The following line adds the latest character typed\n",
    "                    # to verse_response.\n",
    "                    try:\n",
    "                        verse_response += character.decode('ascii')  \n",
    "                        # See https://stackoverflow.com/questions/17615414/how-to-convert-binary-string-to-normal-string-in-python3\n",
    "                    except: # Keys that fall out of the ascii subset, such as \n",
    "                        # arrow keys, could cause the above line to crash. Therefore,\n",
    "                        # when the above line fails to work, the following 'continue'\n",
    "                        # statement will allow the program to ignore the key and move\n",
    "                        # back to the beginning of the loop.\n",
    "                        continue\n",
    "\n",
    "                # Determining which color to use for the text:\n",
    "                if verse[0:len(verse_response)] == verse_response: # If this returns\n",
    "                    # True, the player's response is correct so far.\n",
    "                    text_color = Fore.GREEN\n",
    "                    # Adding the time it took to type the last character\n",
    "                    # to the list: (Note that the time it takes to \n",
    "                    # enter a backspace won't be included.)\n",
    "                    verse_response_minus_one = len(verse_response) -1 # The character\n",
    "                    # Index values in df_word_index_list start at 0, so this variable\n",
    "                    # will help us convert between verse lengths and index positions.        \n",
    "                    \n",
    "                    if (character != b'\\x08') & (character != b'\\x7f'):\n",
    "\n",
    "                        # Checking whether a word has begun or ended:\n",
    "                        # We're placing these checks within the correct response and\n",
    "                        # no backspace clauses so that a typo or backspace won't\n",
    "                        # count as having correctly begun or ended a word. (I'm not sure\n",
    "                        # it would actually be possible to trigger the word start or \n",
    "                        # end checks with Ctrl + Backspace, but I'll include it here\n",
    "                        # just in case a clever user figures out how.)\n",
    "                        # Note that last_letter_indices_reached and \n",
    "                        # first_letter_indices_reached will be checked for the\n",
    "                        # presence of verse_response_minus_one. If it's already\n",
    "                        # there, then no action will be taken, as timing\n",
    "                        # for that word has already ended or started.\n",
    "\n",
    "                        # Checking whether the player has finished typing a word:\n",
    "                        if (verse_response_minus_one == last_letter_index) & (\n",
    "                            last_letter_index not in last_letter_indices_reached):\n",
    "                            word_end_time = character_press_time\n",
    "                            # print(f\"Finished typing {word} in \\\n",
    "            # {(word_end_time - word_start_time) / 1000000} ms. typed_word_without_mistakes \\\n",
    "            # is set to {typed_word_without_mistakes}.\")\n",
    "                            word_stats_list.append({\"word\":word, \n",
    "                            \"word_duration (ms)\": (word_end_time - \n",
    "                            word_start_time) / 1000000, \n",
    "                            \"typed_word_without_mistakes\":typed_word_without_mistakes})\n",
    "                            last_letter_indices_reached.append(last_letter_index)\n",
    "                            # Other analyses can be added to our \n",
    "                            # word stats table later on, so we don't\n",
    "                            # need to compute them now.\n",
    "\n",
    "                        # Checking whether the player has started typing a word:\n",
    "                        if (verse_response_minus_one in df_word_index_list[\n",
    "                            'previous_character_index'].to_list()) & (\n",
    "                            verse_response_minus_one not in first_letter_indices_reached):\n",
    "                            # If this returns true, we know we're\n",
    "                            # at the starting point of a new word.\n",
    "                            # print(verse_response_minus_one, df_word_index_list['previous_character_index'])\n",
    "                            # print(\"Start of new word detected (Point A).\")\n",
    "                            typed_word_without_mistakes = 1\n",
    "                            verse_response_minus_one = len(verse_response) -1\n",
    "                            word_start_time = character_press_time # The start time of \n",
    "                            # this new word is defined as the time that the character \n",
    "                            # preceding the word was pressed.\n",
    "                            last_letter_index = df_word_index_list.query(\n",
    "                            \"previous_character_index == @verse_response_minus_one\").iloc[\n",
    "                            0]['last_letter_index']\n",
    "                            word = df_word_index_list.query(\n",
    "                            \"previous_character_index == @verse_response_minus_one\").iloc[\n",
    "                            0]['word']\n",
    "                            # print(f\" Started typing {word}.\")\n",
    "                            first_letter_indices_reached.append(\n",
    "                            verse_response_minus_one)\n",
    "\n",
    "                else:\n",
    "                    no_mistakes = 0 # This flag will remain at 0 for the \n",
    "                    # rest of the race.\n",
    "                    typed_word_without_mistakes = 0 \n",
    "                    text_color = Fore.RED \n",
    "                    if (character != b'\\x08') & (character != b'\\x7f'): # Backspaces\n",
    "                        # won't be counted as part of the incorrect character tally so that\n",
    "                        # players won't be double-penalized for mistyping\n",
    "                        # a character.\n",
    "                        incorrect_character_count += 1\n",
    "                \n",
    "                # Printing the player's response so far: \n",
    "                \n",
    "                # This process will involve printing the entirety of verse_response\n",
    "                # after each character is pressed rather than \n",
    "                # just the most recent character. \n",
    "                # This code is more complex than a regular print statement, but it has\n",
    "                # several advantages:\n",
    "                # 1. It allows the player to quickly determine when a typo has \n",
    "                # occurred (as the text will show up in red rather than in green).\n",
    "                # 2. It supports the use of backspace to correct responses on \n",
    "                # previous lines. (I wasn't able to navigate to a previous line \n",
    "                # using backspace when printing single characters at a time.)\n",
    "                # 3. It allows the cursor to always appear to the right of the most\n",
    "                # recent character. If the latest typed line takes up the entire\n",
    "                # width of the console, the cursor will appear on the left of \n",
    "                # the following line.\n",
    "                # The development of this code involved a decent amount of trial and \n",
    "                # error, but I'll try to explain the function of each line in order to\n",
    "                # make the final result more intuitive.\n",
    "\n",
    "\n",
    "                line_count = ((len(verse_response)) // column_width) + 1\n",
    "                # Calculates the number of lines on which the player's\n",
    "                # response appears. The inclusion of the max() function ensures\n",
    "                # that line_count will always be at least 1.\n",
    "                # 1 is added to verse_response because, if the response has extended\n",
    "                # to the right side of the terminal, another line will get added\n",
    "                # in (via code below) to make room for the cursor. Thus, line_count\n",
    "                # needs to be incremented by 1 in that case to reflect the terminal's\n",
    "                # output.\n",
    "\n",
    "                # The following code adjusts to changes in the response's line count.\n",
    "                # If the line count goes up (as indicated by line_count's exceeding\n",
    "                # previous_line_count), the response printout will be preceded\n",
    "                # by a newline so that more space is available to print the longer\n",
    "                # text. If the line count goes down (e.g. due to a backspace),\n",
    "                # the the printout will be preceded by an up cursor statement\n",
    "                # since less space will be needed to print the line.\n",
    "                if line_count > previous_line_count:\n",
    "                    line_change_shift_command = '\\n'\n",
    "                elif line_count < previous_line_count: \n",
    "                    line_change_shift_command = \"\\033[A\"\n",
    "                    # \"\\033[A\" is an ANSI escape code that moves the cursor \n",
    "                    # up by one line. See\n",
    "                    # at https://pypi.org/project/colorama/\n",
    "                else: # No command is necessary if the number of lines is the same \n",
    "                    # as before\n",
    "                    line_change_shift_command = ''\n",
    "\n",
    "                previous_line_count = line_count # Updates the value of \n",
    "                # previous_line_count\n",
    "\n",
    "                # If more than one line is present, we'll need to move the cursor\n",
    "                # up by the number of lines -1. Otherwise, an extra line will get\n",
    "                # printed with each character.\n",
    "                if line_count > 1:\n",
    "                    up_command = f\"\\033[{line_count -1}A\"\n",
    "                # This ANSI escape code, based on Richard's response at\n",
    "                # https://stackoverflow.com/a/33206814/13097194 ,\n",
    "                # will move the cursor up line_count -1 times.\n",
    "                else:\n",
    "                    up_command = '' # If the response is still on the first line,\n",
    "                #     # there's no need to move the cursor up, as its vertical\n",
    "                #     # position won't shift in the process of writing the response.\n",
    "\n",
    "                clear_text_to_right_command = '\\033[0K' # This code clears out all text\n",
    "                # to the right of the cursor. This will allow us to make text disappear\n",
    "                # after the user hits Backspace or Ctrl + Backspace.\n",
    "                # Based on\n",
    "                # https://en.wikipedia.org/wiki/ANSI_escape_code\n",
    "                # and on https://pypi.org/project/colorama/\n",
    "\n",
    "                # In order to make the cursor appear on a new line when the player\n",
    "                # has reached the end of a line, a space will get added to the verse\n",
    "                # after it is printed (see below for more details). To compensate for \n",
    "                # this space, the ANSI escape code '\\033[D' will usually get printed;\n",
    "                # this code will then move the cursor one column back. However, \n",
    "                # if the verse is one column away from the end of the line, this\n",
    "                # step won't be necessary, so an empty string can get passed\n",
    "                # to left_cursor_shift instead.\n",
    "                if column_width - (len(verse_response) % column_width) == 1:\n",
    "                    left_cursor_shift = ''\n",
    "                else:\n",
    "                    left_cursor_shift = '\\033[D'\n",
    "\n",
    "                # Printing out various variables related to\n",
    "                # the subsequent print statement can be useful for debugging.\n",
    "                # print(\"\\033A\",line_count, len(verse_response), column_width, \n",
    "                # column_width - (len(verse_response) % column_width) == 1)\n",
    "\n",
    "\n",
    "                # The following print statement displays the latest version\n",
    "                # of verse_response on the screen. It does so using the following steps:\n",
    "                # (Note: Many of these items are ANSI escape codes rather than\n",
    "                # text.)\n",
    "\n",
    "                # 1. \\r (carriage return) moves\n",
    "                # the cursor to the left of the line. \n",
    "                \n",
    "                # 2. clear_text_to_right_command,\n",
    "                # an ANSI escape code, is printed; this clears out all text\n",
    "                # to the right of the cursor (which, due to the cursor's positioning,\n",
    "                # equals all of the line's text).\n",
    "                # This command also appears at the end of the print statement;\n",
    "                # however, both commands are necessary. If the player's use of \n",
    "                # Ctrl + Backspace causes the end of the output to move up one line,\n",
    "                # the first clear_text_to_right command will remove deleted text\n",
    "                # on the lower line, and the last one will remove deleted text on the \n",
    "                # upper line.\n",
    "                \n",
    "                # 3. line_change_shift_command is printed in order to adjust for\n",
    "                # changes in the output's line count. (See notes above for more details.)\n",
    "                \n",
    "                # 4. up_command is printed so that the cursor will be high enough to \n",
    "                # print all of the response's lines without creating an unnecessary\n",
    "                # extra line.\n",
    "                \n",
    "                # 5. text_color gets printed; this will make the verse red if a mistake\n",
    "                # is present and green otherwise.\n",
    "\n",
    "                # 6. Finally, the verse itself gets printed, followed by a space (in order\n",
    "                # to make a new line appear if the player has reached the end of the line).\n",
    "                # Note that no other spaces are present within the print statement.\n",
    "\n",
    "                # 7. In order to compensate for this space, left_cursor_shift is called,\n",
    "                # which usually moves the cursor back one column. (See above for more\n",
    "                # details.)\n",
    "\n",
    "                # 8. Finally, clear_text_to_right_command is printed once again.\n",
    "\n",
    "\n",
    "                print(f\"\\r{clear_text_to_right_command}\\\n",
    "{line_change_shift_command}{up_command}{text_color}\\\n",
    "{verse_response} {left_cursor_shift}{clear_text_to_right_command}\", end = '')\n",
    "                # For the use of Colorama to produce red and green text, see\n",
    "                # https://pypi.org/project/colorama/\n",
    "                # and https://stackoverflow.com/a/3332860/13097194\n",
    "                \n",
    "                if verse_response == verse: # Note that, unlike with version\n",
    "                    # v1, the player does not need to hit 'Enter' in order\n",
    "                    # to end the typing test after writing a completed\n",
    "                    # verse. This should speed up his/her WPM as a result.\n",
    "                    typing_end_time = time.perf_counter_ns()\n",
    "                    typing_time = (\n",
    "                        typing_end_time - typing_start_time) / 1000000000\n",
    "                    # Dividing by 1 billion to convert typing_time from nanoseconds\n",
    "                    # to seconds\n",
    "                    print('\\n'*line_count+'Success!') # The cursor\n",
    "                    # needs to be moved past the lines already printed\n",
    "                    # so that 'Success' won't overwrite any of the words.\n",
    "                    print(Style.RESET_ALL)\n",
    "\n",
    "                    # Accuracy calculations:\n",
    "                    # Calculating backspaces as a percentage of verse length:\n",
    "                    backspaces_as_pct_of_length = (\n",
    "                        100 * backspace_count / len(verse)) # The 100* \n",
    "                        # multiplier converts these values from \n",
    "                        # proportions to percentages.\n",
    "\n",
    "                    # Calculating incorrect entries as a percentage of verse\n",
    "                    # length:\n",
    "                    incorrect_characters_as_pct_of_length = (\n",
    "                    100 * incorrect_character_count / len(verse))\n",
    "                    # print(\"Backspace count and incorrect character count:\",\n",
    "                    # backspace_count, incorrect_character_count)\n",
    "                \n",
    "                    # Calculating timing statistics at the word level:\n",
    "                    word_stats_for_latest_test = pd.DataFrame(word_stats_list)\n",
    "                    # print(word_stats_for_latest_test)\n",
    "                    # print(f\"Code execution stats: Median: {np.median(\n",
    "                    # code_execution_time_list)}, Mean: {np.mean(\n",
    "                    # code_execution_time_list)}, Max: {max(\n",
    "                    # code_execution_time_list)}, Min: {min(\n",
    "                    # code_execution_time_list)}. All execution times: {\n",
    "                    # code_execution_time_list}\")\n",
    "                    break\n",
    "\n",
    "        if verse_response == verse:\n",
    "            print(f\"Well done! You typed the verse correctly.\")\n",
    "            complete_flag = 1 # Setting this flag to 1 allows the player to exit\n",
    "            # out of the while statement.\n",
    "        elif (verse_response.lower() == 'exit') or ('`' in verse_response):\n",
    "            print(\"Exiting typing test.\")\n",
    "            return results_table, word_stats_table # Exits the function without saving the \n",
    "            # current test to results_table or df_Bible. This function has\n",
    "            # been updated to work with both versions of the typing\n",
    "            # test.\n",
    "        else:\n",
    "            print(\"Sorry, that wasn't the correct input.\")   \n",
    "            # Identifying incorrectly typed words:\n",
    "            verse_words = verse.split(' ')\n",
    "            verse_response_words = verse_response.split(' ')[0:len(verse_words)]\n",
    "            # I added in the [0:len(verse_words)] filter so that the following\n",
    "            # for loop would not attempt to access more words that were \n",
    "            # present in the original verse (which would cause the game\n",
    "            # to crash with an IndexError).\n",
    "            for i in range(len(verse_response_words)):\n",
    "                if verse_response_words[i] != verse_words[i]:\n",
    "                    print(f\"Word number {i} ('{verse_words[i]}') \\\n",
    "was typed '{verse_response_words[i]}'.\")\n",
    "                    # If the response has more or fewer words than the original\n",
    "                    # verse, some correctly typed words might appear within\n",
    "                    # this list also.\n",
    "            print(\"Try again!\") # complete_flag will still be 0 in this case,\n",
    "            # so the while loop will continue back to the beginning.\n",
    "\n",
    "    # Calculating typing statistics and storing them within a single-row\n",
    "    # DataFrame:\n",
    "\n",
    "    cps = len(verse) / typing_time # Calculating characters per second\n",
    "    wpm = cps * 12 # Multiplying by 60 to convert from characters to minutes, \n",
    "    # then dividing by 5 to convert from characters to words.\n",
    "    wpm\n",
    "\n",
    "    if test_type == 'v2':\n",
    "        word_stats_table = pd.concat(\n",
    "        [word_stats_table, word_stats_for_latest_test]).reset_index(\n",
    "        drop = True)\n",
    "\n",
    "    # Creating a single-row DataFrame that stores the player's results:\n",
    "    # +1 is added to the length of the results table in order to begin\n",
    "    # test number values at 1 (rather than 0).\n",
    "    df_latest_result = pd.DataFrame(index = [\n",
    "        len(results_table)+1], data = {'Unix_Start_Time':unix_start_time, \n",
    "    'Local_Start_Time':local_start_time,\n",
    "    'UTC_Start_Time':utc_start_time,\n",
    "    'Characters':len(verse),\n",
    "    'Seconds':typing_time, \n",
    "    'CPS': cps,\n",
    "    'WPM':wpm,\n",
    "    'Mistake_Free_Test':no_mistakes,\n",
    "    'Backspaces as % of Verse Length': backspaces_as_pct_of_length,\n",
    "    'Incorrect Characters as % of Verse Length': \\\n",
    "incorrect_characters_as_pct_of_length,\n",
    "    'Book': book,\n",
    "    'Chapter': chapter,\n",
    "    'Verse #': verse_number_within_chapter,\n",
    "    'Verse':verse, \n",
    "    'Verse_Order':verse_order})\n",
    "    df_latest_result.index.name = 'Test_Number'\n",
    "    df_latest_result\n",
    "\n",
    "    # Adding this new row to results_table:\n",
    "    results_table = pd.concat([results_table, df_latest_result])\n",
    "    # Note: I could also have used df.at or df.iloc to add a new row\n",
    "    # to df_latest_result, but I chose a pd.concat() setup in order to ensure\n",
    "    # that the latest result would never overwrite an earlier result.\n",
    "    \n",
    "\n",
    "    # Rank and percentile data needs to be recalculated after each test,\n",
    "    # as later results can affect the rank and percentile of earlier results.\n",
    "    # I could compute these statistics later, but calculating them here\n",
    "    # allows the player to view his/her statistics after each test.\n",
    "\n",
    "    results_table['WPM_Rank'] = results_table['WPM'].rank(\n",
    "    ascending = False, method = 'min').astype('int')\n",
    "    results_table['WPM_Percentile'] = results_table['WPM'].rank(pct=True)*100\n",
    "    latest_rank = results_table.iloc[-1]['WPM_Rank']\n",
    "    # Note: These percentile results may differ from the results\n",
    "    # calculated by np.quartile later in this function, likely a result of\n",
    "    # different calculation methodologies. These differences should narrow\n",
    "    # as more tests are completed.\n",
    "\n",
    "    latest_percentile = results_table.iloc[-1]['WPM_Percentile'].round(3)\n",
    "    number_of_tests = len(results_table)\n",
    "    last_10_avg = results_table['WPM'].rolling(10).mean().iloc[-1]\n",
    "    \n",
    "    # The player's rolling 10-race average will be NaN until he/she has\n",
    "    # completed 10 tests. Therefore, the following if statement will \n",
    "    # return a blank last 10 races report unless at least 10 tests\n",
    "    # are present in results_table.\n",
    "    if len(results_table) >= 10:\n",
    "        last_10_report = f' You have averaged \\\n",
    "{last_10_avg.round(3)} WPM over your last 10 tests.' # The space\n",
    "    # space before 'You' separates this text from the rest of\n",
    "    # the print statement below.\n",
    "    else:\n",
    "        last_10_report = ''\n",
    "\n",
    "    print(f\"Your CPS and WPM were {round(cps, 3)} and {round(wpm, 3)}, \\\n",
    "respectively. Your WPM percentile was {latest_percentile} \\\n",
    "({latest_rank} out of {number_of_tests} tests).{last_10_report}\")  \n",
    "\n",
    "    # Updating df_Bible to store the player's results: (This will allow the\n",
    "    # player to track how much of the Bible he/she has typed so far)\n",
    "    df_Bible.at[verse_order, 'Typed'] = 1 # Denotes that this verse\n",
    "    # has now ben typed. Note that .loc[] would not work here because\n",
    "    # we are updating a value, not merely retrieving one.\n",
    "    df_Bible.at[verse_order, 'Tests'] += 1 # Keeps track of how \n",
    "    # many times this verse has been typed\n",
    "    fastest_wpm = df_Bible.at[verse_order, 'Fastest_WPM']\n",
    "    if ((pd.isna(fastest_wpm) == True) | (wpm > fastest_wpm)): \n",
    "        # In these cases, we should replace the pre-existing Fastest_WPM value\n",
    "        # with the WPM the player just achieved.\n",
    "        # I found that 5 > np.NaN returned False, so if I only checked for\n",
    "        # wpm > fastest_wpm, blank fastest_wpm values would never get overwritten.\n",
    "        # Therefore, I chose to also check for NaN values \n",
    "        # in the above if statement.\n",
    "        df_Bible.at[verse_order, 'Fastest_WPM'] = wpm\n",
    "\n",
    "    # Autosaving results as separate files: (That way, if the script crashes,\n",
    "    # the player won't lose all of his/her progress.)\n",
    "    if verse_order % 10 == 0: # The autosave will only take place for ~10%\n",
    "        # of the user's verses, thus saving processing time (and wear\n",
    "        # on the user's SSD, though I'm not sure how much of a difference\n",
    "        # this would make for the SSD's longevity).\n",
    "        try:\n",
    "            results_table.to_csv('df_results_autosave.csv', index = True)\n",
    "            df_Bible.to_csv('WEB_Catholic_Version_for_game_updated_autosave.csv',\n",
    "            index = True)\n",
    "            word_stats_table.to_csv('word_stats_autosave.csv', \n",
    "            index = False)\n",
    "            print(\"Autosave complete.\")\n",
    "        except:\n",
    "            print(\"At least one of the autosave files could not be saved. Close \\\n",
    "    out of any open autosave files before starting the next test \\\n",
    "    so that they can be updated.\")\n",
    "    return (results_table, word_stats_table)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def select_subsequent_verse(previous_verse_order):\n",
    "    '''This function allows the player to specify which verse to\n",
    "    type next, or, alternatively, to exit the game.'''\n",
    "    print(\"Press 0 to retry the verse you just typed; \\\n",
    "1 to type the next verse; 2 to type the next verse that hasn't yet been typed; \\\n",
    "-3 to select the first verse that hasn't been typed; \\\n",
    "3 to select a different verse; \\\n",
    "-1 to save your results and exit; \\\n",
    "and -2 to save your results without running the analysis \\\n",
    "portion of the script.\") # The analysis portion can take a decent amount of\n",
    "# time to run, which is why an option to save without running these analyses\n",
    "# was added in. These analyses can then get updated during a later session.\n",
    "    verses_not_yet_typed = df_Bible.query(\n",
    "    \"Typed == 0\").copy().index.to_list()\n",
    "    while True: \n",
    "            try:\n",
    "                response = int(input())\n",
    "            except: # The user didn't enter a number.\n",
    "                print(\"Please enter a number.\")      \n",
    "                continue\n",
    "            if response == 0:\n",
    "                return previous_verse_order\n",
    "            elif response == 1:\n",
    "                if previous_verse_order == 35379: # The verse order value\n",
    "                    # corresponding to the final verse of Revelation\n",
    "                    print(\"You just typed the last verse in the Bible, so \\\n",
    "there's no next verse to type! Please enter an option other than 1.\\n\")\n",
    "                    continue\n",
    "                else:\n",
    "                    return previous_verse_order + 1\n",
    "            elif response == 2:\n",
    "                # In this case, we'll retrieve a list of verses that haven't\n",
    "                # yet been typed; filter that list to include only verses\n",
    "                # greater than previous_verse_order; and then select\n",
    "                # the first verse within that list (i.e. the next \n",
    "                # untyped verse).\n",
    "                if len(verses_not_yet_typed) == 0:\n",
    "                    print(\"Congratulations! You have typed all verses from \\\n",
    "the Bible, so there are no new verses to type! Try selecting another option \\\n",
    "instead.\")\n",
    "                    continue\n",
    "                print(f\"{len(verses_not_yet_typed)} verses have not yet \\\n",
    "been typed.\")\n",
    "                verses_not_yet_typed.sort() \n",
    "                next_untyped_verses = [verse for verse in verses_not_yet_typed \n",
    "                if verse > previous_verse_order]\n",
    "                return next_untyped_verses[0]\n",
    "            \n",
    "            elif response == -3:\n",
    "                if len(verses_not_yet_typed) == 0:\n",
    "                    print(\"Congratulations! You have typed all verses from \\\n",
    "the Bible, so there are no new verses to type! Try selecting another option \\\n",
    "instead.\")\n",
    "                    continue\n",
    "                print(f\"{len(verses_not_yet_typed)} verses have not yet \\\n",
    "been typed.\")\n",
    "                verses_not_yet_typed.sort()\n",
    "                return verses_not_yet_typed[0]\n",
    "\n",
    "            elif response == 3:\n",
    "                return select_verse()\n",
    "            elif response in [-1, -2]:\n",
    "                return response\n",
    "            else: # A number other than one of the above options was passed.\n",
    "                print(\"Please enter a whole number from -3 to 3 (inclusive).\\n\")  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def calculate_current_day_results(df):\n",
    "    ''' This function reports the number of characters, total verses, and \n",
    "    unique verses that the player has typed so far today.'''\n",
    "    df_current_day_results = df[pd.to_datetime(\n",
    "        df['Local_Start_Time']).dt.date == datetime.today().date()].copy()\n",
    "    if len(df_current_day_results) == 0:\n",
    "        result_string = \"You haven't typed any Bible verses yet today.\"\n",
    "    else:\n",
    "        characters_typed_today = df_current_day_results['Characters'].sum()\n",
    "        total_verses_typed_today = len(df_current_day_results)\n",
    "\n",
    "        # Allowing for both singular and plural versions of 'verse' to \n",
    "        # be displayed:\n",
    "        if total_verses_typed_today == 1:\n",
    "            total_verses_string = 'verse'\n",
    "        else:\n",
    "            total_verses_string = 'verses'\n",
    "\n",
    "        unique_verses_typed_today = len(df_current_day_results[\n",
    "            'Verse_Order'].unique())\n",
    "\n",
    "        if unique_verses_typed_today == 1:\n",
    "            unique_verses_string = 'verse'\n",
    "        else:\n",
    "            unique_verses_string = 'verses'\n",
    "\n",
    "        average_wpm_today = round(df_current_day_results['WPM'].mean(), 3)\n",
    "        median_wpm_today = round(df_current_day_results['WPM'].median(), 3)\n",
    "        result_string = f\"So far today, you have typed \\\n",
    "{characters_typed_today} characters from {total_verses_typed_today} Bible \\\n",
    "{total_verses_string} (including {unique_verses_typed_today} unique \\\n",
    "{unique_verses_string}). Your mean and median WPM today are \\\n",
    "{average_wpm_today} and {median_wpm_today}, respectively.\"\n",
    "    return result_string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def run_game(results_table, word_stats_table):\n",
    "    '''This function runs Type Through the Bible by \n",
    "    calling various other functions. It allows users to select\n",
    "    verses to type, then runs typing tests and stores the results in\n",
    "    the DataFrame passed to results_table.'''\n",
    "    \n",
    "    print(\"Welcome to Type Through the Bible!\")\n",
    "    # The game will now share the player's progress for the current day:\n",
    "    print(calculate_current_day_results(results_table))\n",
    "\n",
    "    if run_on_notebook == True: # I haven't been able to get version\n",
    "        # v2 of the typing test to work within a Jupyter notebook, \n",
    "        # so the following line forces notebook-based runs to use version v1.\n",
    "        typing_test_version = 'v1'\n",
    "    else: # In this case, the user gets to choose whether to to use \n",
    "        # v1 or v2.\n",
    "        print(\"To switch to a simpler typing test method that doesn't \\\n",
    "check your input as you type, enter v1. Otherwise, to stick with the \\\n",
    "recommended version, press Enter.\")\n",
    "        response = input()\n",
    "        if (response == 'v1') or (run_on_notebook == True): # Version 2 likely\n",
    "            # won't work within Jupyter notebooks, \n",
    "            # so the version will always be kept as v1 for notebook users.\n",
    "            typing_test_version = 'v1'\n",
    "        else:\n",
    "            typing_test_version = 'v2'\n",
    "\n",
    "    # The method for exiting a test in progress differs by typing test\n",
    "    # version, so the game will now explain how the player can exit out of \n",
    "    # his/her version of the test.\n",
    "    if typing_test_version == 'v1':\n",
    "        print(\"Version 1 selected. Note that you can exit a test in \\\n",
    "progress by typing 'exit' and then hitting Enter.\")\n",
    "    if typing_test_version == 'v2':\n",
    "        print(\"Version 2 selected. Note that you can exit a test in progress \\\n",
    "by hitting the ` (backtick) key.\")\n",
    "              \n",
    "              \n",
    "\n",
    "    verse_order = select_verse()\n",
    "    \n",
    "\n",
    "    while True: # Allows the game to continue until the user exits\n",
    "        results_table, word_stats_table = run_typing_test(\n",
    "            verse_order=verse_order, \n",
    "        results_table=results_table, \n",
    "        word_stats_table=word_stats_table,\n",
    "        test_type = typing_test_version)\n",
    "        # The game will next share an updated progress report:\n",
    "        print(calculate_current_day_results(results_table))\n",
    "        \n",
    "        # The player will now be prompted to select a new verse order \n",
    "        # (or to save and quit).\n",
    "        verse_order = select_subsequent_verse(\n",
    "            previous_verse_order=verse_order)\n",
    "        if verse_order == -1: # In this case, the game will quit and the \n",
    "            # user's new test results will be saved to results_table.\n",
    "            run_analyses = 1\n",
    "            return (results_table, word_stats_table, run_analyses)\n",
    "        if verse_order == -2: # In this case, the game will quit and the \n",
    "            # user's new test results will be saved to results_table.\n",
    "            # However, the analysis portion of the script will be skipped \n",
    "            # in order to save time.\n",
    "            run_analyses = 0\n",
    "            return (results_table, word_stats_table, run_analyses)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_results, df_word_stats, run_analyses = run_game(results_table = df_results, \n",
    "word_stats_table = df_word_stats)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If df_results is blank (e.g. because the player exited out of his/her first typing test during his/her first game), some of the following code will likely crash, because they are expecting results to be present within df_results. Therefore, the program will exit out early instead of continuing on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if len(df_results) == 0:\n",
    "    print(\"No results have been entered, so there is nothing to save or \\\n",
    "analyze. Exiting program in 5 seconds.\")\n",
    "    time.sleep(5) # Allows the user to view the above message\n",
    "    raise SystemExit # See https://stackoverflow.com/a/19747562/13097194"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Updating certain df_Bible columns to reflect new results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_Bible['Characters_Typed'] = df_Bible['Characters'] * df_Bible['Typed']\n",
    "df_Bible['Total_Characters_Typed'] = df_Bible['Characters'] * df_Bible['Tests']\n",
    "df_Bible"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "characters_typed_sum = df_Bible['Characters_Typed'].sum()\n",
    "proportion_of_Bible_typed = characters_typed_sum / df_Bible['Characters'].sum()\n",
    "\n",
    "print(f\"You have typed {characters_typed_sum} characters so far, \\\n",
    "which represents {round(100*proportion_of_Bible_typed, 4)}% of the Bible.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Adding additional values and statistics to df_results:\n",
    "\n",
    "(The following cell was derived from [this script](https://github.com/kburchfiel/typeracer_data_analyzer/blob/master/typeracer_data_analyzer_v2.ipynb) that I wrote.)\n",
    "\n",
    "These statistics will get recreated whenever the script is run; this approach allows for the results to be revised as needed (e.g. if certain rows are removed from the dataset)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_results['Last 10 Avg'] = df_results['WPM'].rolling(10).mean()\n",
    "df_results['Last 100 Avg'] = df_results['WPM'].rolling(100).mean()\n",
    "df_results['Last 1000 Avg'] = df_results['WPM'].rolling(1000).mean()\n",
    "\n",
    "df_results['Incorrect Character % Last 10 Avg'] = df_results['Incorrect Characters as % of Verse Length'].rolling(10).mean()\n",
    "df_results['Incorrect Character % Last 100 Avg'] = df_results['Incorrect Characters as % of Verse Length'].rolling(100).mean()\n",
    "df_results['Incorrect Character % Last 1000 Avg'] = df_results['Incorrect Characters as % of Verse Length'].rolling(1000).mean()\n",
    "\n",
    "\n",
    "df_results['Local_Start_Year'] = df_results['Local_Start_Time'].dt.year\n",
    "df_results['Local_Start_Month'] = df_results['Local_Start_Time'].dt.month\n",
    "df_results['Local_Start_Date'] = df_results['Local_Start_Time'].dt.date\n",
    "df_results['Local_Start_Hour'] = df_results['Local_Start_Time'].dt.hour\n",
    "df_results['Local_Start_Minute'] = df_results['Local_Start_Time'].dt.minute\n",
    "df_results['Local_Start_10_Minute_Block'] = df_results[\n",
    "'Local_Start_Minute'] // 10 + 1\n",
    "df_results['Local_Start_15_Minute_Block'] = df_results[\n",
    "'Local_Start_Minute'] // 15 + 1\n",
    "df_results['Local_Start_30_Minute_Block'] = df_results[\n",
    "'Local_Start_Minute'] // 30 + 1\n",
    "\n",
    "df_results['Count'] = 1 # Useful for pivot tables that analyze test counts\n",
    "# by book, month, etc.\n",
    "\n",
    "# In order to more accurately calculate the largest number of characters \n",
    "# typed within a given block of time, we'll want to know the end time\n",
    "# of each test. This can be calculated as the sum of the local start time\n",
    "# and the number of seconds each test took.\n",
    "df_results['Local_End_Time'] = df_results[\n",
    "'Local_Start_Time'] + pd.to_timedelta(df_results['Seconds'], unit = 's')\n",
    "\n",
    "df_results['Local_End_Year'] = df_results['Local_End_Time'].dt.year\n",
    "df_results['Local_End_Month'] = df_results['Local_End_Time'].dt.month\n",
    "df_results['Local_End_Date'] = df_results['Local_End_Time'].dt.date\n",
    "df_results['Local_End_Hour'] = df_results['Local_End_Time'].dt.hour\n",
    "df_results['Local_End_Minute'] = df_results['Local_End_Time'].dt.minute\n",
    "df_results['Local_End_15_Minute_Block'] = df_results[\n",
    "'Local_End_Minute'] // 15 + 1\n",
    "df_results['Local_End_10_Minute_Block'] = df_results[\n",
    "'Local_End_Minute'] // 10 + 1\n",
    "df_results['Local_End_30_Minute_Block'] = df_results[\n",
    "'Local_End_Minute'] // 30 + 1\n",
    "\n",
    "# The following line uses a list comprehension to generate a cumulative average\n",
    "# of all WPM scores up until the current race. .iloc searches from 0 to i+1 for\n",
    "# each row so that that row is included in the calculation.\n",
    "df_results['cumulative_avg'] = [round(np.mean(df_results.iloc[0:i+1]['WPM']),\n",
    "3) for i in range(len(df_results))]\n",
    "df_results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Adding session data to df_results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# df.at() could be used here, but it could fail if there happen to be any\n",
    "# test number gaps within df_results. Therefore, df.iloc[] will be used\n",
    "# instead. \n",
    "\n",
    "# Determining column indices that can be passed to .iloc: (It's better to\n",
    "# calculate these on demand than to hardcode numbers, as any changes to the \n",
    "# column structure of df_results could then make those hardcoded numbers\n",
    "# incorrect.\n",
    "\n",
    "# Initailizing blank Session and Test_#_Within_Session columns that\n",
    "# can then get recognized and updated in subsequent code:\n",
    "df_results['Session'] = 0 # Using 0 instead of np.NaN initializes these\n",
    "# columns as integers (the desired type for them)\n",
    "df_results['Test_#_Within_Session'] = 0\n",
    "\n",
    "\n",
    "session_col = df_results.columns.get_loc('Session')\n",
    "test_number_within_session_col = df_results.columns.get_loc('Test_#_Within_Session')\n",
    "local_start_time_col = df_results.columns.get_loc('Local_Start_Time')\n",
    "local_end_time_col = df_results.columns.get_loc('Local_End_Time')\n",
    "new_session_cutoff = 300 # The loop below will calculate the time between\n",
    "# each test's end time and the next test's start time. If this duration\n",
    "# (in seconds) is greater to or equal than new_session_cutoff,\n",
    "# a new session will be assigned to the second test.\n",
    "\n",
    "\n",
    "# Initializing the first row as the first test within session 1:\n",
    "df_results.iloc[0, session_col] = 1 \n",
    "df_results.iloc[0, test_number_within_session_col] = 1\n",
    "\n",
    "\n",
    "for i in range(1, len(df_results)): # Starting at 1 because we'll be looking\n",
    "    # back one row during each iteration of the loop.\n",
    "    previous_row_session = df_results.iloc[i-1, session_col]\n",
    "    previous_row_test_within_session = df_results.iloc[i-1, test_number_within_session_col]\n",
    "    if (df_results.iloc[i, local_start_time_col] - \n",
    "        df_results.iloc[i-1, local_end_time_col]).seconds >= new_session_cutoff:\n",
    "        # In this case, a new session will be assigned to this row.\n",
    "        df_results.iloc[i, session_col] = previous_row_session + 1\n",
    "        df_results.iloc[i, test_number_within_session_col] = 1\n",
    "    else: # Otherwise, this row will be interpreted as a continuation\n",
    "        # of the previous session.\n",
    "        df_results.iloc[i, session_col] = previous_row_session\n",
    "        df_results.iloc[i, test_number_within_session_col] = previous_row_test_within_session + 1\n",
    "\n",
    "df_results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Adding additional metrics to df_word_stats:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_word_stats['Count'] = 1\n",
    "df_word_stats['characters'] = df_word_stats['word'].str.len()\n",
    "df_word_stats['CPS'] = df_word_stats['characters'] / (df_word_stats['word_duration (ms)'] / 1000)\n",
    "df_word_stats['WPM'] = df_word_stats['CPS'] * 12\n",
    "df_word_stats"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Saving results:\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def attempt_save(df, filename, index):\n",
    "    '''This function attempts to save the DataFrame passed to df to the file\n",
    "    specified by filename. It allows players to retry the save operation\n",
    "    if it wasn't initially successful (e.g. because the file was open at \n",
    "    the time), thus preventing them from losing their latest progress.\n",
    "    The index parameter determines whether or not the DataFrame's index\n",
    "    will be included in the .csv export. Set to True for results.csv\n",
    "    but False for Web_Catholic_Version_for_game_updated.csv.'''\n",
    "    while True:\n",
    "        try: \n",
    "            df.to_csv(filename, index = index)\n",
    "            return\n",
    "        except:\n",
    "            print(\"File could not be saved, likely because it is currently open. \\\n",
    "Try closing the file and trying again. Press Enter to retry.\")\n",
    "            input()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "attempt_save(df_Bible, 'WEB_Catholic_Version_for_game_updated.csv', index = True)\n",
    "# The verse_order index will be stored within this .csv file so that it can\n",
    "# be accessed during the subsequent run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "attempt_save(df_results, 'results.csv', index = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "attempt_save(df_word_stats, 'word_stats.csv', index = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Successfully saved updated copies of the Results, Word Stats, \\\n",
    "and Bible .csv files.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if run_analyses == 0: # In this case, the analysis portion of the script will\n",
    "    # be skipped. These analyses can be updated when the game is next played.\n",
    "    print(\"Analyses won't be updated during this session. \\\n",
    "Exiting program in 3 seconds.\")\n",
    "    time.sleep(3) # Allows the user to view the above message\n",
    "    raise SystemExit # See https://stackoverflow.com/a/19747562/13097194"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visualizing the player's progress in typing the entire Bible:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "analysis_start_time = time.time() # Allows us to determine how long the\n",
    "# analyses took\n",
    "print(\"Updating analyses:\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_Bible['Count'] = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a tree map within Plotly that visualizes the player's progress in typing the entire Bible:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Creating tree map(s).\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This code is based on https://plotly.com/python/treemaps/\n",
    "# It's pretty amazing that such a complex visualization can be created using\n",
    "# just one line of code. Thanks Plotly!\n",
    "fig_tree_map_books_chapters_verses = px.treemap(\n",
    "    df_Bible, path = ['Book_Name', 'Chapter_Name', 'Verse_#'], \n",
    "    values = 'Characters', color = 'Typed')\n",
    "# fig_verses_typed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_tree_map_books_chapters_verses.write_html(\n",
    "    'Analyses/tree_map_books_chapters_verses.html')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # A similar chart that doesn't use the Typed column for color coding:\n",
    "# (This chart, unlike fig_verses_typed above, won't change unless edits are \n",
    "# made to the code itself, so it can be \n",
    "# commented out after being run once.)\n",
    "# fig_Bible_verses.write_html('Bible_tree_map.html')\n",
    "# fig_Bible_verses = px.treemap(df_Bible, path = ['Book_Name', \n",
    "# 'Chapter_Name', 'Verse_#'], values = 'Characters')\n",
    "# fig_Bible_verses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_Bible"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This variant of the treemap shows chapters and verses rather than books,\n",
    "# chapters, and verses.\n",
    "if (run_on_notebook == True) & (extra_analyses == True):\n",
    "    fig_tree_map_chapters_verses = px.treemap(df_Bible, path = [\n",
    "        'Book_and_Chapter', 'Verse_#'], values = 'Characters', color = 'Typed')\n",
    "    fig_tree_map_chapters_verses.write_html(\n",
    "        'Analyses/tree_map_chapters_verses.html')\n",
    "    fig_tree_map_chapters_verses.write_image(\n",
    "        'Analyses/tree_map_chapters_verses.png', width = 7680, height = 4320)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This variant of the treemap shows each verse as its own box, which results in \n",
    "# a very busy graph that takes a while to load within a web browser\n",
    "# (if it even loads at all).\n",
    "\n",
    "if (run_on_notebook == True) & (extra_analyses == True):\n",
    "    fig_tree_map_verses = px.treemap(df_Bible, path = [df_Bible.index], \n",
    "                                     values = 'Characters', color = 'Typed')\n",
    "    fig_tree_map_verses.write_html('Analyses/tree_map_verses.html')\n",
    "    fig_tree_map_verses.write_image('Analyses/tree_map_verses_8K.png', \n",
    "                                    width = 7680, height = 4320) \n",
    "    fig_tree_map_verses.write_image('Analyses/tree_map_verses_16K.png', \n",
    "                                    width = 15360, height = 8640) \n",
    "# fig_tree_map_verses.write_image('Analyses/tree_map_verses.png', width = 30720, \n",
    "# height = 17280) # Didn't end up rendering successfully, probably \n",
    "# because the dimensions were absurdly large!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a bar chart that shows the proportion of each book that has been typed so far:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Creating progress analyses.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_characters_typed_by_book = df_Bible.pivot_table(index = ['Book_Order', \n",
    "'Book_Name'], values = ['Characters', 'Characters_Typed'], \n",
    "aggfunc = 'sum').reset_index()\n",
    "# Adding 'Book_Order' as the first index value allows for the pivot tables\n",
    "# and bars to be ordered by that value.\n",
    "df_characters_typed_by_book['proportion_typed'] = df_characters_typed_by_book[\n",
    "    'Characters_Typed'] / df_characters_typed_by_book['Characters']\n",
    "df_characters_typed_by_book.to_csv(\n",
    "    'Analyses/characters_typed_by_book.csv')\n",
    "df_characters_typed_by_book"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_proportion_of_each_book_typed = px.bar(df_characters_typed_by_book, \n",
    "x = 'Book_Name', y = 'proportion_typed')\n",
    "fig_proportion_of_each_book_typed.update_yaxes(range = [0, 1]) # Setting\n",
    "# the maximum y value as 1 better demonstrates how much of the Bible\n",
    "# has been typed so far\n",
    "fig_proportion_of_each_book_typed.write_html(\n",
    "    'Analyses/proportion_of_each_book_typed.html')\n",
    "fig_proportion_of_each_book_typed.write_image(\n",
    "    'Analyses/proportion_of_each_book_typed.png', \n",
    "    width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_proportion_of_each_book_typed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a chart that compares the number of characters in each book with the number that have been typed:\n",
    "\n",
    "This provides a clearer view of the player's progress in typing the Bible, as each bar's height is based on the number of characters. (In contrast, bars for fully typed small books will be just as high in fig_proportion_of_each_book_typed as those for fully typed large books.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_characters_typed_in_each_book = px.bar(df_characters_typed_by_book, \n",
    "x = 'Book_Name', y = ['Characters', 'Characters_Typed'], barmode = 'overlay')\n",
    "fig_characters_typed_in_each_book.write_html(\n",
    "    'Analyses/characters_typed_by_book.html')\n",
    "fig_characters_typed_in_each_book.write_image(\n",
    "    'Analyses/characters_typed_by_book.png', \n",
    "    width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_characters_typed_in_each_book"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating charts that show both book- and chapter-level data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_characters_typed_by_book_and_chapter = df_Bible.pivot_table(index = [\n",
    "'Book_Order', 'Book_Name', 'Book_and_Chapter'], values = [\n",
    "    'Characters', 'Characters_Typed'], aggfunc = 'sum').reset_index()\n",
    "df_characters_typed_by_book_and_chapter[\n",
    "'proportion_typed'] = df_characters_typed_by_book_and_chapter[\n",
    "'Characters_Typed'] / df_characters_typed_by_book_and_chapter['Characters']\n",
    "df_characters_typed_by_book_and_chapter.to_csv(\n",
    "    'Analyses/characters_typed_by_book_and_chapter.csv')\n",
    "df_characters_typed_by_book_and_chapter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following chart shows both books (as bars) and chapters (as sections of these bars). These sections are also color coded by the proportion of each chapter that has been typed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_characters_typed_in_each_book_and_chapter = px.bar(\n",
    "df_characters_typed_by_book_and_chapter, x = 'Book_Name', y = [\n",
    "    'Characters'], color = 'proportion_typed')\n",
    "fig_characters_typed_in_each_book_and_chapter.write_html(\n",
    "    'Analyses/characters_typed_by_book_and_chapter.html')\n",
    "fig_characters_typed_in_each_book_and_chapter.write_image(\n",
    "    'Analyses/characters_typed_by_book_and_chapter.png', \n",
    "    width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_characters_typed_in_each_book_and_chapter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating similar charts at the chapter level:\n",
    "\n",
    "These proved difficult to interpret due to the narrowness of the bars, so I'm commenting this code out for now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# fig_proportion_of_each_chapter_typed = px.bar(df_characters_typed_by_chapter, \n",
    "# x = 'Book_and_Chapter', y = 'proportion_typed')\n",
    "# fig_proportion_of_each_chapter_typed.update_yaxes(range = [0, 1]) # Setting\n",
    "# # the maximum y value as 1 better demonstrates how much of the Bible\n",
    "# # has been typed so far\n",
    "# fig_proportion_of_each_chapter_typed.write_html(\n",
    "# 'Analyses/proportion_of_each_chapter_typed.html')\n",
    "# fig_proportion_of_each_chapter_typed\n",
    "\n",
    "# fig_characters_typed_in_each_chapter = px.bar(df_characters_typed_by_chapter, \n",
    "# x = 'Book_and_Chapter', y = ['Characters', 'Characters_Typed'], \n",
    "# barmode = 'overlay')\n",
    "# fig_characters_typed_in_each_chapter.write_html(\n",
    "# 'Analyses/characters_typed_by_chapter.html')\n",
    "# fig_characters_typed_in_each_chapter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Endurance statistics (e.g. most characters/verses typed over a given time period):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Analyzing typing activity by period.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calculating the dates with the most characters and verses typed:\n",
    "\n",
    "Note: In order to create more accurate analyses, I will filter the results to only include values with the same start and end periods. (For instance, if a given test began at 9:59 p.m. on 2023-11-17 but ended after 10 p.m., that test would get filtered out of a 'top hours by characters typed' report, since including it would extend the time frame analyzed beyond a 60-minute window.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_top_dates_by_characters = df_results.pivot_table(\n",
    "    index = ['Local_Start_Date', 'Local_End_Date'], values = 'Characters', aggfunc = 'sum').reset_index(\n",
    "    ).sort_values('Characters', ascending = False)\n",
    "# By using both the start and end dates as pivot index values, we've already \n",
    "# separated results with different start and end dates from ones whose \n",
    "# start and end dates are the same. (This will prevent the tests included\n",
    "# in a given date's calculation from extending beyond just that date.)\n",
    "# We'll also filter the DataFrame to exclude any results whose start\n",
    "# and end dates differ:\n",
    "df_top_dates_by_characters = df_top_dates_by_characters.query(\"Local_Start_Date == Local_End_Date\").head(50).copy()\n",
    "df_top_dates_by_characters['Rank'] = df_top_dates_by_characters[\n",
    "    'Characters'].rank(ascending = False, method = 'min').astype('int')\n",
    "# Creating a column that shows both the rank and date: (This also prevents\n",
    "# Plotly from converting the x axis to a date range, which would interfere\n",
    "# with the order of the chart items)\n",
    "df_top_dates_by_characters['Rank and Date'] = '#'+df_top_dates_by_characters[\n",
    "    'Rank'].astype('str') + ': ' + df_top_dates_by_characters[\n",
    "        'Local_Start_Date'].astype('str')\n",
    "df_top_dates_by_characters.reset_index(drop=True,inplace=True)\n",
    "df_top_dates_by_characters.to_csv(\n",
    "    'Analyses/top_dates_by_characters.csv', index = False)\n",
    "df_top_dates_by_characters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_top_dates_by_characters = px.bar(df_top_dates_by_characters, \n",
    "x = 'Rank and Date', y = 'Characters', text = 'Characters')\n",
    "fig_top_dates_by_characters.update_xaxes(tickangle = 90)\n",
    "fig_top_dates_by_characters.write_html('Analyses/top_dates_by_characters.html')\n",
    "fig_top_dates_by_characters.write_image(\n",
    "    'Analyses/top_dates_by_characters.png', \n",
    "    width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_top_dates_by_characters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_top_dates_by_verses = df_results.pivot_table(\n",
    "    index = ['Local_Start_Date', 'Local_End_Date'], \n",
    "    values = 'Count', aggfunc = 'sum').reset_index(\n",
    "    ).rename(columns = {'Count':'Verses'}).sort_values(\n",
    "        'Verses', ascending = False)\n",
    "df_top_dates_by_verses = df_top_dates_by_verses.query(\n",
    "    \"Local_Start_Date == Local_End_Date\").head(50).copy()\n",
    "df_top_dates_by_verses['Rank'] = df_top_dates_by_verses['Verses'].rank(\n",
    "    ascending = False, method = 'min').astype('int')\n",
    "df_top_dates_by_verses['Rank and Date'] = '#'+df_top_dates_by_verses[\n",
    "    'Rank'].astype('str') + ': ' + df_top_dates_by_verses[\n",
    "        'Local_Start_Date'].astype('str')\n",
    "df_top_dates_by_verses.reset_index(drop=True,inplace=True)\n",
    "df_top_dates_by_verses.to_csv('Analyses/top_dates_by_verses.csv', index = False)\n",
    "df_top_dates_by_verses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_top_dates_by_verses = px.bar(df_top_dates_by_verses, \n",
    "x = 'Rank and Date', y = 'Verses', text = 'Verses')\n",
    "fig_top_dates_by_verses.update_xaxes(tickangle = 90)\n",
    "fig_top_dates_by_verses.write_html('Analyses/top_dates_by_verses.html')\n",
    "fig_top_dates_by_verses.write_image(\n",
    "    'Analyses/top_dates_by_verses.png', \n",
    "    width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_top_dates_by_verses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performing similar analyses by month:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_top_months_by_characters = df_results.pivot_table(\n",
    "    index = ['Local_Start_Year', 'Local_End_Year', \n",
    "    'Local_Start_Month', 'Local_End_Month'], \n",
    "    values = 'Characters', aggfunc = 'sum').reset_index(\n",
    "    ).sort_values('Characters', ascending = False)\n",
    "df_top_months_by_characters = df_top_months_by_characters.query(\n",
    "    \"Local_Start_Year == Local_End_Year & \\\n",
    "Local_Start_Month == Local_End_Month\").head(50).copy()\n",
    "\n",
    "df_top_months_by_characters['Rank'] = df_top_months_by_characters[\n",
    "'Characters'].rank(ascending = False, method = 'min').astype('int')\n",
    "df_top_months_by_characters['Rank and Month'] = '#'+df_top_months_by_characters[\n",
    "    'Rank'].astype('str') + ': ' + df_top_months_by_characters[\n",
    "        'Local_Start_Year'].astype('str') + '-' + df_top_months_by_characters[\n",
    "            'Local_Start_Month'].astype('str')\n",
    "df_top_months_by_characters.reset_index(drop=True,inplace=True)\n",
    "df_top_months_by_characters.to_csv('Analyses/top_months_by_characters.csv', index = False)\n",
    "df_top_months_by_characters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_top_months_by_characters = px.bar(df_top_months_by_characters, \n",
    "x = 'Rank and Month', y = 'Characters', text = 'Characters')\n",
    "fig_top_months_by_characters.update_xaxes(tickangle = 90)\n",
    "fig_top_months_by_characters.write_html(\n",
    "    'Analyses/top_months_by_characters.html')\n",
    "fig_top_months_by_characters.write_image(\n",
    "    'Analyses/top_months_by_characters.png', \n",
    "    width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_top_months_by_characters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_top_months_by_verses = df_results.pivot_table(\n",
    "    index = ['Local_Start_Year', 'Local_Start_Month', \n",
    "    'Local_End_Year', 'Local_End_Month'], \n",
    "    values = 'Count', aggfunc = 'sum').reset_index(\n",
    "    ).rename(columns={'Count':'Verses'}).sort_values(\n",
    "        'Verses', ascending = False)\n",
    "df_top_months_by_verses = df_top_months_by_verses.query(\n",
    "    \"Local_Start_Year == Local_End_Year & \\\n",
    "Local_Start_Month == Local_End_Month\").head(50).copy()\n",
    "\n",
    "df_top_months_by_verses['Rank'] = df_top_months_by_verses['Verses'].rank(\n",
    "    ascending = False, method = 'min').astype('int')\n",
    "df_top_months_by_verses['Rank and Month'] = '#'+df_top_months_by_verses[\n",
    "    'Rank'].astype('str') + ': ' + df_top_months_by_verses[\n",
    "        'Local_Start_Year'].astype('str') + '-' + df_top_months_by_verses[\n",
    "            'Local_Start_Month'].astype('str')\n",
    "df_top_months_by_verses.reset_index(drop=True,inplace=True)\n",
    "df_top_months_by_verses.to_csv('Analyses/top_months_by_verses.csv', index = False)\n",
    "df_top_months_by_verses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_top_months_by_verses = px.bar(df_top_months_by_verses, \n",
    "x = 'Rank and Month', y = 'Verses', text = 'Verses')\n",
    "fig_top_months_by_verses.update_xaxes(tickangle = 90)\n",
    "fig_top_months_by_verses.write_html('Analyses/top_months_by_verses.html')\n",
    "fig_top_months_by_verses.write_image(\n",
    "    'Analyses/top_months_by_verses.png', \n",
    "    width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_top_months_by_verses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performing similar analyses for hours and for 30-, 15-, and 10-minute blocks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_top_hours_by_characters = df_results.pivot_table(index = ['Local_Start_Date', \n",
    "'Local_End_Date', 'Local_Start_Hour', 'Local_End_Hour'], values = 'Characters', \n",
    "aggfunc = 'sum').reset_index().sort_values('Characters', ascending = False)\n",
    "df_top_hours_by_characters = df_top_hours_by_characters.query(\n",
    "\"Local_Start_Date == Local_End_Date & \\\n",
    "Local_Start_Hour == Local_End_Hour\").head(100).copy()\n",
    "df_top_hours_by_characters['Hour'] = df_top_hours_by_characters[\n",
    "'Local_Start_Date'].astype('str') + ' ' + df_top_hours_by_characters[\n",
    "'Local_Start_Hour'].astype('str') \n",
    "df_top_hours_by_characters.to_csv('Analyses/top_hours_by_characters.csv', index = False)\n",
    "df_top_hours_by_characters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_top_hours_by_characters = px.bar(df_top_hours_by_characters, \n",
    "x = 'Hour', y = 'Characters', text = 'Characters')\n",
    "fig_top_hours_by_characters.update_xaxes(type = 'category')\n",
    "fig_top_hours_by_characters.write_html('Analyses/top_hours_by_characters.html')\n",
    "fig_top_hours_by_characters.write_image(\n",
    "    'Analyses/top_hours_by_characters.png', \n",
    "    width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_top_hours_by_characters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_top_30m_by_characters = df_results.pivot_table(index = ['Local_Start_Date', \n",
    "'Local_End_Date', 'Local_Start_Hour', 'Local_End_Hour', \n",
    "'Local_Start_30_Minute_Block', 'Local_End_30_Minute_Block'], \n",
    "values = 'Characters', aggfunc = 'sum').reset_index().sort_values(\n",
    "'Characters', ascending = False)\n",
    "df_top_30m_by_characters = df_top_30m_by_characters.query(\n",
    "\"Local_Start_Date == Local_End_Date & Local_Start_Hour == Local_End_Hour \\\n",
    "& Local_Start_30_Minute_Block == Local_End_30_Minute_Block\").head(100).copy()\n",
    "df_top_30m_by_characters['30-Minute Block'] = df_top_30m_by_characters[\n",
    "'Local_Start_Date'].astype('str') + ' ' + df_top_30m_by_characters[\n",
    "'Local_Start_Hour'].astype('str') + '_' + df_top_30m_by_characters[\n",
    "'Local_Start_30_Minute_Block'].astype('str')\n",
    "df_top_30m_by_characters.to_csv('Analyses/top_30m_by_characters.csv', index = False)\n",
    "df_top_30m_by_characters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_top_30m_by_characters = px.bar(df_top_30m_by_characters, \n",
    "x = '30-Minute Block', y = 'Characters', text = 'Characters')\n",
    "fig_top_30m_by_characters.update_xaxes(type = 'category')\n",
    "fig_top_30m_by_characters.write_html(\n",
    "'Analyses/top_30m_blocks_by_characters.html')\n",
    "fig_top_30m_by_characters.write_image(\n",
    "    'Analyses/top_30m_blocks_by_characters.png', \n",
    "    width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_top_30m_by_characters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_top_15m_by_characters = df_results.pivot_table(index = [\n",
    "'Local_Start_Date', 'Local_End_Date', 'Local_Start_Hour', 'Local_End_Hour', \n",
    "'Local_Start_15_Minute_Block', 'Local_End_15_Minute_Block'], \n",
    "values = 'Characters', aggfunc = 'sum').reset_index().sort_values(\n",
    "'Characters', ascending = False)\n",
    "# print(len(df_top_15m_by_characters))\n",
    "df_top_15m_by_characters = df_top_15m_by_characters.query(\n",
    "\"Local_Start_Date == Local_End_Date & Local_Start_Hour == Local_End_Hour & \\\n",
    "Local_Start_15_Minute_Block == Local_End_15_Minute_Block\").head(100).copy()\n",
    "df_top_15m_by_characters['15-Minute Block'] = df_top_15m_by_characters[\n",
    "'Local_Start_Date'].astype('str') + ' ' + df_top_15m_by_characters[\n",
    "'Local_Start_Hour'].astype('str') + '_' + df_top_15m_by_characters[\n",
    "'Local_Start_15_Minute_Block'].astype('str')\n",
    "# print(len(df_top_15m_by_characters))\n",
    "df_top_15m_by_characters.to_csv('Analyses/top_15m_by_characters.csv', index = False)\n",
    "df_top_15m_by_characters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_top_15m_by_characters = px.bar(df_top_15m_by_characters, \n",
    "x = '15-Minute Block', y = 'Characters', text = 'Characters')\n",
    "fig_top_15m_by_characters.update_xaxes(type = 'category')\n",
    "fig_top_15m_by_characters.write_html(\n",
    "'Analyses/top_15m_blocks_by_characters.html')\n",
    "fig_top_15m_by_characters.write_image(\n",
    "    'Analyses/top_15m_blocks_by_characters.png', \n",
    "    width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_top_15m_by_characters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_top_10m_by_characters = df_results.pivot_table(index = ['Local_Start_Date', \n",
    "'Local_End_Date', 'Local_Start_Hour', 'Local_End_Hour', \n",
    "'Local_Start_10_Minute_Block', 'Local_End_10_Minute_Block'], \n",
    "values = 'Characters', aggfunc = 'sum').reset_index().sort_values(\n",
    "'Characters', ascending = False)\n",
    "# print(len(df_top_10m_by_characters))\n",
    "df_top_10m_by_characters = df_top_10m_by_characters.query(\n",
    "\"Local_Start_Date == Local_End_Date & Local_Start_Hour == Local_End_Hour \\\n",
    "& Local_Start_10_Minute_Block == Local_End_10_Minute_Block\").head(100).copy()\n",
    "df_top_10m_by_characters['10-Minute Block'] = df_top_10m_by_characters[\n",
    "'Local_Start_Date'].astype('str') + ' ' + df_top_10m_by_characters[\n",
    "'Local_Start_Hour'].astype('str') + '_' + df_top_10m_by_characters[\n",
    "'Local_Start_10_Minute_Block'].astype('str')\n",
    "# print(len(df_top_10m_by_characters))\n",
    "df_top_10m_by_characters.to_csv('Analyses/top_10m_by_characters.csv', index = False)\n",
    "df_top_10m_by_characters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_top_10m_by_characters = px.bar(df_top_10m_by_characters, \n",
    "x = '10-Minute Block', y = 'Characters', text = 'Characters')\n",
    "fig_top_10m_by_characters.update_xaxes(type = 'category')\n",
    "fig_top_10m_by_characters.write_html(\n",
    "'Analyses/top_10m_blocks_by_characters.html')\n",
    "fig_top_10m_by_characters.write_image(\n",
    "    'Analyses/top_10m_blocks_by_characters.png', \n",
    "    width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_top_10m_by_characters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analyzing WPM and accuracy data:\n",
    "\n",
    "(Some of this section's code derives from my work in [this script](https://github.com/kburchfiel/typeracer_data_analyzer/blob/master/typeracer_data_analyzer_v2.ipynb).)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Creating WPM and accuracy analyses.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Verses with the highest WPMs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_top_100_wpm = df_results.sort_values('WPM', ascending = False).head(\n",
    "    100).copy()\n",
    "df_top_100_wpm.insert(0, 'Rank', df_top_100_wpm['WPM'].rank(\n",
    "    ascending = False, method = 'min').astype('int'))\n",
    "# method = 'min' assigns the lowest rank to any rows that happen to have\n",
    "# the same WPM. See \n",
    "# https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.rank.html\n",
    "df_top_100_wpm.to_csv('Analyses/top_100_wpm.csv')\n",
    "df_top_100_wpm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_top_100_wpm = px.bar(df_top_100_wpm, x = 'Rank', y = 'WPM', \n",
    "text_auto = '.6s')\n",
    "fig_top_100_wpm.write_html('Analyses/top_100_wpm.html')\n",
    "fig_top_100_wpm.write_image('Analyses/top_100_wpm.png', \n",
    "width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_top_100_wpm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Top 'Last 10 Average' values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if len(df_results) >= 10: # If fewer than 10 tests are present in df_results,\n",
    "    # there won't be anything to graph (and the following code will raise\n",
    "    # an error), so this section should be skipped.\n",
    "    df_top_last_10_avg_results = df_results.sort_values(\n",
    "    'Last 10 Avg', ascending = False).head(20).copy()\n",
    "    df_top_last_10_avg_results.insert(0, 'Rank', \n",
    "    df_top_last_10_avg_results['Last 10 Avg'].rank(ascending = False, \n",
    "    method = 'min').astype('int'))\n",
    "    df_top_last_10_avg_results.to_csv('Analyses/top_last_10_avg_results.csv', index = False)\n",
    "    df_top_last_10_avg_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if len(df_results) >= 10:\n",
    "    fig_top_last_10_average_wpm = px.bar(df_top_last_10_avg_results, x = 'Rank', \n",
    "    y = 'Last 10 Avg', text_auto = '.6s')\n",
    "    fig_top_last_10_average_wpm.write_html('Analyses/top_last_10_average_wpm.html')\n",
    "    fig_top_last_10_average_wpm.write_image('Analyses/top_last_10_average_wpm.png', \n",
    "    width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "    fig_top_last_10_average_wpm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Showing WPM results and moving averages by test number:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_df_results_by_test_number = px.line(df_results, x = df_results.index, \n",
    "y = ['WPM', 'Last 10 Avg', 'Last 100 Avg', 'Last 1000 Avg', 'cumulative_avg'])\n",
    "fig_df_results_by_test_number.write_html('Analyses/results_by_test_number.html')\n",
    "fig_df_results_by_test_number.write_image('Analyses/results_by_test_number.png', \n",
    "width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_df_results_by_test_number"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating WPM histograms for (1) all tests and (2) the last 1000 tests:\n",
    "\n",
    "(Until you've taken more than 1,000 tests, these histograms will have the same appearance.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_wpm_histogram = px.histogram(x = df_results['WPM'], nbins = 50, \n",
    "text_auto = True)\n",
    "fig_wpm_histogram.update_layout(bargroupgap = 0.1) # Adds a bit of space\n",
    "# in between histogram bars. See https://stackoverflow.com/a/62925197/13097194\n",
    "fig_wpm_histogram.write_html('Analyses/wpm_histogram.html')\n",
    "fig_wpm_histogram.write_image('Analyses/wpm_histogram.png', \n",
    "width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_wpm_histogram\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_wpm_histogram = px.histogram(x = df_results.tail(1000)['WPM'], nbins = 50, \n",
    "text_auto = True)\n",
    "fig_wpm_histogram.update_layout(bargroupgap = 0.1) # Adds a bit of space\n",
    "# in between histogram bars. See https://stackoverflow.com/a/62925197/13097194\n",
    "fig_wpm_histogram.write_html('Analyses/wpm_histogram_last_1000.html')\n",
    "fig_wpm_histogram.write_image('Analyses/wpm_histogram_last_1000.png', \n",
    "width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_wpm_histogram\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluating average results by month:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_results_by_month = df_results.pivot_table(\n",
    "    index = ['Local_Start_Year', 'Local_Start_Month'], values = [\n",
    "'Count', 'WPM'], aggfunc = {'Count':'sum', 'WPM':'mean'}).reset_index()\n",
    "df_results_by_month['Year/Month'] = df_results_by_month[\n",
    "    'Local_Start_Year'].astype('str') + '-' + df_results_by_month[\n",
    "    'Local_Start_Month'].astype('str')\n",
    "df_results_by_month.to_csv('Analyses/results_by_month.csv', index = False)\n",
    "df_results_by_month"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_results_by_month = px.bar(df_results_by_month, x = 'Year/Month', \n",
    "y = 'WPM', color = 'Count', text_auto = '.6s')\n",
    "fig_results_by_month.update_xaxes(type = 'category') # This line, based on\n",
    "# Pracheta's response at https://stackoverflow.com/a/64424308/13097194,\n",
    "# updates the axes to show the date-month pairs as strings rather than \n",
    "# as Plotly-formatted date values. This will also prevent missing\n",
    "# months from appearing in the graph.\n",
    "fig_results_by_month.write_html('Analyses/results_by_month.html')\n",
    "fig_results_by_month.write_image('Analyses/results_by_month.png', \n",
    "width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_results_by_month"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluating average results by hour of day:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_results_by_hour = df_results.pivot_table(index = ['Local_Start_Hour'], \n",
    "values = ['Count', 'WPM'], \n",
    "aggfunc = {'Count':'sum', 'WPM':'mean'}).reset_index()\n",
    "df_results_by_hour.to_csv('Analyses/results_by_hour.csv')\n",
    "df_results_by_hour"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_results_by_hour = px.bar(df_results_by_hour, x = 'Local_Start_Hour', \n",
    "y = 'WPM', color = 'Count', text_auto = '.6s')\n",
    "fig_results_by_hour.write_html('Analyses/results_by_hour.html')\n",
    "fig_results_by_hour.write_image('Analyses/results_by_hour.png', \n",
    "width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_results_by_hour"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Comparing mean WPMs by Bible books:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_wpm_by_book = df_results.pivot_table(index = 'Book', values = 'WPM', \n",
    "aggfunc = ['count', 'mean'], margins = True, \n",
    "margins_name = 'Total').reset_index()\n",
    "df_wpm_by_book.columns = 'Book', 'Tests', 'WPM'\n",
    "df_wpm_by_book.sort_values('WPM', ascending = False, inplace = True)\n",
    "df_wpm_by_book.reset_index(drop=True,inplace=True)\n",
    "df_wpm_by_book.to_csv('Analyses/mean_wpm_by_book.csv')\n",
    "df_wpm_by_book\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The following chart will display a bar for each book for which at least one \n",
    "# test has been taken. It will also show a line that corresponds to the player's\n",
    "# overall WPM across all books. The bars are colored by test count, making\n",
    "# it easier to identify which bars might be skewed by a low number of results.\n",
    "# The 'Total' value in df_wpm_by_book is displayed as a line instead of as\n",
    "# a color so as not to interfere with the color gradient.\n",
    "\n",
    "# Retrieving the total mean WPM value in df_wpm_by_book:\n",
    "total_mean_wpm = df_wpm_by_book.query(\"Book == 'Total'\").iloc[0]['WPM']\n",
    "total_mean_wpm\n",
    "\n",
    "fig_mean_wpm_by_book = px.bar(df_wpm_by_book.query(\"Book != 'Total'\"), \n",
    "x = 'Book', y = 'WPM', color = 'Tests', text_auto = '.6s')\n",
    "fig_mean_wpm_by_book.add_shape(type = 'line', x0 = -0.5, \n",
    "x1 = len(df_wpm_by_book) -1.5, y0 = total_mean_wpm, y1 = total_mean_wpm)\n",
    "# See https://plotly.com/python/shapes/ for the add_shape() code.\n",
    "# The use of -0.5 and len() - 1.5 is based on gleasocd's answer at \n",
    "# https://stackoverflow.com/a/40408960/13097194 . len(df) - 0.5 would normally\n",
    "# work, except that I reduced the size of the DataFrame by 1 when excluding\n",
    "# the 'Total' book.\n",
    "fig_mean_wpm_by_book.write_html('Analyses/mean_wpm_by_book.html')\n",
    "fig_mean_wpm_by_book.write_image('Analyses/mean_wpm_by_book.png', width = 1920, \n",
    "height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_mean_wpm_by_book\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Comparing mean WPM for error-free and non-error free tests:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We'll only create the graph if there is at least one mistake-free\n",
    "# result and one result with errors. (Otherwise, we won't be able to\n",
    "# compare these two outcomes.)\n",
    "if (len(df_results.query(\"Mistake_Free_Test == 0\")) >= 1) & (\n",
    "    len(df_results.query(\"Mistake_Free_Test == 1\")) >= 1):\n",
    "    df_wpm_by_mistake_free_status = df_results.pivot_table(index = 'Mistake_Free_Test', values = 'WPM', aggfunc = 'mean').reset_index()\n",
    "    df_wpm_by_mistake_free_status['Mistake_Free_Test'].replace({0:'No', 1:'Yes'}, inplace = True)\n",
    "    df_wpm_by_mistake_free_status.to_csv('Analyses/wpm_by_mistake_free_status.csv', index = False)\n",
    "    df_wpm_by_mistake_free_status"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We'll only create the graph if there is at least one mistake-free\n",
    "# result and one result with errors. (Otherwise, we won't be able\n",
    "# to compare these two outcomes.)\n",
    "if (len(df_results.query(\"Mistake_Free_Test == 0\")) >= 1) & (\n",
    "    len(df_results.query(\"Mistake_Free_Test == 1\")) >= 1):\n",
    "    fig_wpm_by_mistake_free_status = px.bar(df_wpm_by_mistake_free_status, x = 'Mistake_Free_Test', y = 'WPM', text_auto = '.6s')\n",
    "    fig_wpm_by_mistake_free_status.write_html('Analyses/mean_wpm_by_mistake_free_status.html')\n",
    "    fig_wpm_by_mistake_free_status.write_image('Analyses/mean_wpm_by_mistake_free_status.png', width = 1920, \n",
    "    height = 1080, engine = 'kaleido', scale = 2)\n",
    "    fig_wpm_by_mistake_free_status"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_results.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluating the relationship between backspaces as a % of verse length and WPM:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_incorrect_characters_wpm_scatter = px.scatter(df_results, \n",
    "x = 'Incorrect Characters as % of Verse Length', y = 'WPM', trendline = 'ols')\n",
    "# Note that you can hover over the best fit line to see the regression results.\n",
    "fig_incorrect_characters_wpm_scatter.write_html('Analyses/incorrect_characters_wpm_scatter.html')\n",
    "fig_incorrect_characters_wpm_scatter.write_image('Analyses/incorrect_characters_wpm_scatter.png', \n",
    "width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_incorrect_characters_wpm_scatter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_accuracy_wpm_histogram = px.histogram(df_results, \n",
    "x = 'Incorrect Characters as % of Verse Length', y = 'WPM', \n",
    "histfunc = 'avg', nbins = 20, text_auto = '.6s')\n",
    "fig_accuracy_wpm_histogram.update_layout(bargroupgap = 0.1)\n",
    "fig_accuracy_wpm_histogram.write_html('Analyses/incorrect_characters_wpm_histogram.html')\n",
    "fig_accuracy_wpm_histogram.write_image('Analyses/incorrect_characters_wpm_histogram.png', \n",
    "width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_accuracy_wpm_histogram\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_accuracy_count_histogram = px.histogram(df_results, \n",
    "x = 'Incorrect Characters as % of Verse Length', nbins = 20, text_auto = True)\n",
    "fig_accuracy_count_histogram.update_layout(bargroupgap = 0.1)\n",
    "fig_accuracy_count_histogram.write_html('Analyses/incorrect_character_pct_histogram.html')\n",
    "fig_accuracy_count_histogram.write_image('Analyses/incorrect_character_pct_histogram.png', \n",
    "width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_accuracy_count_histogram"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizing trends in accuracy over time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_incorrect_characters_by_test_number = px.line(df_results, x = df_results.index, \n",
    "y = ['Incorrect Characters as % of Verse Length',\n",
    "     'Incorrect Character % Last 10 Avg',\n",
    "     'Incorrect Character % Last 100 Avg',\n",
    "     'Incorrect Character % Last 1000 Avg'])\n",
    "fig_incorrect_characters_by_test_number.write_html('Analyses/incorrect_character_pct_over_time.html')\n",
    "fig_incorrect_characters_by_test_number.write_image('Analyses/incorrect_character_pct_over_time.png', \n",
    "width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_incorrect_characters_by_test_number"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Average WPM by day:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_avg_wpm_by_day = df_results.pivot_table(index = 'Local_Start_Date', values = ['WPM', 'Characters'], aggfunc = {'WPM':'mean', 'Characters':'sum'}).reset_index()\n",
    "# Limiting the results to dates with at least 5000 characters typed; this \n",
    "# step will help prevent the chart from getting skewed by dates with\n",
    "# very low character counts (which may have unusually high/low average WPMs).\n",
    "df_avg_wpm_by_day.query(\"Characters >= 5000\", inplace = True)\n",
    "df_avg_wpm_by_day.to_csv('Analyses/average_wpm_by_day.csv', index = False)\n",
    "df_avg_wpm_by_day.reset_index(drop=True,inplace=True)\n",
    "df_avg_wpm_by_day"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_avg_wpm_by_day = px.line(df_avg_wpm_by_day, x = 'Local_Start_Date', y = 'WPM')\n",
    "fig_avg_wpm_by_day.update_xaxes(type='category')\n",
    "fig_avg_wpm_by_day.write_html('Analyses/avg_wpm_by_day.html')\n",
    "fig_avg_wpm_by_day.write_image('Analyses/avg_wpm_by_day.png', \n",
    "width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_avg_wpm_by_day"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_results.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a chart of the days with the highest average WPMs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_top_daily_wpm_averages = px.bar(df_avg_wpm_by_day.sort_values('WPM', ascending = False).head(50).copy(), x = 'Local_Start_Date', y = 'WPM', text_auto = '.6s', color = 'Characters')\n",
    "fig_top_daily_wpm_averages.update_xaxes(type='category')\n",
    "fig_top_daily_wpm_averages.write_html('Analyses/top_daily_wpm_averages.html')\n",
    "fig_top_daily_wpm_averages.write_image('Analyses/top_daily_wpm_averages.png', \n",
    "width = 1920, height = 1080, engine = 'kaleido', scale = 2)\n",
    "\n",
    "fig_top_daily_wpm_averages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculating WPM percentile data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_wpm_by_percentile = df_results['WPM'].quantile(q = np.arange(0, 1.05, 0.05)).transpose().reset_index()\n",
    "df_wpm_by_percentile.columns = ['Percentile', 'WPM']\n",
    "df_wpm_by_percentile['Percentile'] = (\n",
    "df_wpm_by_percentile['Percentile']*100).astype('int')\n",
    "# Some of the WPM values had additional decimal values (e.g. 15.000000000000002,\n",
    "# might currently have additional decimal values, so rounding them as integers\n",
    "# helps simplify them.\n",
    "df_wpm_by_percentile.to_csv('Analyses/wpm_by_percentile.csv', index = False)\n",
    "df_wpm_by_percentile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig_wpm_by_percentile = px.bar(df_wpm_by_percentile, x = 'Percentile', y = 'WPM', text_auto = '.6s')\n",
    "fig_wpm_by_percentile.update_xaxes(type = 'category')\n",
    "fig_wpm_by_percentile.write_html('Analyses/wpm_by_percentile.html')\n",
    "fig_wpm_by_percentile.write_image('Analyses/wpm_by_percentile.png', width = 1920, \n",
    "height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_wpm_by_percentile"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculating WPM data for within-session test numbers:\n",
    "\n",
    "These data can help players determine how long it takes for their fingers to warm up when beginning a session (e.g. a set of consecutive tests) and, conversely, when fatigue begins to set in."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculating average WPMs for within-session test numbers:\n",
    "\n",
    "df_results_by_within_session_test_number = df_results.pivot_table(index = 'Test_#_Within_Session', values = ['WPM', 'Count'], aggfunc = {'WPM':'mean', 'Count':'sum'}).reset_index()\n",
    "df_results_by_within_session_test_number.query(\"Count >= 10\", inplace = True)\n",
    "# The above line prevents rows with low counts from skewing the results.\n",
    "df_results_by_within_session_test_number.to_csv('Analyses/results_by_within_session_test_number.csv', index = False)\n",
    "df_results_by_within_session_test_number"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if len(df_results_by_within_session_test_number) > 0: # The player might not\n",
    "    # yet have completed more than 10 sessions, in which case the following\n",
    "    # code should be skipped (as there wouldn't be anything to graph).\n",
    "    fig_wpm_by_within_session_test_number = px.line(df_results_by_within_session_test_number, x = 'Test_#_Within_Session', y = 'WPM')\n",
    "    fig_wpm_by_within_session_test_number.write_html('Analyses/wpm_by_within_session_test_number.html')\n",
    "    fig_wpm_by_within_session_test_number.write_image('Analyses/wpm_by_within_session_test_number.png', width = 1920, \n",
    "    height = 1080, engine = 'kaleido', scale = 2)\n",
    "    fig_wpm_by_within_session_test_number"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "latest_session = df_results['Session'].max()\n",
    "\n",
    "# The following chart plots the player's most recent sessions \n",
    "# on the same graph. This graph will\n",
    "# be quite 'busy,' but players can double-click a session number in order \n",
    "# to show results for only that session (which may be more useful).\n",
    "fig_session_results = px.line(df_results.query(\"Session >= @latest_session - 9\"), x = 'Test_#_Within_Session', y = 'WPM', color = 'Session')\n",
    "fig_session_results.write_html('Analyses/latest_session_results.html')\n",
    "fig_session_results.write_image('Analyses/latest_session_results.png', width = 1920, \n",
    "height = 1080, engine = 'kaleido', scale = 2)\n",
    "fig_session_results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating word-level analyses:\n",
    "\n",
    "Note: This section used to include code that would calculate the fastest single-word entry times. However, I found that these results could be distorted by computer lag. Specifically, if the player enters keys while the computer is frozen, some of those keypress durations will be reported as shorter than they actually were. (This makes sense, in that while the computer is frozen, the actual keypress time might not get logged; instead, a later post-freeze time will get logged. This post-freeze time will then be closer to the next keypress time, causing a shorter-than-actual keypress time to get reported.) (I simulated computer lag in my tests by entering a time.sleep() command, but I believe actual lag would produce similar errors). These skewed keypress durations can in turn skew the reported time needed to enter each word.\n",
    "\n",
    "Therefore, while the code still shows median word entry times (which are less susceptible to this issue), it will no longer calculate the fastest times that a particular word was entered. These can still be located within the word stats file, but the results should be interpreted with great caution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We'll skip the word-level analyses section if we have no data to analyze.\n",
    "\n",
    "if len(df_word_stats) == 0:\n",
    "    run_word_analyses = 0\n",
    "else:\n",
    "    run_word_analyses = 1\n",
    "    print(\"Creating word-level analyses.\")\n",
    "\n",
    "run_word_analyses"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if run_word_analyses == 1:\n",
    "    df_word_stats_pivot = df_word_stats.pivot_table(index = 'word', values = {'Count', 'typed_word_without_mistakes', 'WPM'}, aggfunc = {'Count':'sum', 'typed_word_without_mistakes':'mean', 'WPM':'median'}).reset_index().sort_values('WPM', ascending = False).reset_index(drop=True)\n",
    "    df_word_stats_pivot.rename(columns = {'WPM':'Median WPM'}, inplace = True)\n",
    "    df_common_word_stats_pivot = df_word_stats_pivot.query(\"Count >= 5\").copy()\n",
    "    df_word_stats_pivot.to_csv('Analyses/word_stats_pivot.csv', index = False)\n",
    "    df_common_word_stats_pivot.to_csv('Analyses/word_stats_pivot_common.csv', \n",
    "    index = False)\n",
    "    df_word_stats_pivot\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_common_word_stats_pivot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if run_word_analyses == 1:\n",
    "    fig_words_with_highest_median_wpms = px.bar(df_word_stats_pivot.head(100), \n",
    "    x = 'word', y = 'Median WPM', color = 'Count', text_auto = '.6s')\n",
    "    fig_words_with_highest_median_wpms.write_html('Analyses/words_with_highest_median_wpms.html')\n",
    "    fig_words_with_highest_median_wpms.write_image('Analyses/words_with_highest_median_wpms.png', width = 1920, \n",
    "    height = 1080, engine = 'kaleido', scale = 2)\n",
    "    fig_words_with_highest_median_wpms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if (run_word_analyses == 1) & (len(df_common_word_stats_pivot) >= 1):\n",
    "    fig_words_with_highest_median_wpms_common = px.bar(df_common_word_stats_pivot.head(100), \n",
    "    x = 'word', y = 'Median WPM', color = 'Count', text_auto = '.6s')\n",
    "    fig_words_with_highest_median_wpms_common.write_html('Analyses/words_with_highest_median_wpms_common.html')\n",
    "    fig_words_with_highest_median_wpms_common.write_image('Analyses/words_with_highest_median_wpms_common.png', width = 1920, \n",
    "    height = 1080, engine = 'kaleido', scale = 2)\n",
    "    fig_words_with_highest_median_wpms_common"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if run_word_analyses == 1:\n",
    "    fig_words_with_lowest_median_wpms = px.bar(\n",
    "    df_word_stats_pivot.sort_values(\n",
    "    'Median WPM').head(100), x = 'word', y = 'Median WPM', color = 'Count', text_auto = '.6s')\n",
    "    fig_words_with_lowest_median_wpms.write_html(\n",
    "    'Analyses/words_with_lowest_median_wpms.html')\n",
    "    fig_words_with_lowest_median_wpms.write_image(\n",
    "    'Analyses/words_with_lowest_median_wpms.png', width = 1920, \n",
    "    height = 1080, engine = 'kaleido', scale = 2)\n",
    "    fig_words_with_lowest_median_wpms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if (run_word_analyses == 1) & (len(df_common_word_stats_pivot) >= 1):\n",
    "    fig_words_with_lowest_median_wpms_common = px.bar(\n",
    "    df_common_word_stats_pivot.sort_values('Median WPM').head(100), \n",
    "    x = 'word', y = 'Median WPM', color = 'Count', text_auto = '.6s')\n",
    "    fig_words_with_lowest_median_wpms_common.write_html('Analyses/words_with_lowest_median_wpms_common.html')\n",
    "    fig_words_with_lowest_median_wpms_common.write_image('Analyses/words_with_lowest_median_wpms_common.png', width = 1920, \n",
    "    height = 1080, engine = 'kaleido', scale = 2)\n",
    "    fig_words_with_lowest_median_wpms_common"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if (run_word_analyses == 1) & (len(df_common_word_stats_pivot) >= 1):\n",
    "    fig_words_with_highest_accuracy_rates_common = px.bar(\n",
    "    df_common_word_stats_pivot.sort_values(\n",
    "        'typed_word_without_mistakes', ascending = False).head(100),\n",
    "    x = 'word', y = 'typed_word_without_mistakes', color = 'Count', text_auto = '.6s')\n",
    "    fig_words_with_highest_accuracy_rates_common.write_html('Analyses/words_with_highest_accuracy_rates_common.html')\n",
    "    fig_words_with_highest_accuracy_rates_common.write_image('Analyses/words_with_highest_accuracy_rates_common.png', width = 1920, \n",
    "    height = 1080, engine = 'kaleido', scale = 2)\n",
    "    fig_words_with_highest_accuracy_rates_common"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if (run_word_analyses == 1) & (len(df_common_word_stats_pivot) >= 1):\n",
    "    fig_words_with_lowest_accuracy_rates_common = px.bar(\n",
    "    df_common_word_stats_pivot.sort_values(\n",
    "        'typed_word_without_mistakes').head(100),\n",
    "    x = 'word', y = 'typed_word_without_mistakes', color = 'Count', text_auto = '.3s')\n",
    "    fig_words_with_lowest_accuracy_rates_common.write_html('Analyses/words_with_lowest_accuracy_rates_common.html')\n",
    "    fig_words_with_lowest_accuracy_rates_common.write_image('Analyses/words_with_lowest_accuracy_rates_common.png', width = 1920, \n",
    "    height = 1080, engine = 'kaleido', scale = 2)\n",
    "    fig_words_with_lowest_accuracy_rates_common"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This script used to show the highest-ever WPMs for individual words, but this code has been removed due to accuracy issues."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "analysis_end_time = time.time()\n",
    "analysis_time = analysis_end_time - analysis_start_time\n",
    "print(f\"Finished updating analyses in {round(analysis_time, 3)} seconds. \\\n",
    "Enter any key to exit.\") # Allows the console to stay open when the\n",
    "# .py version of the program is run\n",
    "\n",
    "input()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "ga15pyd",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}