Analyzing Stock Prices.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Analyzing stock prices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this project, I'll analyse stock market data downloaded from [Yahoo Finance](https://finance.yahoo.com/) using the [yahoo_finance](https://pypi.python.org/pypi/yahoo-finance) Python package. \n",
    "\n",
    "This data consists of the daily stock prices from 2007-1-1 to 2017-04-17 for several hundred stock symbols traded on the NASDAQ stock exchange, stored in the prices folder.\n",
    "\n",
    "Each file in the prices folder has a specific stock symbol for its name, and each contains the following information:\n",
    "\n",
    "- date — the data's date\n",
    "- close — the date's closing price\n",
    "- open — the date's opening price\n",
    "- high — the date's highest stock price during trading\n",
    "- low — the date's lowest stock price during trading\n",
    "- volume — the date's number of shares traded"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading in the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As the data is stored in one csv file per stock, we will need to iterate over all files. We'll be storing them in a dictionary with the stock symbols as the keys and the dataframe of each csv file as the values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "stocks = {}\n",
    "for fn in os.listdir(\"prices\"):\n",
    "    df = pd.read_csv(os.path.join(\"prices\", fn))\n",
    "    stocks[fn[:len(fn) - 4]] = df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>close</th>\n",
       "      <th>open</th>\n",
       "      <th>high</th>\n",
       "      <th>low</th>\n",
       "      <th>volume</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2007-01-03</td>\n",
       "      <td>83.800002</td>\n",
       "      <td>86.289999</td>\n",
       "      <td>86.579999</td>\n",
       "      <td>81.899999</td>\n",
       "      <td>309579900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2007-01-04</td>\n",
       "      <td>85.659998</td>\n",
       "      <td>84.050001</td>\n",
       "      <td>85.949998</td>\n",
       "      <td>83.820003</td>\n",
       "      <td>211815100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2007-01-05</td>\n",
       "      <td>85.049997</td>\n",
       "      <td>85.770000</td>\n",
       "      <td>86.199997</td>\n",
       "      <td>84.400002</td>\n",
       "      <td>208685400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2007-01-08</td>\n",
       "      <td>85.470000</td>\n",
       "      <td>85.959998</td>\n",
       "      <td>86.529998</td>\n",
       "      <td>85.280003</td>\n",
       "      <td>199276700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2007-01-09</td>\n",
       "      <td>92.570003</td>\n",
       "      <td>86.450003</td>\n",
       "      <td>92.979999</td>\n",
       "      <td>85.150000</td>\n",
       "      <td>837324600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2585</th>\n",
       "      <td>2017-04-10</td>\n",
       "      <td>143.169998</td>\n",
       "      <td>143.600006</td>\n",
       "      <td>143.880005</td>\n",
       "      <td>142.899994</td>\n",
       "      <td>18473000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2586</th>\n",
       "      <td>2017-04-11</td>\n",
       "      <td>141.630005</td>\n",
       "      <td>142.940002</td>\n",
       "      <td>143.350006</td>\n",
       "      <td>140.059998</td>\n",
       "      <td>30275300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2587</th>\n",
       "      <td>2017-04-12</td>\n",
       "      <td>141.800003</td>\n",
       "      <td>141.600006</td>\n",
       "      <td>142.149994</td>\n",
       "      <td>141.009995</td>\n",
       "      <td>20238900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2588</th>\n",
       "      <td>2017-04-13</td>\n",
       "      <td>141.050003</td>\n",
       "      <td>141.910004</td>\n",
       "      <td>142.380005</td>\n",
       "      <td>141.050003</td>\n",
       "      <td>17652900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2589</th>\n",
       "      <td>2017-04-17</td>\n",
       "      <td>141.830002</td>\n",
       "      <td>141.479996</td>\n",
       "      <td>141.880005</td>\n",
       "      <td>140.869995</td>\n",
       "      <td>16424000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2590 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            date       close        open        high         low     volume\n",
       "0     2007-01-03   83.800002   86.289999   86.579999   81.899999  309579900\n",
       "1     2007-01-04   85.659998   84.050001   85.949998   83.820003  211815100\n",
       "2     2007-01-05   85.049997   85.770000   86.199997   84.400002  208685400\n",
       "3     2007-01-08   85.470000   85.959998   86.529998   85.280003  199276700\n",
       "4     2007-01-09   92.570003   86.450003   92.979999   85.150000  837324600\n",
       "...          ...         ...         ...         ...         ...        ...\n",
       "2585  2017-04-10  143.169998  143.600006  143.880005  142.899994   18473000\n",
       "2586  2017-04-11  141.630005  142.940002  143.350006  140.059998   30275300\n",
       "2587  2017-04-12  141.800003  141.600006  142.149994  141.009995   20238900\n",
       "2588  2017-04-13  141.050003  141.910004  142.380005  141.050003   17652900\n",
       "2589  2017-04-17  141.830002  141.479996  141.880005  140.869995   16424000\n",
       "\n",
       "[2590 rows x 6 columns]"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "stocks['aapl']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Computing aggregates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "ave_close = {}\n",
    "for s in stocks:\n",
    "    ave_close[s] = stocks[s]['close'].mean()    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "blfs 0.8122763011583011\n"
     ]
    }
   ],
   "source": [
    "# stock with minimum average closing price\n",
    "min_ave_close = min(ave_close, key=ave_close.get)\n",
    "print(min_ave_close, ave_close[min_ave_close])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "amzn 275.13407757104255\n"
     ]
    }
   ],
   "source": [
    "# stock with maximum average closing price\n",
    "max_ave_close = max(ave_close, key=ave_close.get)\n",
    "print(max_ave_close, ave_close[max_ave_close])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Organising trades by date"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "560"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(stocks)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trade_dates = {}\n",
    "for s in stocks:\n",
    "    for index, row in stocks[s].iterrows():\n",
    "        day = row['date']\n",
    "        vol = row['volume']\n",
    "        pair = (vol, s)\n",
    "        if day not in trade_dates:\n",
    "            trade_dates[day] = []\n",
    "        trade_dates[day].append(pair)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finding the most-traded stock each day"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculating a dictionary where the keys are the days and the values are a tuple (trade_vol, stock_symbol). The stock_symbol represents the most-traded stock symbol on that day, and trade_vol represents the trade volume (stored in the volume column)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "most_traded_by_day = {}\n",
    "\n",
    "for day in trade_dates:\n",
    "    trade_dates[day].sort()\n",
    "    most_traded_by_day[day] = trade_dates[day][-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(list(most_traded_by_day.items())[0:2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finding days with unusually high trading volumes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To do this, I'll create a dictionary where the keys are the dates and the values are the total volume of that day (over all stocks)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trade_dates['2007-01-03']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "total_trades = {}\n",
    "for day in trade_dates:\n",
    "    day_vol = 0\n",
    "    for vol, stock in trade_dates[day]:\n",
    "        day_vol += vol  \n",
    "    total_trades[day_vol] = day\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "total_list = list(total_trades.items())\n",
    "total_list.sort(reverse=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 10 days with highest trading volumes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "total_list[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finding the most profitable stocks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I'll do this by:\n",
    "\n",
    "- Subtracting the initial close price (first row) from the final close price (last row), then computing a percentage relative to the initial price. This tells us how much our initial investment would have grown or reduced.\n",
    "- Sorting all of the percentages.\n",
    "- Finding the 10 stocks that grew the most in the time period."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "price_change = []\n",
    "for s in stocks:\n",
    "    initial_price = stocks[s].iloc[0,1]\n",
    "    final_price = stocks[s].iloc[-1,1]\n",
    "    change = (final_price - initial_price) / initial_price * 100\n",
    "    price_change.append((round(change,2), s))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "price_change.sort(reverse=True)\n",
    "price_change[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "stocks['admp'][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "stocks['admp'][-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The most profitable stock to buy would have been ADMP, which appreciated from around 7 cents to its current price of 4.43."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Possible next steps:\n",
    "    \n",
    "- Which stocks would have been the best to short-sell at the start of the period?\n",
    "- Which stocks have the most after-hours trading, and show the biggest changes between the closing price and the next day open?\n",
    "- Can technical indicators like Bollinger Bands help us forecast the market?\n",
    "- Which time periods have resulted in steady increases in prices, and which periods have resulted in steady declines?\n",
    "- Based on price, what was the optimal day to buy each stock if we wanted to hold them until now?\n",
    "- On days with high trading volume, do stocks move in one direction (up or down) more than the other one?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}