Python Standard Library #10 #11 #12 #13.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [Amin M. Boulouma Blog](https://amboulouma.com)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python Standard Library #10 #11 #12 #13.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### OS, Regex, Internet and Math in Python #10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> [Reminder] 🔔\n",
    "- Help the creator channel reach 20k subscribers. He will keep uploading quality content for you: [Amin M. Boulouma Channel](https://www.youtube.com/channel/UCOZbokHO727qeStxeYSKMUQ?sub_confirmation=1)\n",
    "- This tutorial is best understood following the video playlist: [Python Ultimate Tutorial [Official Documentation]](https://www.youtube.com/watch?v=vQqisFjAnsE&list=PLpMTHmi814W0nSToTOC0Q18kREOjcJspW&index=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hosted by Amin M. Boulouma, contact and questions: [amine.boulouma.com](https://amine.boulouma.com)\n",
    "- Python tutorial made simple: https://youtu.be/vQqisFjAnsE\n",
    "- Source code: https://amboulouma.com/python-tutorial\n",
    "- Github: https://github.com/amboulouma/python-ultimate-tutorial"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Operating System Interface"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "os.getcwd() "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "os.chdir('/Users/admin/temp')   # Change current working directory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "os.system('mkdir tempdire')   # Run the command mkdir in the system shell, return value must be 0 for dir creation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "dir(os)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "help(os)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "os.listdir()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import shutil\n",
    "\n",
    "shutil.copyfile('workfile', 'workfile.txt')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "shutil.move('workfile.txt', 'workfile.json')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### File Wildcards"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import glob\n",
    "\n",
    "glob.glob('*.ipynb')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Command Line Arguments"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "print(sys.argv)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Error Output Redirection and Program Termination"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sys.stderr.write('Warning, log file not found starting a new one\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### String Pattern Matching"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "\n",
    "re.findall(r'\\bf[a-z]*', 'which foot or hand fell fastest')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "re.sub(r'(\\b[a-z]+) \\1', r'\\1', 'cat in the the hat')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "'tea for too'.replace('too', 'two')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Mathematics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "math.cos(math.pi / 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "math.log(1024, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "\n",
    "random.choice(['apple', 'pear', 'banana'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "random.sample(range(10000000), 10)   # sampling without replacement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "random.random()    # random float"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "random.randrange(6)    # random integer chosen from range(6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import statistics\n",
    "\n",
    "data = [2.75, 1.75, 1.25, 0.25, 0.5, 1.25, 3.5]\n",
    "statistics.mean(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "statistics.median(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "statistics.variance(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "statistics.stdev(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Internet Access"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from urllib.request import urlopen\n",
    "\n",
    "with urlopen('https://github.com/amboulouma/python-ultimate-tutorial') as response:\n",
    "    for line in response:\n",
    "        line = line.decode('utf-8')  # Decoding the binary data to text.\n",
    "        if 'python' in line or 'Python' in line:  # look for Eastern Time\n",
    "            print(line)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import smtplib\n",
    "\n",
    "server = smtplib.SMTP('localhost') # server running needed here\n",
    "server.sendmail('amine@boulouma.com', 'contact@amboulouma.com',\n",
    "    \"\"\"To: jcaesar@example.org\n",
    "    From: soothsayer@example.org\n",
    "\n",
    "    Beware the Ides of March.\n",
    "    \"\"\"\n",
    "    )\n",
    "server.quit()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dates, Compression and Performance Python Standard Library 1 #11"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dates and Times"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dates are easily constructed and formatted\n",
    "from datetime import date\n",
    "now = date.today()\n",
    "now"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "now.strftime(\"%m-%d-%y. %d %b %Y is a %A on the %d day of %B.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "# dates support calendar arithmetic\n",
    "birthday = date(1995, 11, 18)\n",
    "age = now - birthday\n",
    "math.floor(age.days/365)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Compression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Common data archiving and compression formats are directly supported by modules including: zlib, gzip, bz2, lzma, zipfile and tarfile.Performance Measurement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import zlib\n",
    "\n",
    "s = b'witch which has which witches wrist watch'\n",
    "len(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "t = zlib.compress(s)\n",
    "len(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "help(zlib)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "zlib.decompress(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "zlib.decompress(zlib.crc32(s))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Performance Measurement"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In contrast to timeit’s fine level of granularity, the **profile** and **pstats** modules provide tools for identifying time critical sections in larger blocks of code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from timeit import Timer\n",
    "Timer('t=a; a=b; b=t', 'a=1; b=2').timeit()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "Timer('a,b = b,a', 'a=1; b=2').timeit()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tests, Templating, Multi-threading and Logging Modules in Python #12"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Quality Control"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def average(values):\n",
    "    \"\"\"Computes the arithmetic mean of a list of numbers.\n",
    "\n",
    "    >>> print(average([20, 30, 70]))\n",
    "    40.0\n",
    "    \"\"\"\n",
    "    return sum(values) / len(values)\n",
    "\n",
    "import doctest\n",
    "doctest.testmod()   # automatically validate the embedded tests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://github.com/3aransia/3aransia/blob/develop/aaransia/test.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import unittest\n",
    "\n",
    "class TestStatisticalFunctions(unittest.TestCase):\n",
    "\n",
    "    def test_average(self):\n",
    "        self.assertEqual(average([20, 30, 70]), 40.0)\n",
    "        self.assertEqual(round(average([1, 5, 7]), 1), 4.3)\n",
    "        with self.assertRaises(ZeroDivisionError):\n",
    "            average([])\n",
    "        with self.assertRaises(TypeError):\n",
    "            average(20, 30, 70)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "suite = unittest.TestLoader().loadTestsFromTestCase(TestStatisticalFunctions)\n",
    "runner = unittest.TextTestRunner(verbosity=2)\n",
    "runner.run(suite)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Output Formatting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import reprlib\n",
    "\n",
    "reprlib.repr(set('supercalifragilisticexpialidocious'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pprint\n",
    "\n",
    "t = {'name': 'Amin',\n",
    "    'middle_name': 'M.',\n",
    "    'last_name':'Boulouma',\n",
    "    'websites': {\n",
    "        'contact':'amine.boulouma.com',\n",
    "        'blog':'amboulouma.com',\n",
    "        'projects': {\n",
    "            '3aransia':'3aransia.com',\n",
    "            'tldrlibrary':'tldrlibrary.com'\n",
    "        }\n",
    "    }}\n",
    "\n",
    "print(t)\n",
    "pprint.pprint(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import textwrap\n",
    "\n",
    "doc = \"\"\"The wrap() method is just like fill() except that it returns\n",
    "    a list of strings instead of one big string with newlines to separate\n",
    "    the wrapped lines.\"\"\"\n",
    "\n",
    "print(textwrap.fill(doc, width=20))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Templating"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Nottinghamfolk send $10 to the ditch fund.'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from string import Template\n",
    "\n",
    "t = Template('${village}folk send $$10 to $cause.')\n",
    "\n",
    "t.substitute(village='Nottingham', cause='the ditch fund')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "t.safe_substitute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "t = Template('Return the $item to $owner.')\n",
    "d = dict(item='unladen swallow')\n",
    "t.substitute(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "t.safe_substitute(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter rename style (%d-date %n-seqnum %f-format):  Travel_Hungary_%n%f\n"
     ]
    }
   ],
   "source": [
    "import time, os.path\n",
    "\n",
    "photofiles = ['img_1074.jpg', 'img_1076.jpg', 'img_1077.jpg']\n",
    "class BatchRename(Template):\n",
    "    delimiter = '%'\n",
    "\n",
    "fmt = input('Enter rename style (%d-date %n-seqnum %f-format):  ') # rename as Amin_%n%f"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "img_1074.jpg --> Travel_Hungary_0.jpg\n",
      "img_1076.jpg --> Travel_Hungary_1.jpg\n",
      "img_1077.jpg --> Travel_Hungary_2.jpg\n"
     ]
    }
   ],
   "source": [
    "t = BatchRename(fmt)\n",
    "date = time.strftime('%d%b%y')\n",
    "for i, filename in enumerate(photofiles):\n",
    "    base, ext = os.path.splitext(filename)\n",
    "    newname = t.substitute(d=date, n=i, f=ext)\n",
    "    print('{0} --> {1}'.format(filename, newname))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Multi-threading"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The main program continues to run in foreground.\n",
      "Finished background zip of: myFile.txt\n",
      "Main program waited until background was done.\n"
     ]
    }
   ],
   "source": [
    "import threading, zipfile\n",
    "\n",
    "class AsyncZip(threading.Thread):\n",
    "    def __init__(self, infile, outfile):\n",
    "        threading.Thread.__init__(self)\n",
    "        self.infile = infile\n",
    "        self.outfile = outfile\n",
    "\n",
    "    def run(self):\n",
    "        f = zipfile.ZipFile(self.outfile, 'w', zipfile.ZIP_DEFLATED)\n",
    "        f.write(self.infile)\n",
    "        f.close()\n",
    "        print('Finished background zip of:', self.infile)\n",
    "\n",
    "background = AsyncZip('myFile.txt', 'myarchive.zip')\n",
    "background.start()\n",
    "\n",
    "print('The main program continues to run in foreground.') #synchronously\n",
    "\n",
    "background.join()    # Wait for the background task to finish\n",
    "print('Main program waited until background was done.') # done after both executed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Logging"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Warning:config file server.conf not found\n",
      "ERROR:root:Error occurred\n",
      "CRITICAL:root:Critical error -- shutting down\n"
     ]
    }
   ],
   "source": [
    "import logging\n",
    "\n",
    "logging.debug('Debugging information')\n",
    "logging.info('Informational message')\n",
    "logging.warning('Warning:config file %s not found', 'server.conf')\n",
    "logging.error('Error occurred')\n",
    "logging.critical('Critical error -- shutting down')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## More on Lists, Heaps and Decimal Precision in Python #13"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tools for Working with Lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from array import array\n",
    "a = array('H', [4000, 10, 700, 22222])\n",
    "sum(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "a[1:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import bisect\n",
    "\n",
    "scores = [(100, 'perl'), (200, 'tcl'), (400, 'lua'), (500, 'python')]\n",
    "bisect.insort(scores, (300, 'ruby'))\n",
    "scores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"assets/heap.png\" alt=\"heap\" width=\"400\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from heapq import heapify, heappop, heappush\n",
    "\n",
    "data = [1, 3, 5, 7, 9, 2, 4, 6, 8, 0]\n",
    "heapify(data)                      # rearrange the list into heap order\n",
    "heappush(data, -5)                 # add a new entry\n",
    "[heappop(data) for i in range(3)]  # fetch the three smallest entries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Decimal Floating Point Arithmetic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Uses: \n",
    "- financial applications and other uses which require exact decimal representation,\n",
    "- control over precision,\n",
    "- control over rounding to meet legal or regulatory requirements,\n",
    "- tracking of significant decimal places, or\n",
    "- applications where the user expects the results to match calculations done by hand."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from decimal import *\n",
    "round(Decimal('0.70') * Decimal('1.05'), 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "round(.70 * 1.05, 2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "getcontext().prec = 36\n",
    "Decimal(1) / Decimal(7)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}