doc/4-5-concurrency.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Co-Routinen, Threads & Subprozesse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Concurrent Programming"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Das Konzept ***\"concurrent programming\"*** ist eine Sammlung von Techniken,\n",
    "um innerhalb eines Programmes mehrere Aufgaben unabhängig -- oder fast unabhängig -- voneiner bearbeiten zu können.\n",
    "Auf entsprechenden Machinen, bzw. in einem Verbund von Computern, können diese Aufgaben wenn möglich auch gleichzeitig ausgeführt werden - das nennt sich ***\"parallel computing\"***.\n",
    "\n",
    "* [Concurrent Computing](http://en.wikipedia.org/wiki/Concurrent_computing)\n",
    "* [Parallel Computing](http://en.wikipedia.org/wiki/Parallel_computing)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Co-Routinen\n",
    "\n",
    "Eine der einfachsten Formen von concurrency sind Co-Routinen.\n",
    "Eine Co-Routine verhält sich hierbei ganz ähnlich zu einer Funktion,\n",
    "welche aber mehrmals hintereinander aufgerufen werden kann (wenn sie sich intern wiederholt).\n",
    "Zu jedem Aufruf gibt es ein dazugehöriges Ergebnis oder eine Exception,\n",
    "die andeutet dass die Co-Routine zu Ende ist.\n",
    "\n",
    "Entscheidender Unterschied ist das `yield`-Statement: Es kann an dieser Stelle ein Zwischenergebnis zurückgeben oder auf die Eingabe eines Wertes warten.\n",
    "\n",
    "Verwendet werden Co-Routinen, indem sie zuerst Instanziert werden und dann entweder als Iterator zum Einsatz kommen oder explizit `.next()` zum Abrufen des nächsten Wertes aufgerufen wird.\n",
    "\n",
    "Wichtig ist zu verstehen, dass Co-Routinen nicht erst beim Aufrufen des `.next()` calls zum Arbeiten beginnen.\n",
    "Sie können selbständig im Hintergrund schon so lange beschäftigt sein,\n",
    "bis sie an der Stelle des `yield`-Statements ein fertiges Ergebnis zum Abruf bereit halten.\n",
    "\n",
    "Hier einfache Beispiele:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def coroutine1():\n",
    "    yield \"abc\"\n",
    "    yield \"xyz\"\n",
    "    yield 777"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "abc\n",
      "xyz\n",
      "777\n"
     ]
    },
    {
     "ename": "StopIteration",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mStopIteration\u001b[0m                             Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-2-ac681be2fb60>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[0;32m      3\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnext\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mc1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      4\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnext\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mc1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 5\u001b[1;33m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnext\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mc1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mStopIteration\u001b[0m: "
     ]
    }
   ],
   "source": [
    "c1 = coroutine1()\n",
    "print(next(c1))\n",
    "print(next(c1))\n",
    "print(next(c1))\n",
    "print(next(c1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def even_numbers(start_value = 0):\n",
    "    x = start_value\n",
    "    while True:\n",
    "        x += 1\n",
    "        if x % 2 == 0:\n",
    "            yield x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22\n",
      "24\n",
      "26\n",
      "28\n",
      "30\n"
     ]
    }
   ],
   "source": [
    "en = even_numbers(21)\n",
    "for i in range(5):\n",
    "    print(next(en))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Da Co-Routinen eine spezielle Form von Iteratoren sind,\n",
    "brechen sie in diesem Fall hier nicht von selbst ab.\n",
    "Wir müssen daher in dieser `for i in en2` Schleife sicherstellen,\n",
    "dass irgendwann `break` aufgerufen wird!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2\n",
      "4\n",
      "6\n",
      "8\n",
      "10\n",
      "12\n"
     ]
    }
   ],
   "source": [
    "en2 = even_numbers(1)\n",
    "for i in en2:\n",
    "    print(i)\n",
    "    if i > 10:\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`variable = yield` und `coroutine_instance.send(arg)` erlauben,\n",
    "Objekte an eine Co-Routine zu schicken.\n",
    "Achtung, das Timing von `next` und `send` darf nicht aus dem Takt kommen!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def triple():\n",
    "    while True:\n",
    "        print(\"    Ready to recieve values\")\n",
    "        y = yield\n",
    "        print(\"    I got %s\" % y)\n",
    "        yield y * 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Ready to recieve values\n",
      "init fertig\n",
      "    I got 21\n",
      "habe 63 erhalten\n",
      "    Ready to recieve values\n",
      "    I got -4\n",
      "habe -12 erhalten\n"
     ]
    }
   ],
   "source": [
    "t = triple()\n",
    "next(t)     # initialisierung der Co-Routine!\n",
    "print(\"init fertig\")\n",
    "z = t.send(21)\n",
    "print(\"habe %s erhalten\" % z)\n",
    "next(t)\n",
    "z = t.send(-4)\n",
    "print(\"habe %s erhalten\" % z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Bonus**: Verkettungen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "4\n",
      "9\n",
      "16\n",
      "25\n",
      "36\n",
      "49\n",
      "64\n",
      "81\n",
      "100\n"
     ]
    }
   ],
   "source": [
    "def oddvals():\n",
    "    x = 1\n",
    "    while True:\n",
    "        yield x\n",
    "        x += 2\n",
    "\n",
    "def sumall(co_other):\n",
    "    sum = 0\n",
    "    while True:\n",
    "        sum += next(co_other)\n",
    "        yield sum\n",
    "\n",
    "a1 = oddvals()\n",
    "sa = sumall(a1)\n",
    "for i in range(10):\n",
    "    print(next(sa))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Kontrolle:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 4, 9, 16, 25, 36, 49, 64, 81, 100]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[sum(range(1, 2 * x, 2)) for x in range(1, 11)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Threading"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from threading import Thread\n",
    "from queue import Queue\n",
    "from time import sleep, time\n",
    "from random import random"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     5[ms]: working in   T2\n",
      "     5[ms]: working in T1\n",
      "    94[ms]: working in T1\n",
      "    29[ms]: working in   T2\n",
      "   111[ms]: working in T1\n",
      "   113[ms]: working in   T2\n",
      "   195[ms]: working in T1\n",
      "   198[ms]: working in T1\n",
      "   196[ms]: working in   T2\n",
      "   227[ms]: working in T1\n",
      "   271[ms]: working in T1\n",
      "   248[ms]: working in   T2\n",
      "   313[ms]: working in T1\n",
      "   313[ms]: working in   T2\n",
      "   371[ms]: working in T1\n",
      "   390[ms]: working in   T2\n",
      "   426[ms]: working in T1\n",
      "   447[ms]: working in   T2\n",
      "   493[ms]: working in T1\n",
      "   578[ms]: working in T1\n",
      "   544[ms]: working in   T2\n",
      "   629[ms]: working in   T2\n",
      "   678[ms]: working in   T2\n",
      "   596[ms]: working in T1\n",
      "   692[ms]: working in   T2\n",
      "   695[ms]: working in T1\n",
      "   741[ms]: working in T1\n",
      "   737[ms]: working in   T2\n",
      "   805[ms]: working in   T2\n",
      "   812[ms]: working in   T2\n",
      "            got -1, finishing up\n",
      "   803[ms]: working in T1\n",
      "   894[ms]: working in T1\n",
      "            got -1, finishing up\n"
     ]
    }
   ],
   "source": [
    "t0 = time()\n",
    "\n",
    "def calculate(name, input_queue, return_queue):\n",
    "    sum = 0\n",
    "    while True:\n",
    "        c = input_queue.get()\n",
    "        # simulate some work time\n",
    "        t = 1000. * (time() - t0)\n",
    "        sleep(random() * 0.1)\n",
    "        print(\"%6.0f[ms]: working in %s\" % (t, name))\n",
    "        if c < 0:\n",
    "            print(\"            got -1, finishing up\")\n",
    "            break\n",
    "        sum += c\n",
    "    return_queue.put(sum)\n",
    "    \n",
    "input_queue = Queue()\n",
    "return_queue = Queue()\n",
    "\n",
    "t1 = Thread(target=calculate, args=(\"T1\", input_queue, return_queue))\n",
    "t2 = Thread(target=calculate, args=(\"  T2\", input_queue, return_queue))\n",
    "\n",
    "t1.start()\n",
    "t2.start()\n",
    "\n",
    "for k in range(99999, 99999+30):\n",
    "    input_queue.put(k)\n",
    "\n",
    "input_queue.put(-1)\n",
    "input_queue.put(-1)\n",
    "\n",
    "# wait for both threads to finish\n",
    "# (otherwise they are discarded when main thread finishes)\n",
    "t1.join()\n",
    "t2.join()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}