docs/source/python_environment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Python Environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We show here some examples of how to run Python on a Pynq platform. Python 3.6 \n",
    "is running exclusively on the ARM Cortex-A9 processor.  \n",
    "\n",
    "In the first example, which is based on calculating the factors and primes\n",
    "of integer numbers, give us a sense of the performance available when running \n",
    "on a 650MHz ARM Cortex-\n",
    "A9 dual core processor running Linux.  \n",
    "\n",
    "In the second set of examples, we leverage Python's `numpy` package and `asyncio` \n",
    "module to demonstrate how Python can communicate\n",
    "with programmable logic.\n",
    "\n",
    "\n",
    "## Factors and Primes Example\n",
    "Code is provided in the cell below for a function to calculate factors and \n",
    "primes. It contains some sample functions to calculate the factors and primes \n",
    "of integers. We will use three functions from the `factors_and_primes` module \n",
    "to demonstrate Python programming. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "\"\"\"Factors-and-primes functions.\n",
    "\n",
    "Find factors or primes of integers, int ranges and int lists\n",
    "and sets of integers with most factors in a given integer interval\n",
    "\n",
    "\"\"\"\n",
    "\n",
    "def factorize(n):\n",
    "    \"\"\"Calculate all factors of integer n.\n",
    "    \n",
    "    \"\"\"\n",
    "    factors = []\n",
    "    if isinstance(n, int) and n > 0:\n",
    "        if n == 1:\n",
    "            factors.append(n)\n",
    "            return factors\n",
    "        else:\n",
    "            for x in range(1, int(n**0.5)+1):\n",
    "                if n % x == 0:\n",
    "                    factors.append(x)\n",
    "                    factors.append(n//x)\n",
    "            return sorted(set(factors))\n",
    "    else:\n",
    "        print('factorize ONLY computes with one integer argument > 0')\n",
    "\n",
    "\n",
    "def primes_between(interval_min, interval_max):\n",
    "    \"\"\"Find all primes in the interval.\n",
    "        \n",
    "    \"\"\"\n",
    "    primes = []\n",
    "    if (isinstance(interval_min, int) and interval_min > 0 and \n",
    "       isinstance(interval_max, int) and interval_max > interval_min):\n",
    "        if interval_min == 1:\n",
    "            primes = [1]\n",
    "        for i in range(interval_min, interval_max):\n",
    "            if len(factorize(i)) == 2:\n",
    "                primes.append(i)\n",
    "        return sorted(primes)\n",
    "    else:\n",
    "        print('primes_between ONLY computes over the specified range.')\n",
    "\n",
    "        \n",
    "def primes_in(integer_list):\n",
    "    \"\"\"Calculate all unique prime numbers.\n",
    "    \n",
    "    \"\"\"\n",
    "    primes = []\n",
    "    try:\n",
    "        for i in (integer_list):\n",
    "            if len(factorize(i)) == 2:\n",
    "                primes.append(i)\n",
    "        return sorted(set(primes))\n",
    "    except TypeError:\n",
    "        print('primes_in ONLY computes over lists of integers.')\n",
    "\n",
    "\n",
    "def get_ints_with_most_factors(interval_min, interval_max):\n",
    "    \"\"\"Finds the integers with the most factors.\n",
    "        \n",
    "    \"\"\"\n",
    "    max_no_of_factors = 1\n",
    "    all_ints_with_most_factors = []\n",
    "    \n",
    "    # Find the lowest number with most factors between i_min and i_max\n",
    "    if interval_check(interval_min, interval_max):\n",
    "        for i in range(interval_min, interval_max):\n",
    "            factors_of_i = factorize(i)\n",
    "            no_of_factors = len(factors_of_i) \n",
    "            if no_of_factors > max_no_of_factors:\n",
    "                max_no_of_factors = no_of_factors\n",
    "                results = (i, max_no_of_factors, factors_of_i,\\\n",
    "                            primes_in(factors_of_i))\n",
    "        all_ints_with_most_factors.append(results)\n",
    "    \n",
    "        # Find any larger numbers with an equal number of factors\n",
    "        for i in range(all_ints_with_most_factors[0][0]+1, interval_max):\n",
    "            factors_of_i = factorize(i)\n",
    "            no_of_factors = len(factors_of_i) \n",
    "            if no_of_factors == max_no_of_factors:\n",
    "                results = (i, max_no_of_factors, factors_of_i, \\\n",
    "                            primes_in(factors_of_i))\n",
    "                all_ints_with_most_factors.append(results)\n",
    "        return all_ints_with_most_factors       \n",
    "    else:\n",
    "        print_error_msg() \n",
    "\n",
    "    \n",
    "def interval_check(interval_min, interval_max):\n",
    "    \"\"\"Check type and range of integer interval.\n",
    "    \n",
    "    \"\"\"\n",
    "    if (isinstance(interval_min, int) and interval_min > 0 and \n",
    "       isinstance(interval_max, int) and interval_max > interval_min):\n",
    "        return True\n",
    "    else:\n",
    "        return False\n",
    "\n",
    "    \n",
    "def print_error_msg():\n",
    "    \"\"\"Print invalid integer interval error message.\n",
    "    \n",
    "    \"\"\"\n",
    "    print('ints_with_most_factors ONLY computes over integer intervals where'\n",
    "            ' interval_min <= int_with_most_factors < interval_max and'\n",
    "            ' interval_min >= 1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we will call the factorize() function to calculate the factors of an integer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 13, 26, 41, 82, 533, 1066]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "factorize(1066)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The primes_between() function can tell us how many prime numbers there are in an \n",
    "integer range. Let’s try it for the interval 1 through 1066.  We can also use one \n",
    "of Python’s built-in methods len() to count them all."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "180"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(primes_between(1, 1066))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Additionally, we can combine len() with another built-in method, sum(), to calculate \n",
    "the average of the 180 prime numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "486.2055555555556"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "primes_1066 = primes_between(1, 1066)\n",
    "primes_1066_average = sum(primes_1066) / len(primes_1066)\n",
    "primes_1066_average"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This result makes sense intuitively because prime numbers are known to become less \n",
    "frequent for larger number intervals. These examples demonstrate how Python treats \n",
    "functions as first-class objects so that functions may be passed as parameters to \n",
    "other functions. This is a key property of functional programming and demonstrates \n",
    "the power of Python.\n",
    "\n",
    "In the next code snippet, we can use list comprehensions (a ‘Pythonic’ form of the \n",
    "map-filter-reduce template) to ‘mine’ the factors of 1066 to find those factors that \n",
    "end in the digit ‘3’."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3, 13, 23, 43, 53, 73, 83, 103, 113, 163, 173, 193, 223, 233, 263, 283, 293, 313, 353, 373, 383, 433, 443, 463, 503, 523, 563, 593, 613, 643, 653, 673, 683, 733, 743, 773, 823, 853, 863, 883, 953, 983, 1013, 1033, 1063]\n"
     ]
    }
   ],
   "source": [
    "primes_1066_ends3 = [x for x in primes_between(1, 1066) \n",
    "                     if str(x).endswith('3')]\n",
    "print('{}'.format(primes_1066_ends3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This code tells Python to first convert each prime between 1 and 1066 to a string and \n",
    "then to return those numbers whose string representation end with the number ‘3’. It \n",
    "uses the built-in str() and endswith() methods to test each prime for inclusion in the list.\n",
    "\n",
    "And because we really want to know what fraction of the 180 primes of 1066 end in a \n",
    "‘3’, we can calculate ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.25"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(primes_1066_ends3) / len(primes_1066)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These examples demonstrate how Python is a modern, multi-paradigmatic language. More \n",
    "simply, it continually integrates the best features of other leading languages, including \n",
    "functional programming constructs. Consider how many lines of code you would need to \n",
    "implement the list comprehension above in C and you get an appreciation of the power \n",
    "of productivity-layer languages. Higher levels of programming abstraction really do \n",
    "result in higher programmer productivity!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Numpy Data Movement\n",
    "\n",
    "Code in the cells below show a very simple data movement code snippet that can be used \n",
    "to share data with programmable logic.  We leverage the Python numpy package to \n",
    "manipulate the buffer on the ARM processors and can then send a buffer pointer to \n",
    "programmable logic for sharing data.\n",
    "\n",
    "We do not assume what programmable logic design is loaded, so here we only allocate \n",
    "the needed memory space and show that it can manipulated as a numpy array and contains \n",
    "a buffer pointer attribute.   That pointer can then can be passed to programmable \n",
    "logic hardware."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pynq\n",
    "\n",
    "def get_pynq_buffer(shape, dtype):\n",
    "    \"\"\" Simple function to call PYNQ's memory allocator with numpy attributes\n",
    "    \n",
    "    \"\"\"\n",
    "    return pynq.Xlnk().cma_array(shape, dtype)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the simple wrapper above, we can get access to memory that can be shared by both \n",
    "numpy methods and programmable logic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CMABuffer([[0, 0, 0, 0],\n",
       "           [0, 0, 0, 0],\n",
       "           [0, 0, 0, 0],\n",
       "           [0, 0, 0, 0]], dtype=uint32)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "buffer = get_pynq_buffer(shape=(4,4), dtype=np.uint32)\n",
    "buffer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To double-check we show that the buffer is indeed a numpy array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "isinstance(buffer,np.ndarray)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To send the buffer pointer to programmable logic, we use its physical address which \n",
    "is what programmable logic would need to communicate using this shared buffer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0x16846000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pl_buffer_address = hex(buffer.physical_address)\n",
    "pl_buffer_address"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this short example, we showed a simple allocation of a numpy array that is now ready \n",
    "to be shared with programmable logic devices.  With numpy arrays that are accessible to programmable logic, we can quickly manipulate and move data across software and hardware."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Asyncio Integration\n",
    "\n",
    "PYNQ also leverages the Python asyncio module for communicating with programmable logic \n",
    "devices through events (namely interrupts).\n",
    "\n",
    "A Python program running on PYNQ can use the asyncio library to manage multiple IO-bound \n",
    "tasks asynchronously, thereby avoiding any blocking caused by waiting for responses from \n",
    "slower IO subsystems. Instead, the program can continue to execute other tasks that are \n",
    "ready to run. When the previously-busy tasks are ready to resume, they will be executed \n",
    "in turn, and the cycle is repeated.\n",
    "\n",
    "Again, since we won't assume what interrupt enabled devices are loaded on programmable \n",
    "logic, we will show an example here a software-only asyncio example that uses asyncio's \n",
    "sleep method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import asyncio\n",
    "import random\n",
    "import time\n",
    "\n",
    "# Coroutine\n",
    "async def wake_up(delay):\n",
    "    '''A function that will yield to asyncio.sleep() for a few seconds\n",
    "       and then resume, having preserved its state while suspended\n",
    "\n",
    "    '''\n",
    "    start_time = time.time()\n",
    "    print(f'The time is: {time.strftime(\"%I:%M:%S\")}')\n",
    "    \n",
    "    print(f\"Suspending coroutine 'wake_up' at 'await` statement\\n\")\n",
    "    await asyncio.sleep(delay)\n",
    "    \n",
    "    print(f\"Resuming coroutine 'wake_up' from 'await` statement\")\n",
    "    end_time = time.time()\n",
    "    sleep_time = end_time - start_time\n",
    "    print(f\"'wake-up' was suspended for precisely: {sleep_time} seconds\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the wake_up function defined, we then can add a new task to the event loop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating task for coroutine 'wake_up'\n",
      "\n",
      "The time is: 10:29:45\n",
      "Suspending coroutine 'wake_up' at 'await` statement\n",
      "\n",
      "Resuming coroutine 'wake_up' from 'await` statement\n",
      "'wake-up' was suspended for precisely: 3.011084794998169 seconds\n"
     ]
    }
   ],
   "source": [
    "delay = random.randint(1,5)\n",
    "my_event_loop = asyncio.get_event_loop()\n",
    "    \n",
    "try:\n",
    "    print(\"Creating task for coroutine 'wake_up'\\n\")\n",
    "    wake_up_task = my_event_loop.create_task(wake_up(delay))\n",
    "    my_event_loop.run_until_complete(wake_up_task)\n",
    "except RuntimeError as err:\n",
    "    print (f'{err}' +\n",
    "        ' - restart the Jupyter kernel to re-run the event loop')\n",
    "finally:\n",
    "    my_event_loop.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All the above examples show standard Python 3.6 running on the PYNQ platform.  This entire notebook can be run on the PYNQ board - see the getting_started folder on the Jupyter landing page to rerun this notebook."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}