Assignment 3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SYDE556/750 Assignment 3: Connecting Neurons\n",
    "\n",
    "- Due Date: March 3rd\n",
    "- Total marks: 10 (10% of final grade)\n",
    "- Late penalty: 1 mark per day\n",
    "\n",
    "- It is recommended that you use a language with a matrix library and graphing capabilities.  Two main suggestions are Python and MATLAB.\n",
    "- *Do not use any code from Nengo*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1) Decoding from a population\n",
    "\n",
    "As you did in previous assignments, make a population of 20 LIF neurons representing a 1-dimensional value, and compute a decoder for them.  For parameters, $\\tau_{ref}$=0.002s, $\\tau_{RC}$=0.02s, the maximum firing rates are chosen randomly from a uniform distribution between 100 and 200Hz (at the max radius), and the x-intercepts are chosen randomly from a uniform distribution between -2 and 2.  Remember that the $\\alpha$ and $J^{bias}$ terms are computed based on these x-intercepts and maximum firing rates.\n",
    "\n",
    "It is generally easiest to compute decoders using the original method from Assignment 1, where we use the rate-mode approximation for the neurons to generate the $A$ matrix, then find $\\Gamma=A^T A + \\sigma^2 I$.  You can use this approach to find decoders, and these decoders should work even when you simulate the neurons in terms of spikes (in question 2 on).  The only difference will be that they will need to be scaled by ``dt``, your simulation time step.\n",
    "\n",
    "You can use this same method for computing decoders for this whole assignment.\n",
    "\n",
    "<ol style=\"list-style:lower-alpha outside none;\" start=\"1\">\n",
    "<li>[0.5 marks] Plot the tuning curves (firing rate of each neuron for different $x$ values between -2 and 2)</li>\n",
    "<li> [0.5 marks] Compute the decoders and plot $(x-\\hat{x})$.  When computing decoders, take into account noise ($\\sigma$=0.1 times 200Hz).  When computing $\\hat{x}$, add random gaussian noise with $\\sigma$=0.1 times 200Hz to the activity. Report the Root Mean-Squared Error (RMSE).</li>\n",
    "</ol>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2) Decoding from two spiking neurons\n",
    "\n",
    "Choose a neuron from part 1 that has a firing rate of somewhere between 20-50Hz for $x$=0.  Using that neuron's $\\alpha$ and $J^{bias}$ value, construct two neurons: both with the same $\\alpha$ and $J^{bias}$, but one with $e$=+1 and the other with $e$=-1.  With the function from the last assignment, generate a random input $x(t)$ that is 1 second long, with rms=1, dt=0.001, and an upper limit of 5Hz.  Feed that signal into the two neurons and generate spikes.  Decode the spikes back into $\\hat{x}(t)$ using a post-synaptic current filter $h(t)$ with a time constant of $\\tau$=0.005.\n",
    "\n",
    "<ol style=\"list-style:lower-alpha outside none;\" start=\"1\">\n",
    "<li>[0.5 marks] Plot the post-synaptic current $h(t)=e^{-t/\\tau}/ \\int e^{-t/\\tau} dt$</li>\n",
    "<li>[0.5 marks] Plot the original signal $x(t)$, the spikes, and the decoded $\\hat{x}(t)$ all on the same graph</li>\n",
    "<li>[0.5 marks] Compute the RMSE of the decoding</li>\n",
    "</ol>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3) Decoding from many neurons\n",
    "\n",
    "Repeat question 2, but with more neurons.  Instead of picking particular neurons, randomly generate them with x-intercepts uniformly distributed between -2 and 2 and with maximum firing rates between 100 and 200 Hz.  Randomly choose encoder values to be either -1 or +1.\n",
    "\n",
    "<ol style=\"list-style:lower-alpha outside none;\" start=\"1\">\n",
    "<li>[2 marks] Plot Root Mean-Squared Error as the number of neurons increases, on a log plot.  Try 8 neurons, 16 neurons, 32, 64, 128, up to 256.  For the RMSE for a particular number of neurons, average over at least 5 randomly generated groups of neurons.  For each group of neurons, randomly generate the signal $x(t)$.  Use the same parameters as in question 2. Note: the RMSE should go down as the number of neurons increases</li>\n",
    "</ol>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4) Connecting two groups of neurons\n",
    "\n",
    "For this question, use two groups of neurons with intercepts between [-1, 1] to compute $y = 2x+1$.  The first group of neurons will represent $x$ and the second group will represent $y$.\n",
    "\n",
    "Start by computing decoders.  You will need two decoders: one to decode $f(x)=2x+1$ from the first population, and one to decode $f(y)=y$ (the standard representational decoder) from the second population.  Remember that $\\Upsilon$ can change depending on what function you want to decode.\n",
    "\n",
    "Use the same neuron parameters as for previous questions, and use 200 randomly generated neurons in each population.\n",
    "\n",
    "<ol style=\"list-style:lower-alpha outside none;\" start=\"1\">\n",
    "<li>[1 mark] Show the behaviour of the system with an input of $x(t)=t-1$ for 1 second (a linear ramp from -1 to 0).  Plot the ideal $x(t)$ and $y(t)$ values, along with $\\hat{y}(t)$.</li>\n",
    "<ul>\n",
    "<li>Note that you should use the decoders that work for any input over the range of intercepts: do not re-compute decoders for any particular input (i.e. set of $x$ values).</li>\n",
    "<li>Input $x(t)$ into the first group of neurons and produce spikes.  Decode from those spikes using the decoder for $f(x)=2x+1$.  Input that decoded result into the second group of neurons to produce spikes.  Use the second decoder ($f(y)=y$) to decode $\\hat{y}(t)$.</li>\n",
    "</ul>\n",
    "<p>\n",
    "<li>[0.5 marks] Repeat part (a) with an input that is ten randomly chosen values between -1 and 0, each one held for 0.1 seconds (a randomly varying step input)</li>\n",
    "\n",
    "<li>[0.5 marks] Repeat part (a) with an input that is $x(t)=0.2sin(6\\pi t)$. Briefly discuss the results for this question.</li>\n",
    "</ol>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5) Connecting three groups of neurons\n",
    "\n",
    "For this question, use three groups of neurons with intercepts from [-1, 1] to compute $z = 2y+0.5x$.  Follow the same steps as question 4, but take the decoded outputs from the first two groups of neurons ($f(y)=2y$ and $f(x)=0.5x$), add them together, and feed that into the third group of neurons.\n",
    "\n",
    "<ol style=\"list-style:lower-alpha outside none;\" start=\"1\">\n",
    "<li>[1 mark] Plot $x(t)$, $y(t)$, the ideal $z(t)$, and the decoded $\\hat{z}(t)$ for an input of $x(t)=cos(3\\pi t)$ and $y(t)=0.5 sin (2 \\pi t)$ (over 1.0 seconds)</li>\n",
    "\n",
    "<li>[0.5 marks] Plot $x(t)$, $y(t)$, the ideal $z(t)$, and the decoded $\\hat{z}(t)$ for a random input over 1 second.  For $x(t)$ use a random signal with a limit of 8 Hz and `rms`=1.  For $y(t)$ use a random signal with a limit of 5 Hz and `rms`=0.5.</li>\n",
    "</ol>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6) Computing with vectors\n",
    "\n",
    "Do the same thing as questions 4 and 5, but with 2-dimensional vectors instead of scalars.  Everything else is the same.  For your encoders $e$, randomly generate them over the unit circle.  \n",
    "\n",
    "The function to compute is $w = x-3y+2z-2q$.  This requires five groups of neurons: $x$, $y$, $z$, $q$, and $w$.  Each of them represents a 2-dimensional value.  The outputs from $x$, $y$, $z$, and $q$ all feed into $w$.\n",
    "\n",
    "<ol style=\"list-style:lower-alpha outside none;\" start=\"1\">\n",
    "<li>[1 mark] Plot the decoded output $\\hat{w}(t)$ and the ideal $w$ for $x=[0.5,1], y=[0.1,0.3], z=[0.2,0.1], q = [0.4,-0.2]$.  (Note that these are all constants so they don't change over time, but still plot it for 1.0 seconds on one or more 2D graphs)</li>\n",
    "<li>[0.5 marks] Produce the same plot for $x=[0.5,1], y=[sin(4\\pi t),0.3], z=[0.2,0.1], q = [sin(4\\pi t),-0.2]$.</li>\n",
    "<li>[0.5 marks] Describe your results and discuss why and how they stray from the expected answer.</li></ol>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}