diff --git a/thesis/ch2.tex b/thesis/ch2.tex index c01118b5..a4381cd8 100644 --- a/thesis/ch2.tex +++ b/thesis/ch2.tex @@ -5,7 +5,13 @@ \section{The AdEx neuron model} This is a leaky-integrate-and-fire (LIF) neuron model, with two additions. First, the full upstroke of each spike is simulated, as an exponential runoff. Second, an extra dynamic variable is added: the adaptation current. -This allows the simulation of many non-linear effects of real neurons, like spike-rate adaptation and post-inhibitory rebound. (Note however that we do not focus on any of these effects in this thesis). +This current allows the simulation of many non-linear effects of real neurons, like spike-rate adaptation and post-inhibitory rebound. (Note however that we do not focus on adaptation effects in this thesis). + +\marginpar{ + \includegraphics[w=1]{exIF-fit.png} + \captionof{figure}{A linear-plus-exponential model (red) fit to a real cortical pyrimidal neuron (black), from \cite{Badel2008ExtractingNonlinearIntegrateandfire}} + \label{fig:exIF-fit} +} The AdEx model consists of two differential equations, one to simulate the membrane voltage $V$, and one for the adaptation current $w$: \begin{align} @@ -19,12 +25,13 @@ \section{The AdEx neuron model} \end{align} $I_\syn$ is the synaptic current, explained in \cref{sec:synapse_model}. -We use the sign convention of inter alia Dayan \& Abbott (\cite{Dayan2001TheoreticalNeuroscienceComputational}, ch.~5.3, p.~162) where membrane currents are defined as positive when positive charges flow \emph{out} of the cell. I.e. a positive $I_\syn$ decreases the membrane voltage (itself defined as the electric potential inside minus outside the cell). +We use the sign convention of inter alia Dayan \& Abbott\footnote{\cite{Dayan2001TheoreticalNeuroscienceComputational}, ch.~5.3, p.~162} where membrane currents are defined as positive when positive charges flow \emph{out} of the cell. I.e. a positive $I_\syn$ decreases the membrane voltage (itself defined as the electric potential inside minus outside the cell). Other parameters are explained in \cref{tab:AdEx-params}. In this chapter, we will often analyse $C\d{V}$ as a function of $V$, i.e. analyse it as a dynamical system: will the voltage increase or decrease at the current voltage? -For conciceness, we will call this function $F(V)$. I.e. $F(V) = C \d{V} =$ the right-hand-side of \cref{eq:AdEx-V}, here. +For conciceness, we will call this function $F(V)$. I.e. $F(V) = C \d{V} =$ the right-hand-side of \cref{eq:AdEx-V}, here. We'll mostly analyse $F$ in the absence of synaptic and adaptation currents, i.e. for $I_\syn$ and $w$ both zero. +\Cref{fig:exIF-fit} shows the $F(V)$ curve for an AdEx neuron fit to a real neuron. \begin{table}[h] \begin{sidecaption} @@ -36,8 +43,8 @@ \section{The AdEx neuron model} $V$ & Membrane voltage & V \\ $g_L$ & Input / leak conductance & S (V/A) \\ $E_L$ & Resting / leak potential & V \\ - $Δ_T$ & Steepness of $F(V)$ around the firing threshold & V \\ - $V_T$ & Location of minimum of $F(V)$ & V \\ + $Δ_T$ & Steepness of $F$ around the firing threshold & V \\ + $V_T$ & Location of minimum of $F$ & V \\ $w$ & Adaptation current & A \\ $τ_w$ & Time constant of adaptation current & s \\ $a$ & Sensitivity of adaptation current to $V$ & S \\ @@ -47,18 +54,54 @@ \section{The AdEx neuron model} We solve these equations using first-order (Euler) integration. -In addition to the two differential equations, the AdEx model also consists of an instantaneous reset condition. When the membrane voltage $V$ reaches a certain threshold, a spike is recorded, $V$ is reset, and $w$ is increased: - +In addition to the two differential equations, the AdEx model also consists of an instantaneous reset condition. When the membrane voltage $V$ reaches a certain threshold $θ$, a spike is recorded, $V$ is reset, and $w$ is increased: \begin{align} \text{if}\ V > θ\ \text{then:}\ \label{eq:AdEx-reset} & V ← V_r \\ & w ← w + Δw \notag \end{align} +\subsection{Analysis} + +Where are the fixed points of the dynamical system $C\d{V} = F(V)$? I.e, where is $F = 0$? Like other neuron models, there are two fixed points: a stable one at the leak potential $E_L$, and an unstable one at the instantaneous firing threshold (which we'll call $E_T$). + +We see in \cref{eq:AdEx-V} (for $I_\syn = 0$ and $w = 0$) that the leak potential $E_L$ is indeed a fixed point -- or rather lies very close to a fixed point: the exponential term is negligibly small at $V = E_L$. +\marginpar{ + \includegraphics[w=1]{LambertW.png} + \captionof{figure}{The Lambert $W$ functions for real numbers.} + \label{fig:LambertW} +} +The second fixed point has no straightforward expression. The exact solutions for $F$'s roots need the so called Lambert $W$ or `product logarithm' functions $W_0$ and $W_{-1}$ (see \cref{fig:LambertW}). $F$'s roots are found at: +\begin{equation} + V = E_L - Δ_T W_k \left( -\exp\left( \frac{E_L - V_T}{Δ_T} \right) \right) \label{eq:LambertW} +\end{equation} +..where $k = 0$ gives the resting potential, and $k = -1$ the instantaneous firing threshold.\footnote{ + Here too we see that the true resting potential is almost identical to $E_L$: $W_0$ passes through zero, and its argument is $≈ 0$, because the exponential's argument is negative. For $W_{-1}$ however, \cref{fig:LambertW} shows that the second term is not negligible around $0$.} + +% source: +% - https://www.wolframalpha.com/input?i=real+roots+of+f%28V%29+%3D+-g*%28V-L%29%2Bg*D*exp%28%28V-T%29%2FD%29 +% - https://www.wolframalpha.com/input?i=L+-+D+ProductLog%28-1%2C+-e%5E%28%28L+-+T%29%2FD%29%29+for+L%3D-65%2C+D%3D0.8%2C+T%3D-52 + +Also of interest -- especially when comparing with the Izhikevich neuron later -- is the slope of AdEx's $F(V)$, i.e. its derivative with respect to $V$: +\begin{align} + \d[V]{F} &= \dd{V} \left( -g_L (V - E_L) + + g_L Δ_T \exp\left(\frac{V - V_T}{Δ_T}\right) \right) \notag \\ + &= -g_L + g_L Δ_T \frac{1}{Δ_T} \exp\left(\frac{V - V_T}{Δ_T}\right) \label{eq:dFdV} +\end{align} +This derivative is zero at $V = V_T$. I.e, unlike what is suggested by the `$T$' subscript, and the name `effective threshold potential' given to it in \cite{Naud2008FiringPatternsAdaptive}, $V_T$ is not the instantaneous threshold potential (it is not a zero of $F$), but rather the minimum of $F$ (it is the zero of $\d[V]{F}$). + +From \cref{eq:dFdV}, we also calculate the slope of $F$ at its two roots. At $V = E_L$, the slope (also known as the leak or input conductance here) is $-g_L$, plus a negligibly small exponential term. The negative sign shows that this is a stable fixed point: in a linearization of $F$ around this point, at voltages below the resting potential, $F$ (i.e. $C \d{V}$) is positive and thus $V$ will increase. At voltages above $E_L$, $F$ is negative and $V$ will decrease. Small deviations on either side of the resting potential will thus decay back to this resting potential. + +At the firing threshold $V = E_T$ (i.e. the second solution to \cref{eq:LambertW}), the slope is: +\begin{equation} + g_L \left( \exp\left(\frac{E_T - V_T}{Δ_T}\right) - 1 \right) \label{eq:AdEx-slope} +\end{equation} +..which is positive (as $E_T > V_T$), indicating that this is an unstable fixed point: a small deviation of the voltage above $E_T$ will blow up to infinity (i.e, a spike is generated). + \section{Alternative neuron models} -Why did we choose the AdEx model to simulate neuron voltages? In short, because it strikes a good balance between realism and complexity. (Complexity, as in: still having a compact set of free parameters). We discuss here briefly two alternative neuron models: the simple leaky-integrate-and-fire (LIF) neuron, and the more complex Hodgkin-Huxley (HH) neuron. +Why did we choose the AdEx model to simulate neuron voltages? In short, because it strikes a good balance between realism and complexity. We briefly consider here two alternative neuron models: the simple leaky-integrate-and-fire (LIF) neuron, and the more complex Hodgkin-Huxley (HH) neuron. In the next section, we go into more depth on a third alternative, the very similar Izhikevich neuron. A simpler model than AdEx would be the well-known LIF neuron: @@ -66,22 +109,29 @@ \section{Alternative neuron models} C \d{V} = -g_L (V - E_L) - I_\syn \\[1em] \text{if}\ V > θ\ \text{then:}\ V ← V_r \\ \end{align*} -As is apparent from comparing with \cref{eq:AdEx-V,eq:AdEx-reset}, the AdEx model is an extension of the LIF model. The LIF neuron lacks a simulation of the upstroke of spikes (the exponential term in \cref{eq:AdEx-V}), and the slower time-scale adaptation current (\cref{eq:AdEx-w}), which allows the simulation of many qualitatively different real neuron types. +As is apparent from comparing this with \cref{eq:AdEx-V,eq:AdEx-reset}, the AdEx model is an extension of the LIF model. The LIF neuron lacks a simulation of the upstroke of spikes (the exponential term in \cref{eq:AdEx-V}), and the slower time-scale adaptation current (\cref{eq:AdEx-w}), which allows the simulation of many qualitatively different real neuron types. It is especially this first addition, the full upstroke simulation, that seems relevant in generating realistic voltage traces. Would this thesis have been very different had we used LIF neurons instead? Probably not, though it might depend on the mean voltage level of the simulated neuron: if it is well below the firing threshold, both LIF and AdEx are linear (the exponential term is negligible), and they behave quasi identically. When a spike is generated in the AdEx model, the exponential feedback makes the upstroke very fast, and thus not many timesteps in the simulation are spent on it, versus the linear regime. -[Reference the Vdot-V, dynamical system fig] -On the other hand, when the neuron would continuously teeter just below its firing threshold, the LIF and AdEx models do not behave similarly. LIF's $F(V)$ curve is still fully linear, while AdEx's is not, and AdEx will behave more like a real neuron. [Reference fig with real neuron Vdot-V curve] +On the other hand, when the neuron would continuously teeter just below its firing threshold, the LIF and AdEx models do not behave similarly. LIF's $F(V)$ curve is still fully linear, while AdEx's is not, and AdEx will behave more like a real neuron -- see \cref{fig:exIF-fit}. Another well-known alternative neuron model is the class of Hodgin-Huxley (HH)-like neurons. These models simulate the full trajectory of a spike: both its upstroke and its downstroke. Unfortunately they also have many free parameters. They also take a bit longer to simulate, being higher dimensional (having more differential equations), and containing many more exponential terms, which take the brunt of the time when numerically evaluating a differential expression. +\begin{figure}[h] + \begin{sidecaption} + {Neuron models as dynamical systems: a comparison of the $F(V)$ curves of the Izhikevich and AdEx neurons. `Experimental' is an AdEx neuron fit to a cortical pyramidal neuron (using data from \cite{Badel2008ExtractingNonlinearIntegrateandfire}). The Izhikevich neuron's parameters were chosen to match the fixed points and the leak conductance. Arrows indicate whether the voltage will increase or decrease (1D flow field). →~\textbullet{}~← is a stable fixed point (resting potential), ←~o~→ is an unstable fixed point (spike threshold). $\dot{V} ≡ \d{V}$.} + [fig:dynsys] + \includegraphics[w=0.9]{dynsys.png} + \end{sidecaption} +\end{figure} + \section{The Izhikevich neuron} +Another alternative neuron model is the Izhikevich neuron, which is exceedingly similar to the AdEx neuron. These are the Izhikevich equations, using the same symbols as used before (in \cref{eq:AdEx-V,eq:AdEx-w,eq:AdEx-reset}): - \begin{align} C \d{V} &= k (V - E_L) (V - E_T) - I_\syn - w \label{eq:Izh-V} \\[1em] τ_w \d{w} &= b (V - E_L) - w \label{eq:Izh-w} \\[2em] @@ -90,15 +140,15 @@ \section{The Izhikevich neuron} &w ← w + Δw \notag \end{align} -We have introduced two new parameters not present in the AdEx equations: $k$, the steepness of the parabola; and $E_T$, the 'instantaneous' firing threshold -- that is, the firing threshold in the absence of any synaptic or adaptation currents. +We have introduced two new parameters not present in the AdEx equations: a `steepness' parameter $k$; and $E_T$, the instantaneous firing threshold -- that is, the firing threshold in the absence of any synaptic or adaptation currents. -Note that, in the absence of synaptic and adaptation currents, \cref{eq:Izh-V} is a quadratic function of $V$, and its two zeros, $E_L$ and $E_T$, are apparent. +Note that, in the absence of synaptic and adaptation currents, \cref{eq:Izh-V} is a quadratic function of $V$, and its two zeros, $E_L$ and $E_T$, are directly apparent. These two zeros are respectively the stable fixed point (the resting threshold or leak potential $E_L$), and the unstable fixed point (the firing threshold $E_T$). \subsection{Correspondences with AdEx} -In Izhikevich's book \cite{Izhikevich2007DynamicalSystemsNeuroscience} (section 5.2.4, equations 5.7 \& 5.8), different names are used for the same quantities: +In Izhikevich's book\footnote{\cite{Izhikevich2007DynamicalSystemsNeuroscience}, section 5.2.4, equations 5.7 \& 5.8}, different names are used for the same quantities: \begin{align} C \d{v} &= k(v - v_r)(v-v_t) - u + I \label{eq:IzhIzh-v} \\[1em] \d{u} &= a(b(v-v_r) - u) \\[2em] @@ -129,68 +179,36 @@ \subsection{Correspondences with AdEx} \end{table} Beside these straightforward correspondences, there are some parameters in either model that have no direct equivalent in the other: $k$ and $E_T$ in Izhikevich, and $g_L$, $Δ_T$, and $V_T$ in AdEx. -Here, we find correspondences for those parameters too. +For those, we'll look at the shape of Izhikevich's $F(V)$, as we've done for the AdEx neuron before. First, the AdEx parameter $g_L$. This is the input conductance, a.k.a. the leak conductance, a.k.a. the slope of $F(V)$ around the leak potential. We can find this same conductance for the Izhikevich neuron too, by taking the derivative with respect to $V$ of the right hand side of \cref{eq:Izh-V}, at $w = 0$, $I_\syn = 0$, and $V = E_L$. We find: \begin{equation} \dd{V}(k(V-E_L)(V-E_T)) \Big|_{V=E_L} = k(E_L - E_T) \end{equation} -(this value is negative: the leak potential is a stable fixed point. This corresponds to \cref{eq:AdEx-V}, where we find `$-g_L$'.). -We thus have our first nontrivial correspondence between the AdEx and Izhikevich parameters: +(this value is negative: the leak potential is a stable fixed point. This corresponds to \cref{eq:AdEx-V}, where we find `$-g_L$'). +Thus, our first nontrivial correspondence: \begin{equation} g_L = k (E_T - E_L) \label{eq:leak_conductance} \end{equation} -Next, we look at the AdEx parameter $V_T$. As mentioned before, this is the minimum of ts the AdEx $F(V)$ function. We show this here by finding roots of the derivative of $F(V)$ (\cref{eq:AdEx-V} in the absence of synaptic and adaptation currents): -\begin{align} - \d[V]{F} &= \dd{V} \left( -g_L (V - E_L) - + g_L Δ_T \exp\left(\frac{V - V_T}{Δ_T}\right) \right) \notag \\ - &= -g_L + g_L Δ_T \frac{1}{Δ_T} \exp\left(\frac{V - V_T}{Δ_T}\right) \label{eq:dFdV} -\end{align} -We indeed find a zero of $\d[V]{F}$ at $V = V_T$. - -I.e, unlike what is suggested by the `$T$' subscript, and the name `effective threshold potential' given to it in \cite{Naud2008FiringPatternsAdaptive}, $V_T$ is not the instantaneous threshold potential (it is not a zero of $F$), but rather the minimum of $F$ (it is the zero of $\d[V]{F}$). - -The minimum of Izhikevich's $F$ is easily found as the average of the parabola's two zeros. -We thus find our second nontrivial correspondence between an Izhikevich and an AdEx neuron: -\begin{equation} - V_T = (E_L + E_T) / 2 -\end{equation} - -Where then is the actual instantaneous firing threshold for the AdEx neuron? -There is no solution using only elementary functions. But using the Lambert $W$ or `product logarithm' functions (see \cref{fig:LambertW}), the roots of AdEx's $F$ are at: -\begin{equation} - V = E_L - Δ_T W_k \left( -\exp\left( \frac{E_L - V_T}{Δ_T} \right) \right) \label{eq:LambertW} -\end{equation} -..where $k = 0$ gives the resting potential, and $k = -1$ the instantaneous firing threshold.\\ -The argument of the exponential is negative, and hence the argument of the $W$ function is very small. $W_0$ passes through zero; hence the actual resting potential -- though not exactly equal to it -- is very close to $E_L$. - -\marginpar{ - \includegraphics[w=1]{LambertW.png} - \captionof{figure}{The Lambert $W$ functions for real numbers.} - \label{fig:LambertW} -} -% source: -% - https://www.wolframalpha.com/input?i=real+roots+of+f%28V%29+%3D+-g*%28V-L%29%2Bg*D*exp%28%28V-T%29%2FD%29 -% - https://www.wolframalpha.com/input?i=L+-+D+ProductLog%28-1%2C+-e%5E%28%28L+-+T%29%2FD%29%29+for+L%3D-65%2C+D%3D0.8%2C+T%3D-52 +We've seen that $V_T$ is the minimum of AdEx's $F$. +The minimum of Izhikevich's $F$ is easily found as the average of the parabola's two zeros. I.e, $V_T$ corresponds to $(E_L + E_T) / 2$. -Given that Izhikevich's $F$ is a parabola, with equal slopes at both roots, we already know the slope of its $F$ at the firing threshold: it is the same as the leak conductance, \cref{eq:leak_conductance}. For AdEx's $F$, \cref{eq:dFdV} gives us its slope at the firing threshold $E_T$ (i.e. at \cref{eq:LambertW} for $k = -1$): -\begin{equation} - g_L \left( \exp\left(\frac{E_T - V_T}{Δ_T}\right) - 1 \right) -\end{equation} +Finally, $Δ_T$ co-determines the slope of AdEx's $F$ at the firing threshold (\cref{eq:AdEx-slope}). Given that Izhikevich's $F$ is a parabola, with slopes equal in magnitude at both roots, we already know the firing threshold slope: it is the same as the leak conductance, \cref{eq:leak_conductance}.\\ +Here, the AdEx model is more expressive than Izhikevich's: the slope of $\d{V}$ at the firing threshold can be independently tweaked from the leak conductance; in Izhikevich these two are clung together by the form of the quadratic equation. \subsection{Comparison with AdEx} -These models are very similar. Their phase spaces are topologically identical: -the adaptive current equation is identical (up to a renaming of the variables); and the $\dot{V}(V)$-graph has the same shape, with two fixed points: a stable fixed point at the resting potential, and an unstable one at the firing threshold. +The Izhikevich and AdEx models are very similar. Their phase spaces are topologically identical: +the adaptive current equation is identical (up to a renaming of the variables); and the $F(V)$-graph has the same shape, with two fixed points: a stable fixed point at the resting potential, and an unstable one at the firing threshold (\cref{fig:dynsys}). -They differ in the exact shape: Izhikevich's $F(V)$ is a parabola, while AdEx is the more realistic 'linear subthreshold and then transitioning to an exponential'. [ref bio fig] +They differ in the exact shape: Izhikevich's $F(V)$ is a parabola, while AdEx is the more realistic 'linear subthreshold, and then transitioning to an exponential'. [ref bio fig] As a result, Izhikevich neurons have an unrealistically slow spike upstroke. \clearpage -\subsection{Synapse model} +\section{Synapse model} \label{sec:synapse_model} One unexplained term in our neuron model \cref{eq:AdEx-V} is the synaptic current, $I_\syn$. @@ -241,7 +259,8 @@ \subsection{Synapse model} With this, and when all synapses have the same time constant $\tau$, the two sums in \cref{eq:I_syn_factor} can be factored as follows:\footnotemark \footnotetext{ This is only valid in the time before any new spikes arrive. - To see that the 'summability' still holds after a new spike arrives, the above reasoning can be repeated, but simply with different values for the $G_i$ (all decayed by an amount $e^{-t_\text{spike}/\tau}$, and one increased by a bump $Δg_i$), and then redefining $t_\text{spike}$ to be $t = 0$. + But the `summability' still holds after a new spike. + To see this, the given reasoning can be repeated, but simply with different values for the $G_i$ (all decayed by an amount $e^{-t_\text{spike}/\tau}$, and one increased by a bump $Δg_i$), and then redefining $t_\text{spike}$ to be $t = 0$. } \begin{equation} \sum_i g_i(t) = \sum_i \left( G_i\ e^{-t/\tau} \right) diff --git a/thesis/figs/LambertW.png:Zone.Identifier b/thesis/figs/LambertW.png:Zone.Identifier deleted file mode 100644 index b57b5af2..00000000 --- a/thesis/figs/LambertW.png:Zone.Identifier +++ /dev/null @@ -1,4 +0,0 @@ -[ZoneTransfer] -ZoneId=3 -ReferrerUrl=https://upload.wikimedia.org/wikipedia/commons/thumb/b/b9/Mplwp_lambert_W_branches.svg/900px-Mplwp_lambert_W_branches.svg.png -HostUrl=https://upload.wikimedia.org/wikipedia/commons/thumb/b/b9/Mplwp_lambert_W_branches.svg/900px-Mplwp_lambert_W_branches.svg.png diff --git a/thesis/figs/dynsys.png b/thesis/figs/dynsys.png new file mode 100644 index 00000000..7e855868 Binary files /dev/null and b/thesis/figs/dynsys.png differ diff --git a/thesis/figs/exIF-fit.png b/thesis/figs/exIF-fit.png new file mode 100644 index 00000000..c858f27f Binary files /dev/null and b/thesis/figs/exIF-fit.png differ diff --git a/thesis/main.pdf b/thesis/main.pdf index 74e96b56..09010999 100644 Binary files a/thesis/main.pdf and b/thesis/main.pdf differ diff --git a/thesis/references.bib b/thesis/references.bib index 43ba8b90..b1c29a4f 100644 --- a/thesis/references.bib +++ b/thesis/references.bib @@ -1,3 +1,14 @@ +@article{Badel2008ExtractingNonlinearIntegrateandfire, + title = {Extracting Non-Linear Integrate-and-Fire Models from Experimental Data Using Dynamic {{I}}–{{V}} Curves}, + author = {Badel, Laurent and Lefort, Sandrine and Berger, Thomas K. and Petersen, Carl C. H. and Gerstner, Wulfram and Richardson, Magnus J. E.}, + date = {2008-11-15}, + journaltitle = {Biological Cybernetics}, + shortjournal = {Biol Cybern}, + doi = {10.1007/s00422-008-0259-4}, + url = {https://doi.org/10.1007/s00422-008-0259-4}, + langid = {english} +} + @book{Dayan2001TheoreticalNeuroscienceComputational, title = {Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems}, shorttitle = {Theoretical Neuroscience},