-
Notifications
You must be signed in to change notification settings - Fork 1
Application of Kalman Filter in Neural Signal Decoding
Hongchang Kuang
Word count: 2610 (including Latex formula and HTML code)
Have you imagined a world that we could control things without moving our fingers? Will you fancy playing video games or driving your cars just with your brain? I have to tell you that this fantasy is no longer impossile with the help of Brain Machine Interface (BMI). In recent decades, as technology advancing and social ethics system maturing, a heated debate has been triggered over the usage of BMI. Some argues that current innovations are still not applicable for widely marketing products. However, most engineers and neuroscientists are excited by technological progress that facilitates brain mapping, the most sanguine of them comparing their growing ability to tremendous advances that led to the unimaginable success of the BMI projects. Companies like NeuralLink and Paradromics, as well as government agency have invested on the projects to test the possibilities and work on the real world applications [1]. This articles briefly introduces one of the prevailing technology in BMI: the application of Kalman filter in neural signal decoding, discussing its advantages and potential improvements, and what makes it robust and powerful that could possibly allows BMI to enhance our daily life.
Before we get to the real technique, a simple background in neuroscience is neccessary to unravel you from the potential confusion in the world of jargons, since the mind, with our brain in general, is our main subject of research. The main study theme of neuroscience is to link molecules to mind. Human brain is a highly integrated network of approximately 100 billion individual nerve cells. Therefore, to fully understand the mechanism behind the BMI devices, a brief understanding of how neurons are organized into signaling pathways and how they communicated is necessary. We will introduce how brain functions from three different levels: brain, neuron, and neural signal (action potential). Further readings in related areas are also encouraged.
What does our brain look like? Surprisingly, it is complex and ordered at the same time. The central nervous system (CNS) is bilateral and symmetrical. Studies on brain with modern imaging techniques discovers that different regions on brain are specialized for different functions. As you may have gussed, for BMI to decode human cognitive information, the most important part that we would like to focus on is the cerebral cortex, where brain operations that are responsible for human cognitive abilities occur. As shown in the graph, It consists of four anatomically distinct lobes with different functionalities, including planning future actions, hearing, learning, vision, etc. With the concept of functional engineering prevailing in neuroengineering, it is natural to focus on a specific area on cerebral cortex to gather information of neuron activity. Therefore, to achieve fancy operations like controlling the cursor with our mind, for example, we will mainly focus on motor cortex in frontal lobe, which is responsible for planning future actions and the control of movement.
Source: Principles of Neural Science, Fourth Edition. Eric R. Kandel, James H. Schwartz, Thomas M. Jessell [2]
What is the basic unit of our brain and what makes it complex? We have to give credit to these simple but powerful biological creatures: nerve cells (neurons). Just like VLSI transistors, even though they have relatively basic morphology and architecture, approximately 10^11 neurons in the brain can support long-ranged and intricate anatomical circuits, where the "complexity" arises from. Neuron have four regions: cell body (soma), dendrites, axon, and presynaptic terminals. The main component axon is what we should pay special attention to: it conveys signals (action potential) to other neurons in long distance (0.1mm - 3m), while ensure the signals propagate without distortion or failure and preserving its shape at very high speed (1 - 100m/s). Compared to transmission line in our real life, the efficiency and accuracy of neural signal transmission are mind-blowing. Two neurons can also communicate at the synapse chemically using neurotransmitters. Neurons, whose structure diagram is attached below, are the most powerful building blocks of the most complicared circuits in the world.
Source: Principles of Neural Science, Fourth Edition. Eric R. Kandel, James H. Schwartz, Thomas M. Jessell [2]
Is the neural signal similar to the digital signal used for communication in the real life? Actually, they are nearly identical. Action potentials are the signals by which brain receives, analyzes and conveys information. Even though they are highly stereotyped just like digital signal (0 and 1), they are able to convey all the information that the brain may need, including vision, audition, emotion, etc. However, unlike the 0s and 1s in digital signals, since they do not have distinguishable shapes, all the information is preserved in the path that the signal travels and pattern of action potentials. The maximum magnitude of action potential can be up to 40 mV, and their frequency is about 500 Hz. On the other hand, it is also their highly stereotyped shape makes decoding them feasible. Just like displacement and velocity, in neural signal decoding, we are more interested in the "velocity" of action potential: firing rate, which is the number of spikes of action potentials per unit time. With this concept, we are able to have a criterion of how active a neuron is.
Source: Principles of Neural Science, Fourth Edition. Eric R. Kandel, James H. Schwartz, Thomas M. Jessell [2]
With all the knowledge of neuroscience infused in your barin, I think your neurons should be ready for the real methods of decoding the neural data. Sit back, and I will walk you through this wonderful and inspiring process.
This architecture of our decoder is simple but elegant. As shown in the graph, the whole BMI system consists of multiple components, including signal detection, decode algorithm, and feedback. In this section, we will focus on continuous decoding of neural signal in hand movement (motor prosthesis). Image that you would like to grab the water bottle on the table. Your brain will go over two stages: planning and movement. In the planning stage, it need to plan the according trajectories precisely. In the following movement activity, it also needs to control the acceleration and speed of your arm. In general, the movement activity follows the plan activity, and is specially tuned for direction and speed of arm movement. As a result, we will be mainly concerned with the neural movement activity and the corresponding kinematics like hand position, velocity, acceleration, etc.
Source: Slides from EC ENGR C143A "Neural Signal Processing" Lecture 14 by professor Jonathan Kao [3]
We need to know what are we going to decode at first and what the experiment looks like. Let us set up the following experimental context: a monkey is asked to perform a task. It needs to move its hand to the cursor on the screen. When the cursor appears, it cannot move. As soon as it disappear, the monkey should move its hand to the destination. We call this whole process a reach.
Apparently, for the reaching task, we have two sources of data to consider:
- Observed kinematics (dimensionality
$M$ ):
- Neural activity (dimensionality
$N$ )
We will further denote the decoded kinematics as
where K is the total number of data points. If the time interval between two points
is
The general idea of the decoding operation is straightforward: decode th kinetics
information from neural activities, which means to infer
You might think this sounds easy. One simple method is that, if we could assume that
their relation is linear, we can simply fit a line to them using least square
since we have both observed data for
Basically is to minimize:
With our knowledge in linear algebra, we get:
Now we have this
Problem solved! This is what we called Optimal Linear Estimator (OLE). Is this
the final method that we should concern? Of course not! Unfortunately, as you may
guess, its accuracy and robustness are not very ideal. Obviously, we have ignore
something in this case. First, for the adjacent positions, say
Well, the first problem seems easy to solve: we can simply take history into account by doing something like:
Rewrite it in matrix form:
We can find
Seems more advanced, doesn't it? This technique is called Wiener Filter. We enhance the OPE by considering data from the history. But what about the second problem? it's finally time to welcome Kalman Filter to the stage.
The main concern for now is, we have additional information we have not used in our decoding process: the law of physics, since we understand the the positions of the arm are the integration of its velocity (we will be in real trouble if they are not). The natural way to incorporate this is to use Kalman Filter.
If we add one more equation to the original OLE, we can get:
The first equation is called state update process or dynamic process, where
In real life, we have to add some random noise to them:
We can simply assume that they are gaussian noise. Zero mean and covariance matrix
So how do we get the optimal estimator of
The spoiler is, we will reach to some kind of equation like this:
Apparently, we take all factors into account: history, law of physics, and the
neural signal. All of them have some information of what the newly decoded data
should be. So the goal is to find two matrices
What is Kalman Filter actually doing? In one dimension, we need to find an estimator
In other words, this means "we want to find the expected value of hand position at time
In general, we would write the distribution of
-
Initialize:
$\Sigma_{0|0} = 0$ -
Estimation updated process: Recursively calculate, until convergence of
$K_k$ :
We called the converged value
After some more magic, we can calculate
What about
This means that, with the parameters set
We can do this by taking the derivative of logarithm of
We also need to make sure that
Source: EC ENGR C143A Spring 2023 HW6Q4 by Hongchang Kuang [4]
I hope this method in neural signal processing is inspiring and fantastic. This article introduces the motivation of choosing Kalman Filter in decoding neural signal, and the basic mathematical recursion in arriving at the final answer. In general, when it comes to decode continous data, Kalman Filter is a powerful technique which incorporates factors like history, law of physics, and the characteristics of decoded data into the decoding process. It presents ideal accuracy and robustness. In our project, for example, we can choose to use Kalman Filter in smoothing the data collected from the IMU. Nevertheless, I cannot guarantee that Kalman Filter will be a panacea to all your problems: it also has some drawbacks. Kalman Filter can easily experience overshoot since it may weight too heavily on the history [3]. Some potential improvements in neural signal processing include rotating the velocity toward the destination (take angles into consideration) or use Closed Loop Decoder Adaptation (CLDA). In modern decoder with more advanced features, Deep Neural Network is also widely used. In the near future, we can expect that neural signal decoder would provide more advanced functionality like speech decoding, wating for you to explore further [5].
[1] AI's Next Frontier: Are Brain-Computer Interfaces The Future Of Communication? Bernard Marr, Forbes. https://www.forbes.com/sites/bernardmarr/2023/08/11/ais-next-frontier-are-brain-computer-interfaces-the-future-of-communication/?sh=61d3d2b851d9
[2] Principles of Neural Science, Fourth Edition. Eric R. Kandel, James H. Schwartz, Thomas M. Jessell
[3] Slides from EC ENGR C143A "Neural Signal Processing" by professor Jonathan Kao
[4] Homework answers from EC ENGR C143A "Neural Signal Processing" by Hongchang Kuang
[5] SCIENTISTS SAY NEW BRAIN-COMPUTER INTERFACE LETS USERS TRANSMIT 62 WORDS PER MINUTE. Victor Tangermann, Futurism. https://futurism.com/neoscope/scientists-new-brain-computer-interface-type-62-words-per-minute