Bio-inspired spike-based Hippocampus and Posterior Parietal Cortex robotic system for environment pseudo-mapping and navigation
Code on which the paper entitled "Bio-inspired spike-based Hippocampus and Posterior Parietal Cortex robotic system for environment pseudo-mapping and navigation" is based, published in Advanced Intelligent Systems journal.
A functional robotic system bio-inspired on Hippocampus and Posterior Parietal Cortex cerebral areas for environment pseudo-mapping and navigation implemented on the SpiNNaker hardware platform using the technology of the Spiking Neuronal Network (SNN) is presented. The code is written in Python and makes use of the PyNN library and their adaptation for SpiNNaker called sPyNNaker. Not only software tests of the system were carried out, but also a demo on a robotic platform based on a 2-wheeled car. In addition, the necessary scripts to replicate the tests, robotic demo and plots carried out in the paper are included, together with data and plots of these tests and demo.
Please go to section cite this work to learn how to properly reference the works cited here.
Title: Bio-inspired spike-based Hippocampus and Posterior Parietal Cortex robotic system for environment pseudo-mapping and navigation
Abstract: The brain has a great capacity for computation, adaptability and efficient resolution of complex problems, far surpassing modern computers. In this regard, neuromorphic engineering seeks to mimic the basic principles of the brain to develop systems capable of achieving such capabilities. In the neuromorphic field, navigation systems are of great interest due to their potential applicability to robotics, although this systems are still a challenge to be solved. Navigation is possible due to the activity of differrent regions, such as the hippocampus and the Posterior Parietal Cortex. In this work, we propose a spike-based robotic navigation and environment pseudomapping system. This system is formed by a bio-inspired hippocampal memory model connected to a bio-inspired PPC model. The hippocampal memory is in charge of maintaining a representation of a state map of the environment, and the PPC is in charge of local decision-making. This system was implemented on the SpiNNaker hardware platform using Spiking Neural Networks. A set of real-time experiments was applied to demonstrate the correct functioning of the system components and the complete system in virtual and physical environments on a robotic platform. The system is able to navigate through the environment to reach a goal position starting from an initial position, avoiding obstacles and mapping the environment. To the best of the authors knowledge we propose the first implementation of an environment pseudo-mapping system with dynamic learning based on a bio-inspired spiking hippocampal memory.
Keywords: spatial navigation, environment state map, Spiking Neural Networks, Hippocampus, Posterior Parietal Cortex, Neuromorphic engineering, SpiNNaker
Author: Daniel Casanueva-Morato
Contact: dcasanueva@us.es
- Have or have access to the SpiNNaker hardware platform. In case of local use, follow the installation instructions available on the official website
- Python version 3.8.10
- Python libraries:
- sPyNNaker
- numpy 1.21.4
- matplotlib 3.5.0
- sPyMem: 0.0.4
To run any script, follow the python nomenclature: python script.py
posterior_parietal_cortex.py: class that is responsible for the construction of the PPC module. This module has comparison inputs to which it reacts to generate output and the start-of-operation input that indicates when the hippocampal read cycle begins. It contains the necessary code so that when executed on its own, it generates the example plot of operation of the module itself.
real_time_map_and_nav_app.py: script in charge of generating the navigation and pseudomapping system. Instantiates the previously existing hippocampus and PPC modules, generating the necessary connections to develop the complete system. The inputs and outputs of the system are defined for direct real-time operation, so both the software simulations and the demo use the same code.
memory_sweep.py: a set of functions that calculate the input to be given to the system in order to perform a memory sweep to reconstruct the initial and/or final state of the grid map.
plot.py: functions needed to generate the plots used to understand the correct functioning of the system and in the article.
results folder: where the data and plots of the different simulations of the whole system are stored, as well as a demonstration of the operation of the PPC (posterior parietal cortex module) separately. These files contain the initial and final map in both a compact format for use in the code and in a more "user-friendly" and visual format for easy viewing by the user. They also contain the weights of the memory synapses after the simulation, i.e. the contents of the memory with the pseudomapping of the environment. Finally, it includes other files with the activity at the spike level of the memory and the PPC for the subsequent reconstruction of the desired plots.
robot folder: code of the robotic system in order to carry out the demo. The robotic system is divided into two boards and therefore two codes:
robot_control.ino: is responsible for motor control and collection of ultrasound measurements of the environment under explicit commands.
robot_wifi_redirect_udp.ino: in charge of the wifi communication between the robot and SpiNNaker. To do this, it generates a wifi network to which the system controlling SpiNNaker must connect and redirects the communication in both directions. Communication to the robot is via input and output pins, while communication to the system with SpiNNaker is direct via the serial port.
In order to use the system, you only need to define a set of global variables that define the scenario on which the system will operate and run the script real_time_map_and_nav_app.py. This script is in charge of instantiating the complete system with the configuration indicated through the variables defined above. Each simulation or demo must be identified by a unique id. The id of the experiment to be reproduced must be indicated at the beginning of the script. It is this number that will be used to distinguish within an if which set of global variables is used in the next execution of the system. These global variables are:
xlength and ylength: size of the grid map as a function of the number of rows (y-axis) and columns (x-axis).
xinit, yinit, xend and yend: x and y coordinates of the initial and final positions of the robot within the grid map.
experimentName: name of the experiment, used as the base for the name of the generated folders and files.
simTime: duration of the simulation to be performed in miliseconds, in case of real time demo, use a very long duration.
obstacles: the position within the grid map where the obstacles are located in the case of the software simulations or an empty list in the case of the demo.
robotDirection: initial direction in which the robot is facing from a top-down perspective (0 = top, 1 = left, 2 = bottom, 4 = right).
maxMoveTime: variable indicating the maximum time needed for the robot to move from one cell to another, taking into account turning times. It is only used for the real time demo.
There are other global variables that can be set, following the declaration of the experiment, however, it is recommended to leave them as default.
In the case of a software simulation, nothing more would need to be done, however, in the case of a demo on the robot, it would be necessary to have the robotic platform ready to send and receive the indicated commands. To reproduce the demo presented in the paper, a Romeo BLE board was used to control the hardware system and an Adafruit ESP32 Feather board for wifi communication. In case you want your own robotic platforms, you will have to adapt the code from the repository available in the robot folder.
In both cases, after the end of the experiment, the relevant data from the simulation will be returned as a set of files under the folder with the name of the experiment. These files can then be used in the script called plot.py to generate the desired plots.
APA: Casanueva-Morato, D., Ayuso-Martinez, A., Dominguez-Morales, J.P., Jimenez-Fernandez, A., Jimenez-Moreno, G. and Perez-Peña, F. (2023), Bioinspired Spike-Based Hippocampus and Posterior Parietal Cortex Models for Robot Navigation and Environment Pseudomapping. Adv. Intell. Syst. 2300132. https://doi.org/10.1002/aisy.202300132
BibTeX: @article{https://doi.org/10.1002/aisy.202300132, author = {Casanueva-Morato, Daniel and Ayuso-Martinez, Alvaro and Dominguez-Morales, Juan P. and Jimenez-Fernandez, Angel and Jimenez-Moreno, Gabriel and Perez-Peña, Fernando}, title = {Bioinspired Spike-Based Hippocampus and Posterior Parietal Cortex Models for Robot Navigation and Environment Pseudomapping}, journal = {Advanced Intelligent Systems}, pages = {2300132}, doi = {https://doi.org/10.1002/aisy.202300132} }
The author of the original idea is Daniel Casanueva-Morato while working on a research project of the RTC Group.
This research was partially supported by the CHIST-ERA H2020 grant SMALL (CHIST-ERA-18-ACAI-004, PCI2019-111841-2/AEI/10.13039/501100011033 and by project PID2019-105556GB-C33 funded by MCIN/ AEI /10.13039/501100011033.
D. C.-M. was supported by a "Formación de Profesor Universitario" Scholarship from the Spanish Ministry of Education, Culture and Sport.
This project is licensed under the GPL License - see the LICENSE.md file for details.
Copyright © 2022 Daniel Casanueva-Morato
dcasanueva@us.es
