Skip to content

BackyardBrains/NeuroRobot

master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 

NeuroRobot app by Backyard Brains

A neurorobot is a robot controlled by a computer simulation of a biological brain. At Backyard Brains we use neurorobots to teach computational neuroscience in high schools. This repo contains all the Matlab and C/C++ code needed to run the NeuroRobot app, with or without a robot. The app allows you to design and simulate biologically-based neural networks connected to sensors (camera, microphone, distance sensor) and effectors (speaker, motors, lights).

Get Started

The following setup is recommended:

  • Matlab (iOS beta here), with the following toolboxes

    • Image Processing Toolbox (for vision)
    • Digital Signal Processing Toolbox (for hearing)
    • Image Acquisition Toolbox (for webcam-vision, this support package is also needed)
    • Audio Toolbox (for webcam-hearing)
    • Deep Learning Toolbox and Paralell Computing Toolbox (for object detection, AlexNet and GoogLeNet add-ons are also needed)
  • Comic Book font (find Comic_Book.ttf in the NeuroRobotToolbox folder; in Windows, right-click the file and choose ‘Install for All Users’; on Mac, double-click the file and select ‘Install Font’; restart Matlab)

  • Work with colorful objects in brightly lit rooms for optimal colorvision

To start the NeuroRobot app, open the file NeuroRobotToolbox/neurorobot.m in Matlab, adjust settings, then run the file. This launches the app’s Startup menu.

Please see the User Guide for more information about getting started with the NeuroRobot app in Matlab.

Design Brains

Many neurons in the NeuroRobot app are Izhikevich neurons designed to simulate the spiking patterns of biological neurons. These neurons can be quiet, fire regularly or in bursts, and can respond to inputs in different ways. Synaptic connections between neurons have a strength (“weight”) between -100 and 100 mV. Every time a sending (“presynaptic”) neuron fires a spike, the weight of the synapse is applied to the receiving (“postsynaptic”) neuron. To reliably trigger a spike in a neuron, the incomming synapse spike should have a strength of 25 mV or more. To test this, connect a highly active neuron to several quiet neurons, use a range of synaptic weights, and examine the effect on the postsynaptic neurons.

Synaptic connections can be plastic. This means that if a sending and a receiving neuron are active at the same time, a synapse connecting them will grow stronger. In other words, neurons that fire together, wire together. Some synapses are plastic only in the presence of a dopamine reward.

The NeuroRobot app collects camera and microphone data continuously. Within this data, it can detect simple features such as color and pitch, and complex data such as objects and words. To make a neuron respond to a sensory feature, select the orange square next to the relevant sensor, then select the target neuron. To make a neuron produce movement or sound, select the neuron first, then the orange square of the desired speaker or motor.

The basal ganglia allows vertebrate organisms to select particular actions in particular situations. Specifically, the basal ganglia disinhibits (i.e. activates) one group of neurons, associated with one specific behavior, at a time. Dopamine rewards make the currently selected basal ganglia neuron group and its associated behavior stay selected longer (higher "motivation") and increases their likelihood of being selected in similar situations in the future. Inputs to basal ganglia neurons strongly influence how long the currently selected neuronal group stays selected. Neurons belonging to a particular basal ganglia group are identified by the “ID” variable, by dashed lines emanating from "Striatal" neurons, and (optionally) by color.

Please refer to the Labs and lesson plans for more information about designing brains in the NeuroRobot app.

Troubleshooting

In the Startup menu, click Connect. The Connect button should turn green in 5-20 s. If the button turns red or Matlab crashes, you have a problem. Try restarting Matlab. If you have a robot, make sure the WiFi connection is intact. If you are using a computer with a webcamera, use the 'ver' command to confirm that you have the Image Acquisition Toolbox installed, use the Matlab Add-on Explorer to check that the Support Package for OS Generic Video Interface is installed, and confirm that the ‘rak_cam = videoinput('winvideo', 1); preview(rak_cam)’ command is able to display video from your webcamera.

Further Reading

Getting Started
User Guide
Lesson Plans
2020 paper
2021 preprint

View SpikerBot Toolbox on File Exchange

About

A neurorobot is a robot controlled by a computer simulation of a biological brain. At Backyard Brains we use neurorobots to teach computational neuroscience in high school.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published