Physical visualisation of neural network learning using RGB leds, arduino and raspberry pi.
Switch branches/tags
Nothing to show
Clone or download
Latest commit d49c3d3 Mar 23, 2018

README.md

As featured on OpenSource.com and in HackSpace magazine!!😄😄😄

Issue 5 - April 2018 feature

RGB Neural Net

IOT physical neural network visualization

net learning

What & Why?

This projects goal was to build an intuitive and visually interesting way of seeing a neural network learn. I built it because I had some empty wall space.

A separate computer runs the neural network training program and communicates with the RGB Neural Net over WiFi.

Using the RGB Neural Net

First I generate some data that I want the neural network to learn. In this case it's some multi label classification data, meaning that given an 'x' and a 'y' value (2 continuous features) a data point can either have label A, label B, both label A and B, or no label assigned to it. i.e Label A could be 'if someone likes apples', and label B could be 'if someone like oranges', and therefore someone could like one, or the other, or both, or neither. There generated data looks like the following: net learning

Then in our program (python + sklearn) when create a neural network that is the same shape as the RGB one (1 hidden layer with 3 nodes).

nn = MLPClassifier(hidden_layer_sizes=(3), 
                   activation='logistic', 
                   learning_rate_init=0.02, 
                   max_iter=1, warm_start=True)

Then import the RGB library and create the connection to the RGB neural network:

from rgb_nn import RGB_NN
rgb = RGB_NN(server_loc='http://192.168.1.153:5000')

Finally just loop over the fitting of the code neural network along with the function to update the RGB neural network:

for i in range(50):
   nn.fit(X_train, y_train)
   rgb.display_weights(nn)

Then sit back and watch the learning happen: net learning

What the colours mean

The colours of the connections represents the value of the weights between nodes (RED - Low, BLUE - 0, GREEN - High). The colours of the nodes themselves represent the bias value. net key

Technical components

Hardware:

  • 3D printed nodes
  • Fibre optic tubing
  • RGB LEDs
  • Arduino Uno
  • Raspberry Pi Zero

Software:

  • RGB LED strip driver with serial com script on Arduino
  • Flask API server running on Raspberry Pi taking requests and send instruction to Arduino via serial
  • Python library that converts SciKit-Learn neural network model into the API requests for the Raspberry Pi

Its making

Installing Arduino to Raspberry Pi

sudo apt-get install arduino

Turning on 20th LED from Raspberry Pi

import serial
# create serial connection
arduino_serial_connection = serial.Serial('/dev/ttyACM0',9600)
# send serial message
arduino_serial_connection.write('10010010020\n')

Flask server API command

POST /change_leds sets the RGB values of multiple LEDs at once.

Json body:

{
    "leds":
    [
        { "red": 0, "green": 200, "blue": 100, "led_num": 1 },
        { "red": 0, "green": 50, "blue": 100, "led_num": 2 },
        { "red": 0, "green": 50, "blue": 100, "led_num": 3 },
        { "red": 0, "green": 50, "blue": 100, "led_num": 4 }
    ]
}

^ this probably should have been a single dict with the key being the led number... ¯\(ツ)

To Do

  • Design neuron to 3D print
  • Print nodes
    • Input layer
    • Hidden layer
    • Output layer
  • Build electrics for nodes
    • 1 node
    • 2 node
    • 3 node
    • 4 node
    • 5 node
    • 6 node
    • 7 node
  • Embeded electronics nodes
    • 1 node
    • 2 node
    • 3 node
    • 4 node
    • 5 node
    • 6 node
    • 7 node
  • Test wire up
  • Optimise Arduino code to update LEDs in batches
  • Optimise flask code to use new Arduino code
  • Design mounting solution for nodes
  • Mount nodes on board
    • 1 node
    • 2 node
    • 3 node
    • 4 node
    • 5 node
    • 6 node
    • 7 node
  • Optically connect nodes
    • 1 node
    • 2 node
    • 3 node
    • 4 node
    • 5 node
    • 6 node
    • 7 node
  • Create LED mapper (to map to neuron weights)
    • Re-factor server sending code into one document that:
    • Sets an led to a value (one number converts to RGB)

Optional

  • Get canvas
  • Mount on canvas

Resources

http://www.meccanismocomplesso.org/en/controlling-arduino-raspberry-pi/

https://forum.arduino.cc/index.php?topic=405287.0

https://github.com/FastLED/FastLED

Notes

Serial communication was becoming incredibly slow when testing sending a lot of requests. This turns out to be because I had the Arduino printing to the serial line whilst the Raspberry Pi was not reading it. This filled up the buffer of the Arduino and puts a 1 second delay on all further communication. https://arduino.stackexchange.com/questions/22816/serial-communication-dead-slow-after-a-while

When starting the server, the first led can become frozen at its initial colour, restarting the power to the strip will fix this.

Cooling: it seems that the little pi zero can get a bit flustered running as a server and thus either helping it with some cooling or upgrading to a beefier pi board is necessary.

Progress Images

Initial 3D design of nodes: 3D neuron design First and second interaction of input/output node: printed input node All nodes printed: network Soldered node lights: single node electrics Electronics fitting in node shell: single node electrics All electronics fitted in nodes all nodes electrics All nodes connected and lighting up all nodes light up Nodes mounted on boards all nodes electrics Nodes connected by fiber optic all nodes light up All nodes connected by fiber optic all nodes connected High and low colour weights how node weights look

Finally!!!

RGB network connected up and displaying weights in real time from a learning sklearn NN multi-label classifier model: final learning rgb nn