Neurons

Neurons are the basic unit of the neural network. They can be connected together, or used to gate connections between other neurons. A Neuron can perform basically 4 operations: project connections, gate connections, activate and propagate.

project

A neuron can project a connection to another neuron (i.e. connect neuron A with neuron B). Here is how it's done:

var A = new Neuron();
var B = new Neuron();
A.project(B); // A now projects a connection to B

Neurons can also self-connect:

A.project(A);

The method project returns a Connection object, that can be gated by another neuron.

gate

A neuron can gate a connection between two neurons, or a neuron's self-connection. This allows you to create second order neural network architectures.

var A = new Neuron();
var B = new Neuron();
var connection = A.project(B);

var C = new Neuron();
C.gate(connection); // now C gates the connection between A and B

activate

When a neuron activates, it computes its state from all its input connections and squashes it using its activation function, and returns the output (activation). You can provide the activation as a parameter (useful for neurons in the input layer. It has to be a float between 0 and 1). For example:

var A = new Neuron();
var B = new Neuron();
A.project(B);

A.activate(0.5); // 0.5
B.activate(); // 0.3244554645

propagate

After an activation, you can teach the neuron what should have been the correct output (a.k.a. train). This is done by backpropagating the error. To use the propagate method you have to provide a learning rate, and a target value (float between 0 and 1).

For example, this is how you can train neuron B to activate 0 when neuron A activates 1:

var A = new Neuron();
var B = new Neuron();
A.project(B);

var learningRate = .3;

for(var i = 0; i < 20000; i++)
{
	// when A activates 1
	A.activate(1);
	
	// train B to activate 0
	B.activate();
	B.propagate(learningRate, 0); 
}

// test it
A.activate(1);
B.activate(); // 0.006540565760853365

Squashing function and bias

By default, a neuron uses a Logistic Sigmoid as its squashing/activation function, and a random bias. You can change those properties the following way:

var A = new Neuron();
A.squash = Neuron.squash.TANH;
A.bias = 1;

There are 5 built-in squashing functions, but you can also create your own:

• Neuron.squash.LOGISTIC
• Neuron.squash.TANH
• Neuron.squash.IDENTITY
• Neuron.squash.HLIM
• Neuron.squash.ReLU

See more squashing functions here.