Is this the correct way to implement a hopfield network?

[raimannma]

carrot/src/architecture/architect.js

Lines 466 to 498 in bf9ae22

	/**
	* Creates a hopfield network of the given size
	*
	* @param {number} size Number of inputs and outputs (which is the same number)
	*
	* @example <caption>Output will always be binary due to `Activation.STEP` function.</caption>
	* let { architect } = require("@liquid-carrot/carrot");
	*
	* var network = architect.Hopfield(10);
	* var training_set = [
	* { input: [0, 1, 0, 1, 0, 1, 0, 1, 0, 1], output: [0, 1, 0, 1, 0, 1, 0, 1, 0, 1] },
	* { input: [1, 1, 1, 1, 1, 0, 0, 0, 0, 0], output: [1, 1, 1, 1, 1, 0, 0, 0, 0, 0] }
	* ];
	*
	* network.train(training_set);
	*
	* network.activate([0,1,0,1,0,1,0,1,1,1]); // [0, 1, 0, 1, 0, 1, 0, 1, 0, 1]
	* network.activate([1,1,1,1,1,0,0,1,0,0]); // [1, 1, 1, 1, 1, 0, 0, 0, 0, 0]
	*
	* @return {Network}
	*/
	Hopfield: function(size) {
	const input = new Group(size, 'input');
	const output = new Group(size, 'output');
	input.connect(output, methods.connection.ALL_TO_ALL);
	output.set({
	squash: methods.activation.STEP,
	});
	return new architect.Construct([input, output]);
	},

If this is correct than a hopfield network is just a simple perceptron but with STEP activation function.

I found this pics of hopfield networks:

They seem like there is more ongoing then just a dense layer with step activation, or am I wrong?

Also Wikipedia says: "A Hopfield network is a form of recurrent artificial neural network" (Wikipedia). But there are only feedforward components.

[christianechevarria]

You're absolutely right about it being more complex than just a dense feed-forward network. To my understanding in a hopfield network all nodes connect to all other nodes so implementation-wise this depends on whether the methods.connection.ALL_TO_ALL connection method is implemented correctly inside of Group.connect()

[raimannma]

ALL_TO_ALL means

input0 -> output0
input0 -> output1
input1 -> output0
input1 -> output1

but there should be backward pointing connections right ?

Like this?

input0 -> output0
input0 -> output1
input1 -> output0
input1 -> output1
output0 -->input0
output0 -->input1
output1 -->input0
output1 -->input1

[christianechevarria]

but there should be backward pointing connections right ?

Like this?

input0 -> output0
input0 -> output1
input1 -> output0
input1 -> output1
output0 -->input0
output0 -->input1
output1 -->input0
output1 -->input1

Yes exactly, we could create a new connection method that does this. Also, ALL_TO_ALL is a bit misleading in this context I wonder if we should rename it

[raimannma]

Ok, consider renaming it to ALL_TO_ALL_FORWARD ?

And we need no additional connection type we can just do:

input.connect(output, methods.connection.ALL_TO_ALL);
output.connect(input, methods.connection.ALL_TO_ALL);

[christianechevarria]

That's true and much more elegant 👍 renaming sounds good also

[christianechevarria]

We should also have some basic structural tests for architecture to avoid things like this slipping through