Backpropagation

Backpropagation

Backpropagation computes how much each weight and bias contributed to the loss.

The goal is to calculate gradients and update parameters.

Output Layer Delta

For an output neuron:

delta = dL/dh * f'(z)

Where:

• dL/dh is the derivative of the loss with respect to the output.
• f'(z) is the derivative of the activation function.

Hidden Layer Delta

For a hidden neuron:

delta = f'(z) * sum(w_j * delta_j)

Where:

• w_j is a weight from the hidden neuron to the next layer.
• delta_j is the delta of a neuron in the next layer.

Weight Gradient

The gradient for a weight is:

dL/dw = h_previous * delta_next

Where:

• h_previous is the activation from the previous neuron.
• delta_next is the delta of the target neuron.

Parameter Update

Weights are updated using gradient descent:

w = w - learning_rate * gradient

Biases are updated similarly:

b = b - learning_rate * delta

What the App Shows

The app can visualize:

• Delta values.
• Propagated gradient terms.
• Connection gradients.
• Parameter changes across epochs.