A repo where I do my learning stuffs here

Day1: Artificial Neural Network

1.1 Adaline Learning Unipolar and bipolar case

-> These help in solving only the linearly separable problems such as OR, AND, NAND, NOR gates

-> For non-linearly separable problems such as XOR and XNOR they can't predict well.

-> The activator function here will be: if value > 0.5(for unipolar) return 1 else return 0

bias_new = bias_old + learning_rate × error

weights_new = weight_old + learning_rate × error × inputs

error = t - predicted_value

predicted_value = Σ (weights × inputs) + bias

1.2 Backpropagation

-> This will solve the non-linearity problem by introducing non-linearity through the hidden neurons.

-> Say for the two input XOR gate it will use two hidden neurons to create non-linearity which will aid in solving the problem.

-> This learning uses the activator function such as Sigmoid function. i.e. 1/(1+e^-x)

Forward Pass

hidden_layer_input = Σ(inputs × weights) + bias_input_hidden
hidden_layer_output = Sigmoid(hidden_layer_input)
final_layer_input = Σ(hidden_layer_output × weights) + bias_hidden_output
final_layer_output = Sigmoid(final_layer_input)

Back propagation

δ_k = (t - y_k) × Sigmoid_derivative(y_ink)
δ_j = δy_inj × Sigmoid_derivative(Z_inj)
δy_inj = Σ(δ_k× w_jk) where δ_k = delta output error.

Updating Weights

Δw_ij = α × δ_j × x_i
Δw_jk = α × δ_k × z_j
w_new = w_old + Δw