Neural Network from Scratch in C++

This is a basic feedforward neural net which i implemented from scratch. I dont have a strong background in machine learning, and i mostly know C++. I don’t like using python that much, so I wanted to see if I could try something different and understand the core ideas of ML using just C++. This is more of a learning project for me to explore the math and logic behind neural networks.

I’m happy to connect with anyone who’s learning ML or wants to collaborate on similar learning projects.

PS: Math is not my stronghold, so yeah took me quiet a time to figure things out before I actually implemented it.

How I approached it

1. Matrix Operations

The first step was to implement a basic Matrix class. I created 2D matrices with added support for:

• matrix initialization with zero or random values
• matrix addition and subtraction
• dot product
• transpose
• element-wise multiplication
• applying functions element wise (used for sigmoid)

I referred to:

• The Coding Train - Neural Networks in JavaScript: for how matrix operations are used in forward and backpropagation
• GeeksforGeeks C++ STL vector documentation : for handling dynamic 2D vectors

2. Neural Network Design

After the Matrix class was working, I implemented a neural network with:

• 2 input neurons
• 1 hidden layer (configurable number of neurons)
• 1 output neuron
• Sigmoid activation function in both layers
• Gradient descent-based learning using backpropagation

I implemented the network as a class that stores weights and biases as matrices. For each training step, I used the standard backpropagation formulas to update weights.

References used:

• 3Blue1Brown - What is backpropagation really doing?
• How To Build a Feedforward Neural Network In Python
• CS231n - Neural Networks Part 1

3. Dataset and Training

I trained the model on a basic XOR truth table:

X,Y,Z
0,0,0
0,1,1
1,0,1
1,1,0
...

This dataset is a binary classification problem. The network is expected to output values close to 0 or 1 after sigmoid activation.

4. Logging Loss

For each 100 epochs, I calculated the average absolute loss and logged it to a CSV file at logs/training_loss.csv. This helped me visualize how the model was learning over time.

5. Visualizing the Learning Curve

I wrote a python script (plot_loss.py) to visualize the loss. The curve below shows the training loss decreasing over time, which means the network is learning to predict the XOR outputs correctly.

---

Folder Structure

• include/Matrix.hpp – Header file for the matrix operations
• include/NeuralNetwork.hpp – Header file for neural network
• src/Matrix.cpp – Matrix implementation
• src/NeuralNetwork.cpp – Forward and backpropagation logic
• main.cpp – Entry point, CSV loading, training loop
• data/xor.csv – XOR dataset
• logs/training_loss.csv – Loss values per 100 epochs
• logs/loss_curve.png – Plot of loss over training
• plot_loss.py – Python script to generate the graph

---

how to run?

compiling code

g++ -I./include main.cpp src/Matrix.cpp src/NeuralNetwork.cpp -o xor_nn
xor_nn

This reads the XOR dataset and trains the neural network.

plot the training loss (just for visual rep)

pip install matplotlib pandas
python3 plot_loss.py

This creates logs/loss_curve.png, which helps visualize training progress.

---

Conclusion

This project helped me understand how neural networks actually work at a basic level. It’s not optimized or production-grade, but it gave me a strong intuition for the math behind backpropagation and how predictions are learned.

I would be happy to collaborate or learn more if anyone is working on ML projects and wants to guide or team up :)