Neural-Network-Cpp

Artificial Neural Network implemented in c++ language based on MNIST database

Table of contents

• General Info
• Assumptions
• External specification - setup
• Internal specification
• Construction specification
• Files tree
• Code description
• Neural network structure
• How does it work?
• Sources for acquiring knowledge

General Information

A neural network is a series of algorithms that endeavors to recognize underlying relationships in a set of data through a process that mimics the way the human brain operates.
In this sense, neural networks refer to systems of neurons, either organic or artificial in nature.
Almost all commercial machine learning applications depend on artificial neural networks, which are trained using large datasets with a back-propagation algorithm.
The network first analyzes a training example, typically assigning it to a classification bin. This result is compared to the known “correct” answer, and the difference between the two is used to adjust the weights applied to the network nodes.
The process repeats for as many training examples as needed to (hopefully) converge to a stable set of weights that gives acceptable accuracy.
This standard algorithm requires two distinct computational paths — a forward “inference” path to analyze the data, and a backward “gradient descent” path to correct node weights

Assumptions

The program gets the data from the files of MNIST database on the basis of which it can perform the process of training the neural network - or it’s testing.

Setup

My script implementation is command line program. For the puropse of preparing my solution, to build (compile) the program i used the ’make’ tool, so it is incipiently included in the project. Therefore, to generate the ’training’ and ’testing’ executable files you must use the terminal command:

$ make

Generated executables do not require any additional arguments. After generating them, in order to start the process of training the neural network, you should run the:

$ ./training

Or in case of testing trained neural network:

$ ./testing

The program does not have any in-terminal short manual included - no such option was needed.

Internal Specification

The program is not implemented with structural paradigm.
Script does not contain a specific user interface, therefore such an application was not needed.
Imperative paradigm was only used to separate the partially separated testing and training scripts.

Construction specification

The compiler used during creating the project: g++ (GCC) 12.1.0

language: c++

language standard: 17

Files Tree

File test.cpp - code of testing a neural network process
File train.cpp - code of training a neural network process
File test - generated by compiling executable to perform testing process
File train - generated by compiling executable to perform training process
File neural-network-model.dat - file containing the weights of neural network
Folder /reports/: - folder containing reports of program results
File /reports/testing-report.dat - report file saving the results of testing
File /reports/training-report.dat - report file saving the results of training
Folder /mnist/: - folder containing MNIST database files

Code description

Description of types and functions included in the target files in comments.

Neural network structure

How does it work?

Training:

The program begins its operation by opening and getting data from files: report of any previous learning process and MNIST database - gray scale image and the corresponding label.
The neural network is initialized starting with the memory allocation for its layers. Then its weights are initialized.
The program begins its neural network training based on backward propagation algorithm - calculates the gradient of the error function with respect to the neural network’s weights.
Then, the error threshold for a given iteration of the neural network training process is calculated.
In a loop, the program displays the current image from the database, the number of iteration being performed, the number of iterations of the training process for a given image and the error threshold.
After the process is completed, the program saves the current network - its weights in the model-neural-network.dat file.
The program also saves a report on all individual iterations of the learning process into training-report.dat

Testing:

Identically, the program begins its operation by opening and getting data from the database and the report.
The memory allocation is initialized and the trained neural network model is read (weight matrices).
The network goes through a classification process - perceptron procedure and the prediction.
The screen lists the classification resault and the error threshold.
Aftercompleting the number of iterations of testing indicated in the program, it prints the number of correct samples compared to the number of iterations performed and the overall accuracy.

Sources for acquiring knowledge

online

• AI wiki
• 3Blue1Brown But what is neural network?
• 3Blue1Brown Gradient descent, how neural networks learn
• 3Blue1Brown What is backpropagation really doing?

books

• John Slavio Deep Learning and Artificial Intelligence: A Beginners’ Guide to Neural Networks and Deep Learning
• Jerzy Grębosz Opus Magnum c++