NeuralNetwork.java

Introduction

The Neural Network originally refers to a circuit of neurons that are a part of the human nervous system. The Neural Network referred to here, borrows its ideas from the biological network and is essentially a system of interconnected neurons (or nodes) by edges having a weight component.

Neural Networks have three different layers of nodes:

1. Input layer: This layer contains the input nodes each of which refer to the individual attribute value of the input data set. Hence there are as many input nodes as there are individual attribute values for the dataset.

2. Hidden layers: These are the layer of nodes that lie between the input and the output layers. An NN-learner can have any number of hidden layers (see Note), but it has been proven that single hidden layer NN are universal approximators.

3. Output layer: This layer contains the output nodes, one each for the output class labels. You would expect an ideal NN to output 1 for the node that corresponds to the actual output and 0 for the other nodes.

Implementation Details

Node Definition

Essentially, each node in the decision tree contains the following details:

1. Inputs from the previous layer nodes: Except for the input layer nodes, each node is fully connected with nodes from the immediate preceding layer nodes.
2. Sum of the inputs: Each node is able to receive a value from the previous layer’s nodes output multiplied by the weight of the connecting edge. The node is capable of adding all these values to get a sum.
3. Activation function: Each node has an activation function that converts the sum to an output value. This library uses the sigmoid function as it can translate any value to a range over (0,1)
4. Output value: Each node produces an output value after applying the activation function on the sum.
5. Node attribute value: This is a feature only for input and output layer nodes. Essentially it records which input attribute value or output class value is being represented by that node.

Backpropagation

The Neural Network learns by a method called Backpropagation. More details about this learning method can be found at Wikipedia or any standard machine learning textbook.

Handling Continuous Variables

Certain attributes in the provided datasets can be continuous variables. The library handles continuous attributes by discretizing them into “n”-equal sized bins, depending on the actual continuous attribute value.

The user has an option to specify “n”. For instance consider the list 1.0, 1.2, 1.3, 1.5 and n=2. In this case the attribute value list will be transformed to a, a, b, b where a and b are the two discretized attribute values. So there are two bins here each of size 2.

Note: Discrete values mean “discrete independent values”. So even though (by actual ordering of the original list) a < b, it is assumed that a and b are two discrete independent values.

Handling missing attribute values

In case of missing attribute values; the library employs the following techniques depending on if the attribute is continuous or discrete.

1. If the attribute is continuous: The missing value is assigned the mean value of all the non-missing values of that attribute. So essentially if you have values of “1.0, 1.5, 2.0, ?” for four test instances of a particular attribute – the “?” is replaced with the average of 1.0, 1.5 and 2.0, i.e. 1.5.

2. If the attribute is discrete: The missing value is assigned the most commonly occurring attribute value for all the non-missing values of that attribute. So essentially if you have values of “c, a, b, b, b, ?” the missing value is replaced with “b”.

Functions Summary

Constructor Summary

public NeuralNetwork (String fName, int numAttributes, int testCases, String delimiter, int limitSplits) throws IOException, FileNotFoundException

Creates a new NeuralNetwork instance with the given parameters. The actual Neural Network structure is not created here.

Parameters
      -fName: File containing the training data.
      -numAttributes: number of attributes in the training data (not counting the output)
      -testCases: number of test cases to be considered from the training file. 
      -delimiter: delimiter between the attributes in a test case.
      -limitSplits: number of splits for continuous valued attributes 

Throws
      -IOException: Failed or interrupted while reading from file. 
      -FileNotFoundException: File fname is not found. 

Method Summary

public void createNeuralNetwork (int numLevels, int[] numNodesArray) (see Note)

Creates the Neural Network with the given parameters. 

Parameters
      -numLevels: Number of Hidden levels. 
      -numNodesArray: Array containing number of nodes for each hidden level. 
      
Throws
      None

public void trainNeuralNetwork (double learnRate, double errorLimit)

Trains the constructed network by backpropagation learning.

Parameters
     -learnRate: Learning rate for backpropagation.  
     -errorLimit: Specifies an error limit. If the learner (in one epoch) records an error rate less than this value - the learner halts. 

Throws
     None

public float findAccuracy (String fTestName, int validationSize, String delim)

Find accuracy of learner on the given test set. (Can be called only after training). 

Parameters
      -fTestName: name of the file containing the test data. 
      -validationSize: number of test data instances. 
      -delim: delimiter to seperate attributes in a single instance of the test data. 

Throws
      -IOException: Failed or interrupted while reading from file. 
      -FileNotFoundException: File fTestName is not found. 

*Note

Currently, the library does not perform as expected for multiple-hidden layer networks. I will look to rectify this problem when I get time.