Skip to content

Mohsen MD

Jump to bottom
mohsensalari edited this page Mar 25, 2016 · 15 revisions

#Team Members

  • Sahar Harati, Second Year PhD Student, Computer Science Department, Emory University
  • Mohsen Salari, First Year PhD Student, Biomedical Informatics Department, Emory University
  • Pooya Mobadersany, First Year PhD Student, Biomedical Informatics Department, Emory University

#Introduction Machine learning is an iterative experimental process. The common methodology in solving a problem involves a preliminary idea, upon which the first solution is developed. After building the proposed solution and performing experiments on the solution, more often than not one finds himself redesigning the architecture in an effort to overcome the shortcomings of the original solution. To illustrate the point, imagine that we want to build a speech recognition system capable of deciphering uttered letters in a wave file. One of the first solutions that may come to mind would be a multilayer neural network similar to the one depicted in the figure below.

simple network

Picture credits: Andrew Ng

After implementing one such architecture and evaluating the results, one may notice that refining the network to a recurrent neural network instead of a simple network may be better able to capture the temporal dependency of the uttered letters and hence improve the performance of the system. So the architecture is transformed into a network similar to figure below.

RNN

Picture credits: Andrew Ng

This process of constantly implementing solutions, experimenting with them and then coming up with a redesign is very time consuming. A good portion of the time invested in this process goes towards running experiments with typically large data. Using techniques from HPC, and running experiments on large clusters is bound to reduce the timing of this iterative process and give the engineer the ability to try more novel solutions in shorter timeframes.

Given any computational model we also have the intuition that increasing the training data, we can achieve higher accuracies. But it is suggested [1] that there is an upper bound on the added performance when just increasing the volume of training data and after some point, one has to use more complicated learning techniques. In this work, we want to verify this hypothesis by testing different machine learning architectures on a dataset, increasing the volume of the data to a point where no significant improvement is resulted, and then improving the machine learning architecture each time. As the experiment time would become prohibitive, we will set-up a high performance computing environment for performing the experiments.

#Methodology After cleaning up the data, we will apply a simple one layer neural network to the data. The optimization is performed using a Logloss cost function. We start by a small batch of data, and gradually increase the size of the training data up to a point when no significant increase in the performance is noted.

At this point, we will increase the complexity of our model, by iteratively changing our architecture to one that includes many more hidden layers. Each time the amount of data that will produce the optimum accuracy is noted, along with performance metrics discussed in our Metrics section.

To be able to note the effect of using larger volumes of data, we will set up a high performance computing center and balance the load on as many GPUs as our budget allows. This would make the experimenting phase possible within a reasonable time frame.

Clone this wiki locally