Skip to content

Pooya

Jump to bottom
Pooya Mobadersany edited this page Mar 25, 2016 · 19 revisions

TensorFlow:

TensorFlow is an open source software library for numerical computation using data flow graphs in which nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them.

TensorFlow is Google Brain's second generation machine learning system, with a reference implementation released as open source software last November. It provides a Python API, as well as a less documented C/C++ API.

TensorFlow can run on multiple CPUs and GPUs (with optional CUDA extensions for general-purpose computing on graphics processing units). It runs on 64-bit Linux or Mac OS X desktop or server systems, as well as on mobile computing platforms, including Android and Apple's iOS.

The purpose is to train neural networks to detect and decipher patterns and correlations. There are some other libraries for numerical computations in python. One of these libraries is Theano, which is one of the strong python libraries for machine learning tasks. However, Theano's applicability in different GPU's is not experimentally verified yet, which was one of the major reasons for pushing us through TensorFlow, which gives strong computational functions to use in neural networks like activation functions, calculating loss and gradient descent.

Evaluation:

In our project we want to implement different architectures of neural networks with different complexities on a large dataset to spot the effect of complexity of neural networks on its accuracy on large datasets. There is an assumption in machine learning which says simple machine architectures don't learn from extra provided data. However, if we increase the complexity of machine then the performance would increase by the increase in amount of data. It is very important to consider that it is not applicable to train very complex neural networks on very large datasets, since it needs a lot of time. Therefore, it is very important to take advantage of some high performance computing techniques like using multiple GPUs/CPUs to split the dataset in different batches and load these batches to the designed complex neural network while it is training on some other data batches.

In this project we aim to measure the strong/weak scaling and the speed up of different neural network architectures to evaluate the effect of using hpc techniques in complex neural networks and measure the accuracy of different architectures on large datasets to evaluate how complexity affects neural network's learning and accuracy. To measure the accuracy of the network we will use logloss function which is generally used to measure the logarithm of the likelihood function for a Bernouli random distribution.

Time Table:

The time scheduling for this project is as follows:

  1. Data Clean Up

  2. Setting up and learning TensorFlow

  3. Setting up a small Neural Network (NN) in TensorFlow

  4. Running a full experimental NN on the dataset

  5. Probable debuggings

  6. Setting up Amazon Cloud

  7. Deploying the Architecture to Amazon

  8. Fine-Tuning

  9. Evaluation of the results

Reference: [1] https://www.tensorflow.org/

[2] https://en.wikipedia.org/wiki/TensorFlow

Clone this wiki locally