Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving modularity in EMI-RNN Implementation #43

Merged
merged 162 commits into from
Aug 23, 2018
Merged

Conversation

metastableB
Copy link
Contributor

@metastableB metastableB commented Jul 16, 2018

This PR attempts to improve the EMI-RNN implementation to make it more generic. Specifically, we want to support the following use-cases:

  1. Arbitrary RNN models: We want to allow support for RNN/LSTM/GRU/FastGRNN etc in a more natural way.
  2. Arbitrary graph support: A generalization of 1, allowing any arbitrary graph (RNN/CNN/Stacked RNN/ Fully Connected/ProtoNN) to be supported as part of the general EMI-RNN idea and algorithm.
  3. Easy extensible: extensibility of loss functions, regularizers etc with potential to support arbitrary use cases (like exponentially decayed error used by the predictive maintenance team)

We require that all of the above be provided with full support for proper checkpointing, restoring from meta graphs, restoring from numpy files etc.

To allow the above, we've split the entire pipeline in to two parts:

  1. The tensorflow computation graph:
  2. The EMI driver algorithm

Tensorflow Computation Graph

The tensorflow computation graph for our framework has the following three parts. Outputs from one part is fed as input to the next stage.

  1. An efficient data feeder - exposed in edgeml.graph.rnn:EMI_DataPipelin.
  2. The generic computation graph. An abstract class is defined and an extension with LSTM is provided in edgeml.graph.rnn:EMI_BasicLSTM
  3. The trainer. The trainer deals with attaching various objective functions to the computation graph declared above and is exposed in edgeml.trainer.emiTrainer.

The EMI driver algorithm

Since the EMI algorithm requires saving and restoring models from checkpoints during the training and belief update stages, we need close control on session management, graph construction and checkpoint managerment. Further, the additional ability to initialize the graphs from numpy files requires even more dirty coupling between the graph and the driver routines. I've henced moved all these dirty parts of the algorithm into a driver defined in edgeml.trainer.EMI_Driver.

The upside is that user side code is cleaner with the ability to create and modify graphs as and when you want. The user though will have to forgo control on tensor flow sessions and will have to be careful when using session.

@metastableB metastableB changed the title [WIP] Improving modularity in EMI-RNN Implementation Improving modularity in EMI-RNN Implementation Aug 19, 2018
@metastableB
Copy link
Contributor Author

@harsha-simhadri we are ready to merge. Please have a look and proceed with the merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants