ForwardThinking

A Python library implementing Forward Thinking for training deep neural networks, based on the paper:

Training Deep Networks with Forward Thinking

Forward Thinking is an approach to training deep neural networks layer-by-layer, which can be faster and more stable than traditional end-to-end backpropagation.

Models

• Forward Thinking
• Push Forward Thinking
• Pull Forward Thinking
• Adaptive Push Forward Thinking

Installation

git clone https://github.com/smwade/forwardThinking
cd forwardThinking
pip install -r requirements.txt
python setup.py install

Requirements

• TensorFlow >= 1.0
• NumPy
• Keras

Usage

Models are located in forwardThinking/models/. Each model implements the forward thinking training procedure:

from forwardThinking.models import forwardThinking

# Train a forward thinking network
model = forwardThinking.ForwardThinkingModel()
model.train(X_train, y_train)
predictions = model.predict(X_test)

Project Structure

forwardThinking/
  forwardThinking/
    models/
      forwardThinking.py          # Base forward thinking
      passForwardThinking.py      # Push forward thinking
      adaptiveForwardThinking.py  # Adaptive variant
      dnn.py                      # Standard DNN baseline
      weddingCake.py              # Wedding cake architecture
  tests/                          # Unit tests
  docs/                           # Documentation