Skip to content
🔎 Attentional interfaces in TensorFlow.
Jupyter Notebook CSS Shell Python Makefile
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data
images
.gitignore
Makefile
README.md
__init__.py
attention.ipynb
custom.css
dockerfile.cpu
fetch_data.sh
jupyter_notebook_config.py added docker info and updated code to tfv1.0 Mar 3, 2017
requirements.txt
run_jupyter.sh
version.txt

README.md

Interpretability via Attentional and Memory-based Interfaces Using TensorFlow

A closer look at the reasoning inside your deep networks.

TLDR: This post will serve as a gentle introduction to attentional and memory-based interfaces in deep neural architectures using TensorFlow. Incorporation of attention mechanisms is very simple and can improve transparency interpretability in our complex models. We will conclude with extensions and caveats of the interfaces. The intended audience for this notebook are developers and researchers who have some basic understanding of Tensorflow and fundamental deep learning concepts. Check out this post for a nice introduction to Tensorflow. image1 Note: This code has been written for TensorFlow v1.0.0

Before running the Jupyter notebook attention.ipynb, you'll need to ensure that your machine is set up to run TensorFlow and install a handful of helpful additional libraries.

Setup

  1. Download this entire repo from GitHub

  2. Open your terminal and use cd to navigate into the top directory of the repo on your machine

  3. Unzip the processed reviews data by entering the following inside the data directory.

unzip processed_reviews.zip

Now, set up your environment by using the Dockerfile included with this repo (option A) or configuring it manually (option B).

Option A: use the Dockerfile configured for this notebook

(Note: For GPU versions, etc. check out TensorFlow's Docker repo.)

  1. After downloading this repo to your machine, open your terminal and use cd to navigate to the directory that contains Dockerfile.cpu.

  2. To build the Dockerfile, enter

docker build -t dockerfile_cpu -f dockerfile.cpu .

If you get a permissions error on running this command, you may need to run it with sudo:

sudo  build -t dockerfile_cpu -f dockerfile.cpu .
  1. Run Docker from the Dockerfile you've just built
docker run -it -p 8888:8888 -p 6006:6006 dockerfile_cpu bash

or

sudo docker run -it -p 8888:8888 -p 6006:6006 dockerfile_cpu bash

if you run into permission problems.

  1. Launch Jupyter by entering
jupyter notebook

and, using your browser, navigate to the URL shown in the terminal output (usually http://localhost:8888/)

Option B: set up environment manually on local machine

For instructions on installing TensorFlow, please see TensorFlow's platform-specific installation instructions.

Required:
python 3.3+
pip 9.0.1 (sudo easy_install pip)
virtualenv (pip install virtualenv)
Optional: CUDA enabled GPU (explicity define cpu components (embedding, etc.)
  1. Enter the attention directory in your terminal and enter the following commands
make venv
source oreilly-attention-venv/bin/activate
make load-data
ipython3 notebook attention.ipynb
  1. Launch Jupyter by entering
jupyter notebook

and, using your browser, navigate to the URL shown in the terminal output (usually http://localhost:8888/)

Contact:

Goku Mohandas

Author Bio:

Goku Mohandas is an artificial intelligence (AI) researcher in Silicon Valley with a focus on using deep learning for natural language tasks. His interests include research on AI for intelligent search and question answering augmented by attentional and memory-based interfaces. He also strongly believes in the democratization of AI with a focus on interpretability and transparency. Previous work includes working on the intersection of AI and biotechnology at the Johns Hopkins University Applied Physics Laboratory.

You can’t perform that action at this time.