Skip to content
Minimal Interactive Attention Visualization
HTML Python CSS JavaScript
Branch: master
Clone or download
David Bau
David Bau Add host flag.
Latest commit 4b2fc77 Jan 30, 2020
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
client merge and add button Jan 29, 2020
.gitignore
LICENSE Create LICENSE Jan 29, 2020
README.md
api.py
attnvis.gif
environment.yml
server.py Add host flag. Jan 29, 2020

README.md

Interactive Attention Visualization

A small example of an interactive visualization for attention values as being used by transformer language models like GPT2 and BERT.

by Hendrik Strobelt and Sebastian Gehrmann for the SIDN IAP class at MIT, Jan 2020

Preparation

  • Install Anaconda or Miniconda
  • run conda env create -f environment.yml to create a new environment called attnvis

Running the demo

  • activate conda environment: conda activate attnvis
  • run server: python server.py
  • visit http://localhost:8888/

Structure

api.py             -- contains the interface to the pytorch/huggingface backend 
server.py          -- defines a REST interface for the api.py calls

client/*           -- contains all client files
client/index.html  -- main file including JS code
client/styles.css  -- all CSS styles defined in here
client/tools.js    -- helper functions

Libraries used

Thanks

Thanks to Ben Hoover and David Bau for testing the code. Supported by MIT-IBM Watson AI Lab.

You can’t perform that action at this time.