Skip to content
PyTorch deep learning models for text processing
Python JavaScript Java Other
Branch: master
Clone or download
achyudh and daemon Remove document classification code that's been moved to hedwig (#186;
…close #185)

* Remove document classification datasets

* Remove ReutersEvaluator

* Remove ReutersTrainer

* Remove document classification models

* Remove document classification from README.md
Latest commit fa2f595 Apr 7, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
anserini_dependency Migrate to from GitHub castorini/data to uWaterloo Castor-data (#103) May 23, 2018
common Remove document classification code that's been moved to hedwig (#186; Apr 8, 2019
conv_rnn Tidy up some code (#174) Feb 6, 2019
datasets Remove document classification code that's been moved to hedwig (#186; Apr 8, 2019
decatt Tidy up some code (#174) Feb 6, 2019
docs Update README to use Castor-models and Instructions for Internal Users ( May 25, 2018
esim Add ESIM model (#169) Jan 29, 2019
idf_baseline Tidy up some code (#174) Feb 6, 2019
mp_cnn Tidy up some code (#174) Feb 6, 2019
nce add storing prediction/qrel files option for test set (#125) Jun 19, 2018
sm_cnn Update README to use Castor-models and Instructions for Internal Users ( May 25, 2018
sse Tidy up some code (#174) Feb 6, 2019
utils Add SSE model (#168) Dec 18, 2018
vdpwi Tidy up some code (#174) Feb 6, 2019
.gitignore Make Kim CNN ONNX-exportable (#136) Aug 4, 2018
LICENSE Add Apache 2.0 license (#132) Jul 11, 2018
README.md Remove document classification code that's been moved to hedwig (#186; Apr 8, 2019
__init__.py TrecQA for MP-CNN (#77) Nov 4, 2017
config.cfg.example Connected to E2E pipeline (#85) Nov 25, 2017
requirements.txt Use PackedSequence for ConvRNN (#126) Jun 24, 2018
run_ui.sh Connected to E2E pipeline (#85) Nov 25, 2017
setup.py renaming sm_model; faster bridge (#18) (#22) May 3, 2017

README.md

Castor

This is the common repo for deep learning models implemented in PyTorch by the Data Systems Group at the University of Waterloo.

Models

Predictions Over One Input Text Sequence

Moved to https://github.com/castorini/hedwig

Predictions Over Two Input Text Sequences

For paraphrase detection, question answering, etc.

Each model directory has a README.md with further details.

Setting up PyTorch

If you are an internal Castor contributor using GPU machines in the lab, follow the instructions here.

Castor is designed for Python 3.6 and PyTorch 0.4. PyTorch recommends Anaconda for managing your environment. We'd recommend creating a custom environment as follows:

$ conda create --name castor python=3.6
$ source activate castor

And installing the packages as follows:

$ conda install pytorch torchvision -c pytorch

Other Python packages we use can be installed via pip:

$ pip install -r requirements.txt

Code depends on data from NLTK (e.g., stopwords) so you'll have to download them. Run the Python interpreter and type the commands:

>>> import nltk
>>> nltk.download()

Finally, run the following inside the utils directory to build the trec_eval tool for evaluating certain datasets.

$ ./get_trec_eval.sh

Data and Pre-Trained Models

If you are an internal Castor contributor using GPU machines in the lab, follow the instructions here.

To fully take advantage of code here, clone these other two repos:

Organize your directory structure as follows:

.
├── Castor
├── Castor-data
└── Castor-models

For example (using HTTPS):

$ git clone https://github.com/castorini/Castor.git
$ git clone https://git.uwaterloo.ca/jimmylin/Castor-data.git
$ git clone https://git.uwaterloo.ca/jimmylin/Castor-models.git

After cloning the Castor-data repo, you need to unzip embeddings and run data pre-processing scripts. You can choose to follow instructions under each dataset and embedding directory separately, or just run the following script in Castor-data to do all of the steps for you:

$ ./setup.sh
You can’t perform that action at this time.