Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Update README to use Castor-models and Instructions for Internal Users (
#113) * Update instructions to use Castor-models * Consolidate requirements.txt * Refine README with convenience scripts * Update internal instructions * MP-CNN working dir minor edit
- Loading branch information
1 parent
62f8abe
commit 5bf33bf
Showing
6 changed files
with
89 additions
and
16 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Original file line | Diff line number | Diff line change |
---|---|---|---|
@@ -0,0 +1,49 @@ | |||
# Instructions for DSG Castor Contributors | |||
|
|||
Please follow these instructions if you are a graduate student or undergrad research assistant working with the group | |||
in the Data Systems Lab and want to run Castor on the lab desktop GPU machine (dragon). | |||
|
|||
If you have trouble / questions with instructions on this page, ping @tuzhucheng on Slack. | |||
|
|||
## PyTorch Environment | |||
|
|||
We already have a multi-user Conda environment with PyTorch and all other dependencies installed, so you do not need to | |||
install anything yourself. However, you can create [Conda environments](https://conda.io/docs/user-guide/tasks/manage-environments.html) | |||
if you need to experiment with different library versions etc. | |||
|
|||
The multi-user Conda environment is located at `/anaconda3/`. | |||
To use this multi-user environment, just add the following to your `.bashrc` or configuration file for your favourite shell. | |||
|
|||
```bash | |||
export PATH="/anaconda3/bin:$PATH" | |||
export LIBRARY_PATH="/usr/lib/nvidia-375" | |||
``` | |||
|
|||
Please also ensure `/usr/local/cuda-8.0/lib64` is in the `LD_LIBRARY_PATH` environment variable **if it is not already**. | |||
If not, you should add it in the `.bashrc` similar to above. | |||
|
|||
Please re-login or re-source your shell configuration after `.bashrc` is updated for the updated environment variables | |||
to take effect. | |||
|
|||
## Data and Pre-Trained Models | |||
|
|||
We use shared cloned versions of the Castor-data and Castor-models repositories. | |||
Instead of making your own cloned copies, you can just create symbolic links to the shared version instead | |||
in your own working directory to save disk space. Assuming you want to put `Castor`, `Castor-data`, and `Castor-models` | |||
under a directory called `castorini` and you are currently in the `castorini` directory, you can enter these commands: | |||
|
|||
```bash | |||
ln -s /Castor-data Castor-data | |||
ln -s /Castor-models Castor-models | |||
``` | |||
|
|||
So after you clone Castor, you have a directory structure under `castorini` that looks like this: | |||
|
|||
``` | |||
. | |||
├── Castor | |||
├── Castor-data | |||
└── Castor-models | |||
``` | |||
|
|||
where `Castor-data` and `Castor-models` are actually symbolic links to `/Castor-data` and `/Castor-models`. |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Original file line | Diff line number | Diff line change |
---|---|---|---|
@@ -1,7 +1,9 @@ | |||
Flask==0.12.1 | |||
gensim==1.0.1 | gensim==1.0.1 | ||
numpy==1.12.1 | nltk==3.2.5 | ||
numpy==1.14.0 | |||
pandas==0.19.2 | pandas==0.19.2 | ||
Flask==0.12.1 | |||
nltk==3.2.2 | |||
pyjnius==1.1.1 | pyjnius==1.1.1 | ||
-e git+https://github.com/castorini/Castor.git#egg=sm-cnn-1.0.0 | scikit-learn==0.19.1 | ||
scipy==1.0.0 | |||
torchtext==0.2.3 |
This file was deleted.
Oops, something went wrong.