Skip to content

Commit

Permalink
Revert "merge xingjian's"
Browse files Browse the repository at this point in the history
This reverts commit ea1f1aa.
  • Loading branch information
zheyuye committed Aug 20, 2020
1 parent fe74dda commit 770f079
Show file tree
Hide file tree
Showing 11 changed files with 53 additions and 496 deletions.
201 changes: 0 additions & 201 deletions LICENSE

This file was deleted.

11 changes: 0 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,14 +72,3 @@ python3 -m gluonnlp.cli.preprocess help

# Run Unittests
You may go to [tests](tests) to see all how to run the unittests.


# Use Docker
You can use Docker to launch a JupyterLab development environment with GluonNLP installed.

```
docker pull gluonai/gluon-nlp:v1.0.0
docker run --gpus all --rm -it -p 8888:8888 -p 8787:8787 -p 8786:8786 gluonai/gluon-nlp:v1.0.0
```

For more details, you can refer to the guidance in [tools/docker].
2 changes: 1 addition & 1 deletion src/gluonnlp/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
__version__ = '1.0.0'
__version__ = '1.0.0.dev0'
from . import base
from . import data
from . import models
Expand Down
26 changes: 12 additions & 14 deletions tools/batch/README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,18 @@
# Launch AWS Jobs
For contributors of GluonNLP, you can try to launch jobs via AWS Batch.
Once you've correctly configured the AWS CLI, you may use the following command
to submit a dummy example in which `remote` and `source-ref` denote the repository
url and branch name respectively.
# Batch Job Examples
Based on AWS Batch, we provide a quick way to submit a job, including Conversion, Question Answering, Machine Translation, Text Translation and Pre-trained Model Training, which can be used for regular inspections or for rapid deployment of experiments.

```
See a dummy submission for example in which `remote` and `source-ref` denote the repository url and branch name respectively.

```bash
python3 submit-job.py \
--region us-east-1 \
--source-ref master \
--job-type p3.2x \
--work-dir tools/batch \
--remote https://github.com/dmlc/gluon-nlp \
--command "python3 hello_world.py" \
--wait
--region us-east-1 \
--source-ref master \
--job-type g4dn.12x \
--name dummy_example \
--work-dir scripts/preprocess \
--remote https://github.com/dmlc/gluon-nlp/ \
--command 'git rev-parse HEAD | tee stdout.log'
```

## Conversion Toolkits
Following the instruction of [converting scripts](../../scripts/conversion_toolkits), several pre-trained models could be converted through the corresponding conversion tool as below command where `${MODEL_TYPE}` could be selected from `[albert, bert, electra, mobilebert, bart, robert, xmlr]`.
```bash
Expand Down
48 changes: 24 additions & 24 deletions tools/batch/docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,27 +1,27 @@
FROM nvidia/cuda:10.2-cudnn7-devel-ubuntu18.04

RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
locales \
cmake \
wget \
subversion \
git \
curl \
vim \
unzip \
sudo \
ca-certificates \
libjpeg-dev \
libpng-dev \
libfreetype6-dev \
python3-dev \
python3-pip \
python3-setuptools \
libxft-dev &&\
rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
locales \
cmake \
wget \
subversion \
git \
curl \
vim \
unzip \
sudo \
ca-certificates \
libjpeg-dev \
libpng-dev \
libfreetype6-dev \
python3-dev \
python3-pip \
python3-setuptools \
libxft-dev &&\
rm -rf /var/lib/apt/lists/*

RUN pip3 install --upgrade pip && pip3 install awscli && pip3 install --pre 'mxnet-cu102' -f https://dist.mxnet.io/python
RUN git clone https://github.com/dmlc/gluon-nlp
WORKDIR gluon-nlp
ADD gluon_nlp_job.sh .
RUN pip3 install --upgrade pip && pip3 install awscli && pip3 install --pre 'mxnet-cu102' -f https://dist.mxnet.io/python
RUN git clone https://github.com/dmlc/gluon-nlp
WORKDIR gluon-nlp
ADD gluon_nlp_job.sh .
10 changes: 0 additions & 10 deletions tools/batch/hello_world.py

This file was deleted.

0 comments on commit 770f079

Please sign in to comment.