Skip to content
Deep learning with text doesn't have to be scary.
Python Shell Dockerfile
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Initial commit for Github Sep 4, 2019
benchmark Fix errors installing package during Docker build Sep 5, 2019
ci-gpu Initial commit for Github Sep 4, 2019
ci
docs Upgrade pytorch-transformers to transformers, minor refactoring Dec 30, 2019
gobbli
img Add logos to README/docs Sep 9, 2019
.dockerignore Initial commit for Github Sep 4, 2019
.gitattributes Add .gitattributes excluding .ipynb from Linguist Dec 30, 2019
.gitignore
.readthedocs.yml Remove optional requirements from readthedocs install Sep 4, 2019
.travis.yml Fix typo Sep 4, 2019
CONTRIBUTING.md Fill in CLA signing link Sep 4, 2019
Dockerfile Fix errors installing package during Docker build Sep 5, 2019
LICENSE Initial commit for Github Sep 4, 2019
MANIFEST.in
README.md Upgrade pytorch-transformers to transformers, minor refactoring Dec 30, 2019
conftest.py Add --low-resource arg for tests Sep 4, 2019
docker-compose.yml Fix spacy GPU build/test errors, add compose services for new images Jan 2, 2020
generate_docs.sh Move sphinx-apidoc invocation into Sphinx build process Sep 4, 2019
meta.json Bump version number Oct 22, 2019
pyproject.toml Initial commit for Github Sep 4, 2019
pytest.ini Initial commit for Github Sep 4, 2019
requirements.txt Upgrade spaCy model used in development requirements.txt Oct 22, 2019
run_ci.sh
run_dist.sh Remove old dist artifacts before generating new ones Sep 12, 2019
setup.cfg Initial commit for Github Sep 4, 2019
setup.py Working SKLearnClassifier implementation Jan 2, 2020
test_remote_gpu.sh

README.md

gobbli logo
PyPI version PyPI version PyPI - Python Version DOI

This is a library designed to provide a uniform interface to various deep learning models for text via programmatically created Docker containers.

Usage

See the docs for prerequisites, a quickstart, and the API reference. In brief, you need Docker installed with appropriate permissions for your user account to run Docker commands and Python 3.7. Then run the following:

pip install gobbli

You may also want to check out the benchmarks to see some comparisons of gobbli's implementation of various models in different situations.

Development

Assuming you have all prerequisites noted above, you need to install the package and all required + optional dependencies in development mode:

pip install -e ".[augment,tokenize]"

Install additional dev dependencies:

pip install -r requirements.txt

Run linting, autoformatting, and tests:

./run_ci.sh

To avoid manually fixing some of these errors, consider enabling isort and black support in your favorite editor.

If you're running tests in an environment with less than 12GB of memory, you'll want to pass the --low-resource argument when running tests to avoid out of memory errors.

NOTE: If running on a Mac, even with adequate memory available, you may encounter Out of Memory errors (exit status 137) when running the tests. This is due to not enough memory being allocated to your Docker daemon. Try going to Docker for Mac -> Preferences -> Advanced and raising "Memory" to 12GiB or more.

If you want to run the tests GPU(s) enabled, see the --use-gpu and --nvidia-visible-devices arguments under py.test --help. If your local machine doesn't have an NVIDIA GPU, but you have access to one that does via SSH, you can use the test_remote_gpu.sh script to run the tests with GPU enabled over SSH.

Docs

To generate the docs, install the docs requirements:

pip install -r docs/requirements.txt

Since doc structure is auto-generated from the library, you must have the library (and all its dependencies) installed as well.

Then, run the following from the repository root:

./generate_docs.sh

Then browse the generated documentation in docs/_build/html.

Attribution

gobbli wouldn't exist without the public release of several state-of-the-art models. The library incorporates:

Original work on the library was funded by RTI International.

Logo design by Marcia Underwood.

You can’t perform that action at this time.