Skip to content
A git repository containing an NLP example using DL4J (cuda) in Java
Java Shell Dockerfile
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docker
docs
resources-archive
src/main
.gitignore
README.md
buildUberJar.sh
download-model-and-review-database.sh
download-src-main-resources.sh
know-your-gpus.sh
pom.xml
runUberJar.sh
valohai.yaml
version.txt

README.md

DL4J NLP examples NLP using DL4J License

This area in the repo is a result of the blog post Applying NLP in Java, all from the command-line | Original post. Please refer to the post before considering using this repo to understand better on how to use the different aspects of it.


Open built-in Terminal from the IntelliJ IDEA. If you prefer, use the terminal app on your computer. In the terminal window, cd to the project root directory dl4j-nlp-cuda-example if not already in. (you can use IDE's run-configuration green run button, if you like.)

Quick startup

    $ git clone [url to this repo]
    $ cd dl4j-nlp-cuda-example
  • create a Valohai project using the Valohai CLI tool, and give it a name
    $ vh project create
    $ vh project open
    
    ### Go to the Settings page > Repository tab and update the git repo address with this git repo: https://github.com/.../dl4j-nlp-cuda-example/
  • update Valohai project with the latest commits from the git repo
    $ vh project fetch

And you are ready to start using the power of performing Machine Learning tasks from the command-line.

Advanced installation and setup

To be able to run the apps and tasks we will cover in this project on your local machine (in addition to using the Valohai platform) we would need to have the below installed and configured on your local machine (how to do each one of these are outside the scope of the project, although links to resources are provided):

Build and run app on local machine

Build and run app inside a docker container

  • Docker CE 19.03.2 or higher

Do the above and make use of GPUs - CUDA, cuDNN, etc.

  • Nvidia, CUDA 10.1 and cuDNN (7.0) drivers version (Linux and Windows machines only, support for MacOS is unavailable) (see Resources at the bottom of this post for download and installation details on GPUs, CUDA, cuDNN, etc...)

Valohai CLI installation and Valohai project setup

See Valohai CLI installation and Valohai project setup

Build the app using the Valohai platform

See Build the app using the Valohai platform

Gathering Nvidia GPU, Nvidia drivers, CUDA details

$ vh exec run know-your-gpus

Will run a bunch of commands to gather Nvidia GPU, Nvidia drivers, CUDA details on a machine, this step is also used by other steps. The captured result is stored in a log file and archive after each run. See GPU related resources

Build the app on the local machine

CPU version
$ BACKEND=cpu ./buildUberJar.sh
GPU version

Note: please ensure you have access to an Nvidia GPU and the necessary drivers i.e. CUDA, cuDNN etc... are installed, see GPU related resources.

$ BACKEND=gpu ./buildUberJar.sh

Run the app on the local machine

See Run the app on the local machine

Building the docker container to run CUDA-enabled Java apps

See Building the docker container to run CUDA-enabled Java apps

Run app in docker container

$ cd dl4j-nlp-cuda-example/docker
$ ./docker-runner --runContainer   ### neomatrix369 or DOCKER_USER_NAME is set to your Docker Hub username 

or

$ ./docker-runner --dockerUserName [neomatrix369 or your Docker Hub username] \
                  --runContainer

At the prompt in the container, do the same as mentioned in section Run app on your local machine.

You would use your Docker Hub username only when you have an Docker image with your username available locally or on Docker Hub.

Running other examples on the local machine

See Running other examples on the local machine

Credits

This example has been inspired by the DL4J NLP examples: nlp | convolution: sentence classifier | recurrent: character | recurrent: process news | recurrent: word2vec and sentiment analysis examples from DL4J. Credits to the original authors of this example on https://github.com/deeplearning4j/dl4j-examples.

License

Original authors of this DL4J example project remain the license holders of the work, although the original work has been modified to a good extent, and the Apache 2.0 License is cited for all those file/files in the matter (see License.txt in the root of the project). Exception to the above: four bash scripts have been authored by Mani Sarkar and have been cited under the Apache 2.0 license.

Resources


Return to Awesome AI/ML/DL

You can’t perform that action at this time.