This area in the repo is a result of the blog post Applying NLP in Java, all from the command-line | Original post. Please refer to the post before considering using this repo to understand better on how to use the different aspects of it.
Open built-in Terminal from the IntelliJ IDEA.
If you prefer, use the terminal app on your computer.
In the terminal window,
cd to the project root directory
dl4j-nlp-cuda-example if not already in.
(you can use IDE's run-configuration green run button, if you like.)
- open an account on https://valohai.com, see https://app.valohai.com/accounts/signup/
- install Valohai CLI on your local machine
- clone this repo (https://github.com/.../dl4j-nlp-cuda-example/)
$ git clone [url to this repo] $ cd dl4j-nlp-cuda-example
- create a Valohai project using the Valohai CLI tool, and give it a name
$ vh project create
- link your Valohai project with the url to this github repo on the Repository tab of the Settings page (https://app.valohai.com/p/[your-user-id]/dl4j-nlp-cuda-example/settings/repository/)
$ vh project open ### Go to the Settings page > Repository tab and update the git repo address with this git repo: https://github.com/.../dl4j-nlp-cuda-example/
- update Valohai project with the latest commits from the git repo
$ vh project fetch
And you are ready to start using the power of performing Machine Learning tasks from the command-line.
Advanced installation and setup
To be able to run the apps and tasks we will cover in this project on your local machine (in addition to using the Valohai platform) we would need to have the below installed and configured on your local machine (how to do each one of these are outside the scope of the project, although links to resources are provided):
Build and run app on local machine
- Python 2.7 or Python 3.4 or higher
- GraalVM CE 19.2.1 or higher (download from https://github.com/oracle/graal/releases)
- Maven 3.6.0 or higher
Build and run app inside a docker container
- Docker CE 19.03.2 or higher
Do the above and make use of GPUs - CUDA, cuDNN, etc.
- Nvidia, CUDA 10.1 and cuDNN (7.0) drivers version (Linux and Windows machines only, support for MacOS is unavailable) (see Resources at the bottom of this post for download and installation details on GPUs, CUDA, cuDNN, etc...)
Valohai CLI installation and Valohai project setup
Build the app using the Valohai platform
Gathering Nvidia GPU, Nvidia drivers, CUDA details
$ vh exec run know-your-gpus
Will run a bunch of commands to gather Nvidia GPU, Nvidia drivers, CUDA details on a machine, this step is also used by other steps. The captured result is stored in a log file and archive after each run. See GPU related resources
Build the app on the local machine
$ BACKEND=cpu ./buildUberJar.sh
Note: please ensure you have access to an Nvidia GPU and the necessary drivers i.e. CUDA, cuDNN etc... are installed, see GPU related resources.
$ BACKEND=gpu ./buildUberJar.sh
Run the app on the local machine
Building the docker container to run CUDA-enabled Java apps
Run app in docker container
$ cd dl4j-nlp-cuda-example/docker $ ./docker-runner --runContainer ### neomatrix369 or DOCKER_USER_NAME is set to your Docker Hub username or $ ./docker-runner --dockerUserName [neomatrix369 or your Docker Hub username] \ --runContainer
At the prompt in the container, do the same as mentioned in section Run app on your local machine.
You would use your Docker Hub username only when you have an Docker image with your username available locally or on Docker Hub.
Running other examples on the local machine
This example has been inspired by the DL4J NLP examples: nlp | convolution: sentence classifier | recurrent: character | recurrent: process news | recurrent: word2vec and sentiment analysis examples from DL4J. Credits to the original authors of this example on https://github.com/deeplearning4j/dl4j-examples.
Original authors of this DL4J example project remain the license holders of the work, although the original work has been modified to a good extent, and the Apache 2.0 License is cited for all those file/files in the matter (see License.txt in the root of the project). Exception to the above: four bash scripts have been authored by Mani Sarkar and have been cited under the Apache 2.0 license.
- dl4j-nlp-cuda-example project on GitHub
- CUDA enabled docker container on Docker Hub (use the latest tag: v0.5)
- GPU, Nvidia, CUDA and cuDNN
- Awesome AI/ML/DL resources
- Java AI/ML/DL resources
- Deep Learning and DL4J Resources
- Awesome AI/ML/DL: NLP resources
- DL4J NLP resources
- Language processing
- Valohai resources
- Other resources
- Other related posts
- How to do Deep Learning for Java on the Valohai Platform?
- Blog posts on how to use the Valohai CLI tool:  | 
Return to Awesome AI/ML/DL