Deploying apps in Discovery Environment
The CyVerse Discovery Environment (DE) provides a simple yet powerful web portal for managing data, analyses, and workflows. The DE uses containers (via Docker and Singularity through Agave) to support customizable, non-interactive, reproducible workflows using data stored in the CyVerse Data Store, based on iRODS. Agave is a "Science-as-a-Service" API platform for high-performance computing (HPC), high-throughput computing (HTC) and big-data resources.
Deploying Docker images as apps in DE
Instruction guide: This paper will guide you to bring your dockerized tools into CyVerse DE.
Significant changes have been made as to how you can bring your tools into DE and so we are working on a separate paper that will show all those changes. Meanwhile you can follow the below tutorial for integrating your tools.
Here are the basic steps for deploying Docker images as apps in DE. For this tutorial I am going to show an example of Tensor image classifier docker image that I dockerized and pushed to dockerhub.
- Build and test your Docker images
- Push your Docker image to public repositories
- Add Docker images as tool in DE
- Create a UI for the tool in DE
- Test the app using appropriate test data
If you already have your own Docker image or Docker image of interest is hosted on public repositories (Dockerhub or quay.io or some other public repository), then you can skip to step 3
1. Build and test your Docker images
The first step is to dockerize your tool or software of interest. Detailed steps of how to dockerize your tool and test your dockerized images can be found in sections intro to docker and advanced docker.
For this tutorial I will use the
tensorflow image classifier docker image that I built using this code
docker run -v $(pwd):/data -w /data tensorflow_up:1.0 sample_data/16401288243_36112bd52f_m.jpg
2. Push your Docker image to public repositories
Here is the docker image for the tensorflow image classifier on docker hub - https://hub.docker.com/r/upendradevisetty/tensorflow_image_classifier/
3. Add Docker images as tool in DE
All tools now run installed as Docker images in the DE. Once the software is dockerized and available as Docker images on dockerhub then you can add those docker images as a tool in DE.
Check if the tool and correct version are already installed in the DE by following the below steps.
- Log in to the Discovery Environment by going to https://de.cyverse.org/de/, entering your CyVerse username and password, and clicking LOGIN. If you have not already done so, you will need to sign up for a CyVerse account.
- Click the
Appswindow to open the Apps window.
- Click the
Manage Toolsbutton on the top-right of the Apps window.
- In the search tools field, enter the first few letters of the tool name and then click enter.
- If the tool is available then you can skip to skip to step 3 for creating a UI for that tool.
If the tool is not available in DE then do the following:
Click open the
Manage Toolswindow and then click
Then enter the fields about your tool and then click "Ok".
- Tool Name: It should be the name of the tool. For example "tensorflow_image_classifier".
- Description: A short Description about the tool. For example "Tensorflow image classifier".
- Version: What is the version number of the tool. For example "1.0".
- Image name: Name of the Docker image on dockerhub or quay.io. For example "upendradevisetty/tensorflow_image_classifier".
- Tag: What is the tag of your Docker image. This is optional but is highly recommended. If non specified, it will pull the default tag
latest. If the
latesttag is not avaiable the tool integration will fail. For example "1.0"
- Entrypoint: Do you want a entrypoint for your Docker image? This optional.
- Docker Hub URL: URL of the Dockerhub docker image. Option but is recommended. In this example "".
- If there is no error, it indicates successful integration of the tool.
4. Create a UI for the tool in DE
Once the Dockerized tool is added, you can create the app UI for the tool. The
Create App window consists of four distinct sections:
- The first section contains the different app items that can be added to your interface. To add an app item, select the one to use (hover over the object name for a brief description) and drag it into position in the middle section.
- The second section is the landing place for the objects you dragged and dropped from the left section, and it updates to display how the app will look when presented to a user.
- The third section (Details) displays all of the available properties for the selected item. As you customize the app in this section, the middle section updates dynamically so you can see how it will look and act.
- Finally, the fourth section at the bottom (Command line view) contains the command-line commands for the current item’s properties. As you update the properties in the Details section, the command-line view updates as well to let you make sure that you are passing the correct arguments in the correct order.
Creating a new app interface requires that you know how to use the tool. With that knowledge, you create the interface according to how you want options to be displayed to a user.
Here is an example of the
Tensorflow image classifier - 1.0 app UI in DE
5. Test the app using appropriate test data
After creating the new app according to your design, test your app in the your Apps under development folder in the DE using appropriate test data to make sure it works properly.
For testing, we'll use the the same image that we have used earlier.
- First open the
Tensorflow image classifier - 1.0app in the app window
- Next browse the test file in the app and click launch analysis
- After the analysis is completed, open the folder and check to see if the image classifier correctly predicts
Congrats!!! It works. The image classifier correctly predicts that image is a daisy..
- If your app works the way you expect it to you can share your app or make the app public
- If your app doesn't work, then you may need to make changes to the app UI or you need to make changes to your Docker image. If you make changes to the Docker image, then you don't need to create a new app UI again as the Docker image updates will be propagated automatically.