AstraPortia

We did stuff and now you get to read stuff. The general idea here was to create an application that leveraged new capabiltites for Astra and the Stargate API for Astra. Given the climate lately we thought it would be a good idea to find other planets we could live on and help others do the same while also keeping track of any near Earth objects that might decide to end our existence on our wonderful planet.

This application is a MicroService implemented with Java using Spring Boot and Maven. Commands issued will reflect this. Be sure to check out our FRICTION DOC to see what the experience was like for our team.

1. Run the DEMO

Choose how to start the APP:

• For Heroku and Docker you are connected to our ASTRA instance.
• For gitpod and Maven you can edit variables to put to your instances.

Run with Docker

Prequisites: Have docker install on your machine

docker run -p 8080:8080 clunven/astra-portia

To build the image from the code you can use:

git clone https://github.com/DataStax-Academy/AstraPortia.git
cd astra-portia
mvn clean package -DskipTests dockerfile:build

or

git clone https://github.com/DataStax-Academy/AstraPortia.git
cd astra-portia
mvn clean package -DskipTests
docker build -t clunven/astra-portia .

Run in gitpod

Run with maven

Prequisites: Have JDK11+ and Maven installed on your machine.

• Clone the repo

git clone https://github.com/DataStax-Academy/AstraPortia.git

• Start the application *(all tests are OK but import the dataset)

mvn clean package -DskipTests spring-boot:run

Run on Heroku

The application is deployed on Heroku using the container feature. This is how we built the image

heroku container:login
heroku container:push web --app astra-portia
heroku container:release web --app astra-portia

3. What is cool about this App

Demo project fo Stargate Hackathon by Clavis Team

• We create a Java Client for Stargate Document API including the automation of authToken renewal.
• We created a batch to import a DataSet in ASTRA with Document API
• We implemented Create, Read, Update, Delete leveraging the API

4. More about the development Process

Create an Instance

✅ Step 1a. Create an Astran instance (DBaas by Datastax with FREE TIER). If you still don't have your Astra instance please follow those instructions.

You can use the credentials and keyspace names of your choice, remember tough that you would need to provide those fields to setup the application, so note them.

• the username
• the userpassword
• the keyspace

Working with organization

Since we needed to collaborate using a single database it seemed like a good idea to try out the new Organizations feature. With a single org we all had a single, shared database to use. Nice!

2. Data Model and Data Loading

Ok, so we had a team database, great. Now we needed a data model and data to work with for our application. We took two approaches. One used DSBulk and the secure connect bundle with a CSV file. The other used the new Stargate API via a test harness within our java application to create the data model and import the dataset via JSON.

DSBulk

Planetary systems composite dataset

✅ Step 2a. Create data model In this case, we created our data model first by creating the planetary_systems_composite.cql file located HERE and executing those within the CQLSH of Astra. Just paste the contents of the planetary_systems_composite.cql into the CQLSH console after logging in and setting the context of your database to USE the stargate keyspace.

✅ Step 2b. Import data into the database

Take note of the use of the option -b "secure-connect-clavis.zip" that allows us to connect to our Astra database pretty seamlessly using DSBulk. You can download the secure bundle for your Astra database by using the Download secure connect bundle link in the Connection Details section of Astra. NOTE that your database name will most likely be different than ours.

In order for this command to work you will need to install DSBulk if you have not done so already. Once installed, ensure to either set the PATH for the location of dsbulk or simply run dsbulk from the /bin dir of your installation directory.

📘 Command to execute

dsbulk load -url 1_datasets/PSCompPars_2020.09.10_11.00.24.csv -k stargate -t planetary_systems_composite -b "secure-connect-clavis.zip" -u gateuser -p gatepassword -header false -m '0=pl_name,1=hostname,2=sy_snum,3=sy_pnum,4=discoverymethod,5=disc_year,6=disc_facility'

📗 Expected output Once you run the above command you should see something that looks similar to the following.

JSON load using Stargate API

Near Earth Object dataset

✅ Step 2c. Create data model and import dataset In this next case we used the Stargate API to both create the data model and load the dataset via a test harness within our java application.

Whether you are using GitPod or a local environment ensure to navigate to the astra-portia-api directory from the root AstraPortia. This is where our Spring Boot application lives.

📘 Command to execute

cd astra-portia-api
pwd

📗 Expected output

Note the the absolute path will show your username, not the one you see in the image from my instance. The important thing is that you are located in the AstraPortia/astra-portia-api directory.

✅ Step 2d. Execute the test to create the data model and import data

This step will use the a JSON file embedded in the application located HERE

In the application please locate the file src/test/resources/application-test.properties and edit the following keys based on your configuration.

stargate.url=https://eabba33b-dc22-4055-bced-e84e87cfa92c-europe-west1.apps.astra.datastax.com/api/rest
stargate.username=gateuser
stargate.password=gatepassword
stargate.namespace=stargate

The URLis the REST API url then can be copy from the Astra UI in "Connection Details (left part)" then REST API.

📘 Command to execute

mvn test

About a dozen tests will be executed from connectivity, CRUD with a client a also the import batch that will log a line for each document inserted.

📗 Expected output

3. The Application

Ok, now we have our data model and data. The next step is to run the application and see what it does. Since we are using Spring Boot this is really quite simple.

✅ Step 3a. Run the app

As for the test you need to setup the configuration file. Locate src/main/resources/application.yaml and fill the same keys as before

##### PARAMETERS TO EDIT ###########
stargate:
  username: gateuser
  password: gatepassword
  namespace: stargate
  url: https://eabba33b-dc22-4055-bced-e84e87cfa92c-europe-west1.apps.astra.datastax.com/api/rest
####################################

📘 Command to execute

mvn spring-boot:run

📗 Expected output

✅ Step 3b. Go to your browser

The application is configured to run at port 8081 so if you are running locally navigate to localhost:8081 in your browser. If you are running via GitPod a dialog will give you the option to Run in Browser once you launch the app.

✅ Step 3c. Have fun traveling and exploring

Once there notice there are a set of tabs at the top of the interface.

While at the Stargate be sure to click on all of the chevrons to activate the portal.

Use the Earth Catalog to look at the very, very small list of items we have in there right now.

Finally, the API Docs will bring you to a swagger interface you can use to browse the back end API we implemented to work with the Stargate API. This was done to abstract the API a bit and make it very easy to start getting data from the database.