Skip to content

Commit

Permalink
cloud section updates
Browse files Browse the repository at this point in the history
  • Loading branch information
Trevhall52 committed Dec 15, 2022
1 parent 78b4f40 commit aae2e2e
Show file tree
Hide file tree
Showing 4 changed files with 10 additions and 10 deletions.
2 changes: 1 addition & 1 deletion docs/tutorials/cumulus1.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ A flavor defines the compute, memory, and storage capacity of our instance.
* Now select from available IP addresses if needed (e.g., if you'll be adding a web portal to your instance)
* The _Select port to be associated_ option should already be pre-populated with the internal IP of your new instance

![]images/instance_ip2.png)
![](images/instance_ip2.png)

---
### Part 3: Logging into your Instance
Expand Down
8 changes: 4 additions & 4 deletions docs/tutorials/cumulus2.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Our main objective in tutorial is to demonstrate potential workflows that could
## Tutorial


Before we can get this application up and running in a CUmulus instance we have some house-keeping to take care of. Many of steps below assume you have worked through [tutorial1](./cumulus1.md), which describes the instance creation process.
Before we can get this application up and running in a CUmulus instance we have some house-keeping to take care of. Many of the steps below assume you have worked through [tutorial1](./cumulus1.md), which describes the instance creation process.

---
#### Part 1: Instance Setup
Expand Down Expand Up @@ -92,7 +92,7 @@ Okay, now we can get into the real content of this Demo!
---
#### Part 5: Setting up Docker
---
The next thing we'll need to do is install Docker. Docker is an open source containerization platform. It enables developers to package applications into containers—standardized executable components combining application source code with the operating system (OS) libraries and dependencies required to run that code in any environment. For a great overview of Docker containerization visit https://docs.docker.com/get-started/overview/. In this tutorial we will use Docker to "pull" and "run" a mysql database images that we can store data (in this case tweets) into. A Docker image is a file used to execute code in a Docker container. Docker images act as a set of instructions to build a Docker container, like a template. Docker images also act as the starting point when using Docker. An image is comparable to a snapshot in virtual machine (VM) environments. Instead of downloading mysql directly to the host machine (your instance) and having to deal with dependencies and uninstalling/fixing if anything goes wrong we can use the mysql image which we can easily create/remove a Docker container from.
The next thing we'll need to do is install Docker. Docker is an open source containerization platform. It enables developers to package applications into containers—standardized executable components combining application source code with the operating system (OS) libraries and dependencies required to run that code in any environment. For a great overview of Docker containerization visit https://docs.docker.com/get-started/overview/. In this tutorial we will use Docker to "pull" and "run" a mysql database image that we can store data (in this case tweets) into. A Docker image is a file used to execute code in a Docker container. Docker images act as a set of instructions to build a Docker container, like a template. Docker images also act as the starting point when using Docker. An image is comparable to a snapshot in virtual machine (VM) environments. Instead of downloading mysql directly to the host machine (your instance) and having to deal with dependencies and uninstalling/fixing if anything goes wrong we can use the mysql image which we can easily create/remove a Docker container from.

The python application will also be containerized when run, but in order to give you a platform to iterate and develop this application on your own going forward we are going to include the source code and Dockerfile which will allow you to tweak the application, build the container back up to use it, and even [publish the container](https://docs.docker.com/docker-hub/) if you'd like.

Expand All @@ -112,7 +112,7 @@ The python application will also be containerized when run, but in order to give
```
sudo docker run hello-world
```
It will download a test container and run it. You should see an out similar to the one below:
It will download a test container and run it. You should see an output similar to the one below:
```
Hello from Docker!
This message shows that your installation appears to be working correctly.
Expand Down Expand Up @@ -174,7 +174,7 @@ When a container runs, it uses the various layers from an image for its filesyst
---
#### Part 7: Running the Application
---
We're all set and ready to run out application and database! From the `~/app` directory run:
We're all set and ready to run our application and database! From the `~/app` directory run:
```
$ sudo docker-compose up --build -d
```
Expand Down
8 changes: 4 additions & 4 deletions docs/tutorials/cumulus3.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

## CUmulus integration with CURC HPC

One potentially useful application of CUmulus is the ability to integrate your VMs with CU Research Computing High Performance Computing (HPC) resources. HPC compute is typically time-limited (at CURC 24 hours for regular joband 7 days long job) due finite resources and user competition for those resources. One way to deal with this problem is to schedule your jobs over time (e.g. by using cronjobs) though this isn't always practical for more complex workflows. Using authentication keys (in this case Java Web Tokens or JWTs) you can setup a connection from your CUmulus instance to CURC HPC and schedule jobs remotely to set up more complex workflow specific pipelines.
One potentially useful application of CUmulus is the ability to integrate your VMs with CU Research Computing High Performance Computing (HPC) resources. HPC compute is typically time-limited (at CURC 24 hours for regular job and 7 days long job) due to finite resources and user competition for those resources. One way to deal with this problem is to schedule your jobs over time (e.g. by using cronjobs) though this isn't always practical for more complex workflows. Using authentication keys (in this case Java Web Tokens or JWTs) you can setup a connection from your CUmulus instance to CURC HPC and schedule jobs remotely to set up more complex workflow specific pipelines.

We have documented the process for an **Ubuntu 20.04 instance on CUmulus connecting to the CURC Blanca cluster**. Below is an outline with links to specific sections:
1. Create your CUmulus instance
Expand All @@ -15,7 +15,7 @@ We have documented the process for an **Ubuntu 20.04 instance on CUmulus connect
### Instructions for Ubuntu 20.04:

---
#### Part 1 Create your CUmlus instance
#### Part 1 Create your CUmlus instance:
---
The first thing we will need to do is create a Ubuntu 20.04 CUmulus instance. Log in to the [CUmlus portal](https://cumulus.rc.colorado.edu) and follow our [tutorial1](./cumulus1) to create a CUmulus instance with the following specifications:
* Image: Ubuntu 20.04
Expand All @@ -40,7 +40,7 @@ The second thing we'll need to do is install SLURM on our CUmulus instance. To d
$ sudo apt-get update
$ sudo apt install -y libmysqlclient-dev libjwt-dev munge gcc make
```
* **Install SLURM**. It looks like there's a lot going on with this step, but all we're doing is downloading SLURM from github to the `/opt` directory of your instance, configuring the compilation to include Java Web Tokens (JWT) functionaliy, and then compiling and installing SLURM. Note that it is **VERY IMPORTANT** that the SLURM version on your CUmulus instance matches the CURC HPC version otherwise it will not connect.
* **Install SLURM**. It looks like there's a lot going on with this step, but all we're doing is downloading SLURM from github to the `/opt` directory of your instance, configuring the compilation to include Java Web Tokens (JWT) functionality, and then compiling and installing SLURM. Note that it is **VERY IMPORTANT** that the SLURM version on your CUmulus instance matches the CURC HPC version otherwise it will not connect.

> You can check the SLURM version on CURC HPC resources by loading your cluster specific SLURM module from a login node (either `module load slurm/summit`, `module load slurm/blanca`, or `module load slurm/alpine`) then checking the version of a SLURM command (e.g. `sbatch --version`).

Expand Down Expand Up @@ -99,7 +99,7 @@ Now that we have SLURM installed we can start to configure our instance to make
#### Part 4. Generate Java Web Token (JWT) in CURC HPC
---
* Next we will genrate the Java Web Token on a CURC login node. Keep in mind that these tokens are generated with an expiration. The max time CURC systems will keep a JWT is **#TODO**. Note the JWT output in order submit jobs.
> Note: In this example we are genrating this token on the Blanca cluster
> Note: In this example we are generating this token on the Blanca cluster
```
$ module load slurm/blanca
$ scontrol token lifespan=72000 #token with 2 hour duration
Expand Down
2 changes: 1 addition & 1 deletion docs/tutorials/cumulus4.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ sudo apt-get install sshfs
yum install fuse-sshfs
```

...or, if you are on a _Windows_ VM, see our documenation [here](https://curc.readthedocs.io/en/latest/storage/petalibrary/mounting.html?highlight=sshfs#sshfs-on-windows)
...or, if you are on a _Windows_ VM, see our documentation [here](https://curc.readthedocs.io/en/latest/storage/petalibrary/mounting.html?highlight=sshfs#sshfs-on-windows)

---

Expand Down

0 comments on commit aae2e2e

Please sign in to comment.