Skip to content

Commit

Permalink
added overview and cleaned up other pages
Browse files Browse the repository at this point in the history
  • Loading branch information
krokicki committed May 11, 2022
1 parent b2f2cec commit aaf56b6
Show file tree
Hide file tree
Showing 9 changed files with 59 additions and 63 deletions.
30 changes: 7 additions & 23 deletions content/docs/Overview/_index.md
Expand Up @@ -5,36 +5,20 @@ weight: 1
description: >
What is HortaCloud?
---

HortaCloud is a streaming 3D annotation platform for large microscopy data that runs entirely in the cloud. It is based on the Janelia Workstation software and was originally developed in support of the MouseLight Team Project.
HortaCloud is a streaming 3D annotation platform for large microscopy data that runs entirely in the cloud. It is a free, open source research software tool, developed by Janelia Research Campus.

It combines state-of-the-art volumetric visualization, advanced features for 3D neuronal annotation, and real-time multi-user collaboration with a set of enterprise-grade backend microservices for moving and processing large amounts of data rapidly and securely. HortaCloud takes advantage of cloud-based Virtual Desktop Infrastructure (VDI) to perform all 3D rendering in cloud-leased GPUs which are data-adjacent, and only transfer a high-fidelity interactive video stream to each annotator’s local compute platform through a web browser.

{{< imglink src="system_architecture.png" link="system_architecture.png" alt="system architecture diagram" width="700px" >}}

The Overview is where your users find out about your project. Depending on the size of your docset, you can have a separate overview page (like this one) or put your overview contents in the Documentation landing page (like in the Docsy User Guide).

Try answering these questions for your user in this page:

## What is it?
* **What is it good for?**: HortaCloud is a powerful tool for 3D visualization and annotation of large-scale microscopy data. It has been used to trace axons across the entire mouse brain by Janelia's [MouseLight Team Project](https://www.janelia.org/project-team/mouselight).

Introduce your project, including what it does or lets you do, why you would use it, and its primary goal (and how it achieves it). This should be similar to your README description, though you can go into a little more detail here if you want.
* **What is it not good for?**: HortaCloud is a very specialized tool for sparse microscopy data. It is not intended for annotation of dense data sets, such as electron microscopy (EM) imagery.

## Why do I want it?
* **What makes HortaCloud unique?**: Leveraging state-of-the-art services on AWS allows HortaCloud to run entirely in the cloud, making it possible to visualize terabyte-scale 3D volumes without moving all of that data over the Internet (see diagram below).

Help your user know if your project will help them. Useful information can include:

* **What is it good for?**: What types of problems does your project solve? What are the benefits of using it?

* **What is it not good for?**: For example, point out situations that might intuitively seem suited for your project, but aren't for some reason. Also mention known limitations, scaling issues, or anything else that might let your users know if the project is not for them.

* **What is it *not yet* good for?**: Highlight any useful features that are coming soon.
{{< imglink src="system_architecture.png" link="system_architecture.png" alt="system architecture diagram" width="500px" >}}

## Where should I go next?

If you are a HortaCloud user, read through the [User Manual](/docs/UserManual) to get familiar with the tools.

If you are a system administrator or developer looking to deploy an instance of HortaCloud, read the [Deployment Guide](/docs/Administration/Deployment).
If you are a HortaCloud user, read through the [User Manual](/docs/user_manual) to get familiar with the tools.

* [Getting Started](/docs/getting-started/): Get started with $project
* [Examples](/docs/examples/): Check out some example code!
If you are a system administrator or developer looking to deploy an instance of HortaCloud, start with the [AWS Deployment Guide](/docs/administration/aws).
2 changes: 0 additions & 2 deletions content/docs/_index.md
@@ -1,6 +1,4 @@

---

title: "Documentation"
linkTitle: "Documentation"
weight: 20
Expand Down
15 changes: 1 addition & 14 deletions content/docs/administration/aws/Operation.md
Expand Up @@ -16,7 +16,7 @@ Open the Data Explorer (**Window** → **Core** → **Data Explorer**) and navig

### Batch Sample Import

The workstation offers an option to import all your samples from an S3 Bucket mapped on the backend into the workstation. This can be done from the Workstation, select **Services****Load Horta Data**. This opens a [dialog box](docs/images/horta_data_import.png) that allows you to enter the location of your samples and the data owner. The location of your data can be entered like `<s3_bucket_name>/<full_mouselight_data_prefix>`, e.g. `janelia-mouselight-imagery/images`. For sample owner typically select `Mouselight Users (mouselight)` from the drop down selection. Once you click OK the system will start a background task that will try to create new samples from all folders at the specified location and you will start seeing new samples under `Home (mouselight)/3D Tile Microscope Samples` as they are imported.
The workstation offers an option to import all your samples from an S3 Bucket mapped on the backend into the workstation. This can be done from the Workstation, select **Services****Load Horta Data**. This opens a [dialog box](docs/images/horta_data_import.png) that allows you to enter the location of your samples and the data owner. The location of your data can be entered like `<s3_bucket_name>/<full_moouselight_data_prefix>`, e.g. `janelia-mouselight-imagery/images`. For sample owner typically select `Mouselight Users (mouselight)` from the drop down selection. Once you click OK the system will start a background task that will try to create new samples from all folders at the specified location and you will start seeing new samples under `Home (mouselight)/3D Tile Microscope Samples` as they are imported.

## System backup

Expand All @@ -25,16 +25,3 @@ The system can be configured to run nightly backups. All it is needed is to spec
## System restore

If system backups are available the sample and tracing data can be restored from a specified backup bucket (HORTA_RESTORE_BUCKET) and prefix (HORTA_RESTORE_FOLDER) which typically was created by the nightly backup job.

## Troubleshooting

### Troubleshooting client app installation

If the client app installation fails for any reason, before you attempt the install again you must remove everything that was installed by the install script. Uninstall all applications installed with scoop and remove the 'C:\apps' folder. To do that run:

```powershell
scoop uninstall scoop
del c:\apps
```

When prompted whether you really want to uninstall everything, select "yes" or "all".
3 changes: 2 additions & 1 deletion content/docs/administration/aws/_index.md
Expand Up @@ -2,8 +2,9 @@
title: "AWS"
linkTitle: "AWS"
weight: 10
simple_list: true
description: >
Learn how to deploy and operate your own HortaCloud instance on AWS
---

HortaCloud is easily deployed on Amazon Web Services (AWS) using the AWS CDK to automatically provision resources.
HortaCloud is easily deployed on Amazon Web Services (AWS) using the AWS CDK to automatically provision resources.
13 changes: 13 additions & 0 deletions content/docs/administration/aws/deployment.md
Expand Up @@ -203,3 +203,16 @@ By default the application will have a very long url that is not easy to remembe
* "Viewer protocol policy" - Change this to "Redirect HTTP to HTTPS"
* "Custom SSL certificate" - Select the certificate that you registered with AWS Certificate Manager
* Finally, click the "Create distribution" button.

## Troubleshooting

### Troubleshooting client app installation

If the client app installation fails for any reason, before you attempt the install again you must remove everything that was installed by the install script. Uninstall all applications installed with scoop and remove the 'C:\apps' folder. To do that run:

```powershell
scoop uninstall scoop
del c:\apps
```

When prompted whether you really want to uninstall everything, select "yes" or "all".
6 changes: 3 additions & 3 deletions content/docs/administration/bare_metal/SwarmDeployment.md
Expand Up @@ -48,7 +48,7 @@ The JACS system has its own self-contained authentication system, and can manage

If you'd prefer that users authenticate against your existing LDAP or ActiveDirectory server, edit `$CONFIG_DIR/jacs-sync/jacs.properties` and add these properties:

```bash
```properties
LDAP.URL=
LDAP.SearchBase=
LDAP.SearchFilter=
Expand Down Expand Up @@ -191,7 +191,7 @@ services:

This will expose the path to both JADE agent containers. Now you need to configure the JADE agents to serve this data. On both hosts, edit /opt/jacs/config/jade/config.properties and add the following:

```bash
```properties
StorageVolume.mouseLightNFS.RootDir=/path/to/your/nfs
StorageVolume.mouseLightNFS.VirtualPath=/path/to/your/nfs
StorageVolume.mouseLightNFS.Shared=true
Expand All @@ -201,7 +201,7 @@ StorageVolume.mouseLightNFS.VolumePermissions=READ

You can use any name you want instead of mouseLightNFS. Then you should add this name to StorageAgent.BootstrappedVolumes:

```bash
```properties
StorageAgent.BootstrappedVolumes=jade1,mouseLightNFS
```

Expand Down
46 changes: 29 additions & 17 deletions content/docs/administration/bare_metal/TwoServerDeployment.md
Expand Up @@ -47,42 +47,54 @@ To install Docker and Docker Compose on Oracle Linux 8, follow [these instructio

On **HOST1**, bring up swarm as a manager node:

docker swarm init
```bash
docker swarm init
```

On **HOST2**, copy and paste the output of the previous command to join the swarm as a worker.

docker swarm join --token ...
```bash
docker swarm join --token ...
```

All further commands should be executed on **HOST1**, i.e. the master node. One final step is to label the nodes. Each node needs the "jacs=true" label, as well as "jacs_name=nodeX".

docker node update --label-add jacs_name=node1 $(docker node ls -f "role=manager" --format "{{.ID}}")
docker node update --label-add jacs_name=node2 $(docker node ls -f "role=worker" --format "{{.ID}}")
docker node update --label-add jacs=true $(docker node ls -f "role=manager" --format "{{.ID}}")
docker node update --label-add jacs=true $(docker node ls -f "role=worker" --format "{{.ID}}")
```bash
docker node update --label-add jacs_name=node1 $(docker node ls -f "role=manager" --format "{{.ID}}")
docker node update --label-add jacs_name=node2 $(docker node ls -f "role=worker" --format "{{.ID}}")
docker node update --label-add jacs=true $(docker node ls -f "role=manager" --format "{{.ID}}")
docker node update --label-add jacs=true $(docker node ls -f "role=worker" --format "{{.ID}}")
```

Finally, you can run this command to ensure that both nodes are up and in Ready status:

docker node ls
```bash
docker node ls
```

## Download the installer

Download the installer and extract it onto the master node, as follows. `VERSION` should be set to the [latest stable version](https://github.com/JaneliaSciComp/jacs-cm/releases) available on the releases page.

export VERSION=<version_number_here>
cd /opt
sudo mkdir deploy
sudo chown $USER deploy
cd deploy
curl https://codeload.github.com/JaneliaSciComp/jacs-cm/tar.gz/$VERSION | tar xvz
ln -s jacs-cm-$VERSION jacs-cm
cd jacs-cm
```bash
export VERSION=<version_number_here>
cd /opt
sudo mkdir deploy
sudo chown $USER deploy
cd deploy
curl https://codeload.github.com/JaneliaSciComp/jacs-cm/tar.gz/$VERSION | tar xvz
ln -s jacs-cm-$VERSION jacs-cm
cd jacs-cm
```

## Configure The System

Next, create a `.env.config` file inside the intaller directory. This file defines the environment (usernames, passwords, etc.) You can copy the template to get started:

cp .env.template .env.config
vi .env.config
```bash
cp .env.template .env.config
vi .env.config
```

At minimum, you must customize the following:

Expand Down
3 changes: 2 additions & 1 deletion content/docs/administration/bare_metal/_index.md
Expand Up @@ -2,10 +2,11 @@
title: "Bare metal"
linkTitle: "Bare metal"
weight: 20
simple_list: true
description: >
Learn how to deploy a Horta instance on your own servers
---

The recommended way to deploy HortaCloud is to use [Amazon Web Services (AWS)](aws), but you can also deploy the Horta services locally on your own client/server machines.
The recommended way to deploy HortaCloud is to use [Amazon Web Services (AWS)](aws), but you can also deploy the Horta services locally on your own client/server machines.

Using Docker Swarm, you can choose to deploy on two or three servers. The primary benefit of using three servers is that the MongoDB cluster will have a complete replica set for high availability.
4 changes: 2 additions & 2 deletions content/docs/development/ComposeDeployment.md
Expand Up @@ -82,7 +82,7 @@ The JACS system has its own self-contained authentication system, and can manage

If you'd prefer that users authenticate against your existing LDAP or ActiveDirectory server, edit `$CONFIG_DIR/jacs-sync/jacs.properties` and add these properties:

```bash
```properties
LDAP.URL=
LDAP.SearchBase=
LDAP.SearchFilter=
Expand Down Expand Up @@ -144,6 +144,6 @@ Now you can checkout the [Janelia Workstation](https://github.com/JaneliaSciComp

The client will ask you for the API Gateway URL, which is just `http://$HOST1`. In order to automatically connect to your standalone gateway instance, you can create a new file at `workstation/modules/Core/src/main/resources/my.properties` with this content (replacing the variables with the values from your .env.config file):

```bash
```properties
api.gateway=https://$HOST1
```

0 comments on commit aaf56b6

Please sign in to comment.