Skip to content

Commit

Permalink
improvement: added in 7 pages from docs v1
Browse files Browse the repository at this point in the history
  • Loading branch information
bgcurran authored and claymcleod committed Dec 28, 2020
1 parent 7003aa6 commit 9143816
Show file tree
Hide file tree
Showing 31 changed files with 600 additions and 0 deletions.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
67 changes: 67 additions & 0 deletions docs/genomics-platform/accounts-and-billing/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Accounts and Billing

St. Jude Cloud is built on top of [DNAnexus](https://www.dnanexus.com/), a data-analysis and management platform that specializes in the field of bioinformatics. All of our account management is shared between St. Jude Cloud and DNAnexus. In other words, if you have a DNAnexus account, you automatically have a St. Jude Cloud account, and vice versa.

Each new user receives a $50 credit upon creation of their St. Jude Cloud account with DNAnexus (see the [note](#billing-setup) in the Billing Setup section). If you use this initial credit and are interested in additional funding, please reach out to support@stjude.cloud as additional collaboration funds may be available.

!!!warning "St. Jude Employees"
The account creation and login process is slightly different if you are an internal user (you work at St. Jude). Internal users please go to the [intranet home page](https://home.stjude.org) and type 'Bioinformatics Self-Service' into the search bar. From there, click on the link that says 'Bioinformatics Self-Service on St. Jude Cloud' to access the internal guide to creating an account.
!!!





## St. Jude Employees
### Create An Account

Go to the [St. Jude Employee login page](https://auth.stjude.org/SecureAuth61/) and log in with your current St. Jude credentials.

!!!note
If you are unable to log in at this link, and you know you have been on St. Jude Cloud before, you may have already set up a DNAnexus account through the DNANexus log in page using your St. Jude email address. To continue using this account, you will need to log in through the [DNAnexus log in page](https://platform.dnanexus.com/login?client_id=sjcloudplatform). You can request that your accounts be merged by contacting [DNAnexus support](mailto:support@dnanexus.com).


!!!quote "Example Email to DNAnexus Support"
Hello DNAnexus support,

I am a St. Jude employee and I would like to have my account switched over so I can use my St. Jude credentials to log in.

Thank you,
!!!
!!!

### Billing

Go to the [Bioinformatics Self Service page](https://home.stjude.org/computational-biology/Pages/bioinformatics-self-service-cloud.aspx) on the Intranet for information on how to set up your billing account.

If your account is already set up and you would like to access your Billing information:

1. Click on the drop down next to your user name in the far right of the DNAnexus navigation bar, and select 'Profile'.
2. Click on 'Billing Account' from the tabs listed just under the navigation bar.


## Non-St. Jude Employees
### Create An Account

1. Go to the [St. Jude Cloud log in page](https://platform.dnanexus.com/register?client_id=sjcloudplatform) on DNAnexus.
2. Click "Create an Account".
3. Fill in your information.
4. On the Create New Account page, make sure to select "Microsoft Azure (westus)" as the Default Cloud Region.
5. Click 'CREATE ACCOUNT'

![](./create-DX-account.gif)

### Billing

1. Click on the drop down next to your user name in the far right of the DNAnexus navigation bar, and select 'Profile'.
2. Click on 'Billing Account' from the tabs listed just under the navigation bar.
3. Click on the green 'ADD BILLING INFO' button to the right of your account name.
![](./DX-setup_billing_ext.gif)
4. A window labeled 'ACCOUNT UPGRADE' will pop up. In this window agree to DNAnexus's Terms of Service, agree to DNAnexus's pricing model, review your account information, and finally enter your billing information.
5. Click 'Upgrade Account'. This will send an email to the individual listed as the billing contact requesting that they verify the change.

!!!note
On step 4, you must enter the billing contact's name, physical address, email address and phone number. You do not need to enter any credit card information. Once the billing contact has verified the account upgrade request, your account will be credited $50.

Please [contact us](mailto:support@stjude.cloud) for help if you encounter any problems creating an account.
!!!
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
152 changes: 152 additions & 0 deletions docs/genomics-platform/analyzing-data/command-line/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
# Command Line Interaction

Before you begin interacting with St. Jude Cloud Platform from the
command line, you'll need to understand some details on the underlying
architecture of the platform. The St. Jude Cloud Platform is built on
top of a genomics cloud ecosystem provided by [DNAnexus](https://www.dnanexus.com/).

## Overview

Workspaces in DNAnexus are organized by projects, which are essentially
folders in the cloud. Each data request and tool in St. Jude Cloud
creates its own unique cloud workspace (DNAnexus project). For instance,
a data request creates a DNAnexus project behind the scenes with the
same name as the request name you specify when you request data.

## Installation

Open-source software provided by DNAnexus called the [dx-toolkit](https://github.com/dnanexus/dx-toolkit) is
used to interact with the St. Jude Cloud Platform from the command line.
You can use this to create these projects, upload and download data, and
many other operations. You'll need to install that software on your
computer by following [this guide](https://documentation.dnanexus.com/downloads#DNAnexus-Platform-SDK).

!!!tip
A quickstart to getting up and running with the dx-toolkit:

1. Install Python 2.7.13+. Note that using the system-level Python is usually not a good idea (by default, system level Python is typically too old/does not support the latest security protocols required). You can install using [Anaconda](https://conda.io/docs/user-guide/getting-started.html) (recommended) or using the default [Python installer](https://www.python.org/downloads/).
2. Run `pip install dxpy`.
3. Type `dx --help` at the command line.
!!!

## A quick tour

### Logging in

To log in using the dx-toolkit, run the following command:

```bash
dx login --noprojects
# enter username and password when prompted
```

!!!note
If you are a St. Jude employee, you'll need to follow [this
guide](https://documentation.dnanexus.com/user/login-and-logout#generating-an-authentication-token) to log in instead.
!!!

### Selecting a project

First, you'll need to choose which cloud workspace you would like to
access. This depends on if you are downloading data from a request or
working with input/output files from a tool. You can see the workspaces
available to you by running the following command in your terminal:

```bash
dx select
```

This will present with a prompt similar to the below screenshot. A list
of your available cloud workspaces will be shown with a number out to
the left of each. You should enter the number corresponding to the
workspace you are wanting to interact with. In the example below, the
user has selected the Rapid RNA-Seq tool.

![](./select-project.png)

### Some useful commands

Moving data back and forth between the cloud and your local computer is
simple once you have selected the correct project for your tool.

You will find that many common Linux commands with `dx` prepended work as expected.

```bash
# list available files for the tool for the main folder
dx ls

# list all available files for the tool
dx find .

# list all commands
dx --help
```

## Uploading data

You can use the following process to upload data to be used by St. Jude
Cloud Platform tools:

1. First, click "View" on the tool you'd like to run from [this page](https://stjude.cloud/tools.html). In this example, we will choose the Rapid RNA-Seq tool.

2. If you have not already, click "Start" on the tool you'd like to run. This will create a cloud workspace for you to upload your data to with the same name as the tool.

![](./RapidRNASeq_startPage.png)

3. Open up your terminal application and select the cloud workspace with the same name as the tool you are trying to run.

![](./select-rapid-rnaseq.png)

4. Last, navigate to the local files you'd like to upload to the cloud and use the `dx upload` command as specified in [upload-download-data]{role="ref"} to upload your data to St. Jude Cloud.

![](./rapid-rnaseq-upload-data.png)

## Downloading data

!!!warning
To download data from a St. Jude Cloud data request, you must have
indicated that you wished to download the data in your Data Access
Agreement (DAA) during your submission. Any downloading of St. Jude data without completing this step is strictly PROHIBITED.
!!!

You can use the following steps to download data from a St. Jude Cloud
data request:

1. Complete a data request using the Genomics Platform application. See this [guide](../requesting-data/data-request.md) for instructions.

2. Open up your terminal application and select the cloud workspace
relevant to your data request. For instance, in this case we
would type `#!bash dx select "Retinoblastoma Data"`.

![](./select-retinoblastoma.png)

3. You can use typical commands like `#!bash dx ls`,
`#!bash dx pwd`, and `#!bash dx cd` to navigate around
your cloud folder as you would a local folder. Your project may look
different based on what data you requested and whether you were
previously approved to access the data. Your data should either be
in the **restricted** folder (if this is your first time
requesting access) or the **immediate** folder (if you
were previously granted access permission).

![](./navigate-data-request.png)

4. In the root of every data request is a file called
`SAMPLE_INFO.txt`. This should contain all of the
information about the samples you checked out as well as the
associated metadata we provide.

5. To download data from the cloud to local storage, use the
`#!bash dx download` command as specified in
[upload-download-data]{role="ref"}. For instance, if I wanted to
download all of the BAM files to my local computer, I would type
`#!bash dx download immediate/bam/*`.

![](./download-bams.png)

## Similar Topics

[About our Data](../requesting-data/about-our-data.md)
[Making a Data Request](../requesting-data/data-request.md)
[Working with our Data Overview](../managing-data/working-with-our-data.md)
[Upload/Download Data (local)](../managing-data/upload-local.md)
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
145 changes: 145 additions & 0 deletions docs/genomics-platform/faq/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
# Frequently Asked Questions

[TOC]

## Will I be charged for using St. Jude Cloud Genomics Platform?

Within St. Jude Cloud Genomics Platform, any St. Jude data you receive through a
data request is *sponsored*, meaning that you do not have to pay a fee to store
this data in St. Jude Cloud. You **will not** incur any costs except in the
following situations:

* Any *other* files, such as input files uploaded to or results produced by St.
Jude Cloud, will incur a monthly fee. See your [DNAnexus billing
information](https://platform.dnanexus.com/profile/settings/billing) for the
cost per GB.
* Any analysis workflows, including those provided by St. Jude or your own that
you have uploaded and packaged into the cloud, will incur a charge. The charge
depends on the underlying compute resources used and the amount of time taken.
Documentation for specific workflows we provide should contain guidance on how
much the workflow costs. See your [DNAnexus billing
information](https://platform.dnanexus.com/profile/settings/billing) for the
price of each VM size per hour.
* At the current time, downloading data is free to end-users. Note, however,
that downloading is not without cost to St. Jude. All cloud environments
charge an **egress fee** for anyone downloading data outside of the cloud. At
the current time, St. Jude has chosen to sponsor any egress fees associated
with downloading data. However, we reserve the right to alter this in the
future.

## How can I set up billing for my lab?

Billing setup is different based on whether you are an internal user (you work
at St. Jude) or an external user. If you are a St. Jude Employee, please refer
to the dedicated [intranet
page](https://home.stjude.org/computational-biology/Pages/bioinformatics-self-service-cloud.aspx)
for instructions. If you are not a St. Jude Employee, please refer to our
[Create an Account](./accounts-and-billing.md#billing-setup) guide.

## Does St. Jude Cloud allow for-profit companies to access genomics data?

We do not allow for-profit companies to access any of our restricted access genomics data. We are persistently working with our institution to create a path forward for companies. If you work for a for-profit company and would like to be notified if this rule changes, feel free to email us at
[support@stjude.cloud](mailto:support@stjude.cloud).

## Why do I need to sign the Data Access Agreement (DAA)?

The [data access
agreement](./requesting-data/glossary.md#data-access-agreement)
serves many purposes. Ultimately, the terms included in the data access
agreement are in place to protect our patients. We take patient security very
seriously, and we require that requesters are committed to protecting that
privacy to the fullest extent.

## How do I submit edits/revisions to the DAA?

We do not alter the terms of the data access agreement **for any reason** except
when the terms are found to be directly in conflict with state or national law.
In this case, please send a reference to law and a short description to
[support@stjude.cloud](mailto:support@stjude.cloud). Otherwise, please
understand that simply cannot manage the operational overhead of
differing agreements with different parties.

## Can I get a Microsoft Word version of the DAA?

We do not provide any editable format of the DAA, as we do not accept edits or
revisions from external parties.

## Where can I find the latest version of the Data Access Agreement (DAA)?

You can download the latest version of the DAA
[here](https://platform.stjude.cloud/access_form).

## Where do I submit the Data Access Agreement (DAA)?

You can submit your Data Access Agreement in the drag and drop box on the [last
step of the data request
process](./requesting-data/data-request.md#requesting-data).

## What if I did not fill out the Data Download Permission section of the original DAA, but now I want to download data?

This would be a change in terms from the original agreement, you would need to
fill out a new DAA (including the [Data Download Permission
section](./requesting-data/how-to-fill-out-DAA.md#the-data-access-agreement)
for any data sets you want to download.

## What clinical information is available about samples in St. Jude Cloud?

You can view the basic clinical and phenotypic information we currently provide
[here](./requesting-data/about-our-data.md#metadata).

## Can I gain access to further clinical information than what is currently available?

We are working towards being able to provide additional clinical annotations
such as treatment, outcome, and survival data in the future. Unfortunately, we
do not offer it today and we do not have a timeline for when it will be
available.

## Can I get a copy of IRB consent forms?

We do not provide individual consent forms or blank consent forms for any
samples on St. Jude Cloud. We have chosen to remain consistent with the
requirements of the other major genomic data repositories in that (1) there is
an internal vetting process by the St. Jude IRB to ensure samples may be shared
with the research community, but (2) we do not share the informed consents with
data requesters.

## Can I request FASTQ files on St. Jude Cloud?

We do not store FASTQ files in St. Jude Cloud because it would double the
storage cost without any benefit. Several tools exist that you can leverage to
revert BAM to FASTQ files — we recommend using Picard SamToFastq to revert BAM
files. You can efficiently revert BAMs to FASTQs in the cloud by wrapping the
conversion tool of your choice into a [Cloud
App](./analyzing-data/creating-a-cloud-app.md)

## How can I work with genomics data in the cloud?

You can view [this guide](./analyzing-data/creating-a-cloud-app.md) to learn how
create a cloud application.

## Why am I getting a connectivity error when connecting to DNAnexus API via SSH?

If you are trying to run something like
`$ dx run --ssh <executable> `
and are getting a connectivity error, it may be that your firewall is too
restrictive. Are you able to perform the command from an unrestricted network
(like a home network)? If yes, you can resolve this issue by asking your network
administrator to whitelist connections to Azure US West. All subnets (Region
Name="uswest") are provided
[here](https://www.microsoft.com/en-us/download/details.aspx?id=41653).

## How can I delete my account?

Today, a St. Jude Cloud Genomics Platform account is simply a DNAnexus account.
Thus, of you'd like to delete your account, you'll need to email DNAnexus asking
for it to be removed. You can do so by contacting DNAnexus support at
<support@dnanexus.com> with the following email.

!!!note "Subject: St. Jude Cloud account deletion"

Hi DNAnexus,

Would you please assist me in deleting my St. Jude Cloud account? My username is _____.

Thank you!
!!!
Binary file added docs/genomics-platform/getting-started/1.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/genomics-platform/getting-started/2.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 9143816

Please sign in to comment.