-
Notifications
You must be signed in to change notification settings - Fork 19
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
improvement: added in 7 pages from docs v1
- Loading branch information
1 parent
7003aa6
commit 9143816
Showing
31 changed files
with
600 additions
and
0 deletions.
There are no files selected for viewing
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
# Accounts and Billing | ||
|
||
St. Jude Cloud is built on top of [DNAnexus](https://www.dnanexus.com/), a data-analysis and management platform that specializes in the field of bioinformatics. All of our account management is shared between St. Jude Cloud and DNAnexus. In other words, if you have a DNAnexus account, you automatically have a St. Jude Cloud account, and vice versa. | ||
|
||
Each new user receives a $50 credit upon creation of their St. Jude Cloud account with DNAnexus (see the [note](#billing-setup) in the Billing Setup section). If you use this initial credit and are interested in additional funding, please reach out to support@stjude.cloud as additional collaboration funds may be available. | ||
|
||
!!!warning "St. Jude Employees" | ||
The account creation and login process is slightly different if you are an internal user (you work at St. Jude). Internal users please go to the [intranet home page](https://home.stjude.org) and type 'Bioinformatics Self-Service' into the search bar. From there, click on the link that says 'Bioinformatics Self-Service on St. Jude Cloud' to access the internal guide to creating an account. | ||
!!! | ||
|
||
|
||
|
||
|
||
|
||
## St. Jude Employees | ||
### Create An Account | ||
|
||
Go to the [St. Jude Employee login page](https://auth.stjude.org/SecureAuth61/) and log in with your current St. Jude credentials. | ||
|
||
!!!note | ||
If you are unable to log in at this link, and you know you have been on St. Jude Cloud before, you may have already set up a DNAnexus account through the DNANexus log in page using your St. Jude email address. To continue using this account, you will need to log in through the [DNAnexus log in page](https://platform.dnanexus.com/login?client_id=sjcloudplatform). You can request that your accounts be merged by contacting [DNAnexus support](mailto:support@dnanexus.com). | ||
|
||
|
||
!!!quote "Example Email to DNAnexus Support" | ||
Hello DNAnexus support, | ||
|
||
I am a St. Jude employee and I would like to have my account switched over so I can use my St. Jude credentials to log in. | ||
|
||
Thank you, | ||
!!! | ||
!!! | ||
|
||
### Billing | ||
|
||
Go to the [Bioinformatics Self Service page](https://home.stjude.org/computational-biology/Pages/bioinformatics-self-service-cloud.aspx) on the Intranet for information on how to set up your billing account. | ||
|
||
If your account is already set up and you would like to access your Billing information: | ||
|
||
1. Click on the drop down next to your user name in the far right of the DNAnexus navigation bar, and select 'Profile'. | ||
2. Click on 'Billing Account' from the tabs listed just under the navigation bar. | ||
|
||
|
||
## Non-St. Jude Employees | ||
### Create An Account | ||
|
||
1. Go to the [St. Jude Cloud log in page](https://platform.dnanexus.com/register?client_id=sjcloudplatform) on DNAnexus. | ||
2. Click "Create an Account". | ||
3. Fill in your information. | ||
4. On the Create New Account page, make sure to select "Microsoft Azure (westus)" as the Default Cloud Region. | ||
5. Click 'CREATE ACCOUNT' | ||
|
||
![](./create-DX-account.gif) | ||
|
||
### Billing | ||
|
||
1. Click on the drop down next to your user name in the far right of the DNAnexus navigation bar, and select 'Profile'. | ||
2. Click on 'Billing Account' from the tabs listed just under the navigation bar. | ||
3. Click on the green 'ADD BILLING INFO' button to the right of your account name. | ||
![](./DX-setup_billing_ext.gif) | ||
4. A window labeled 'ACCOUNT UPGRADE' will pop up. In this window agree to DNAnexus's Terms of Service, agree to DNAnexus's pricing model, review your account information, and finally enter your billing information. | ||
5. Click 'Upgrade Account'. This will send an email to the individual listed as the billing contact requesting that they verify the change. | ||
|
||
!!!note | ||
On step 4, you must enter the billing contact's name, physical address, email address and phone number. You do not need to enter any credit card information. Once the billing contact has verified the account upgrade request, your account will be credited $50. | ||
|
||
Please [contact us](mailto:support@stjude.cloud) for help if you encounter any problems creating an account. | ||
!!! |
Binary file added
BIN
+403 KB
docs/genomics-platform/analyzing-data/command-line/RapidRNASeq_startPage.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
152 changes: 152 additions & 0 deletions
152
docs/genomics-platform/analyzing-data/command-line/index.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,152 @@ | ||
# Command Line Interaction | ||
|
||
Before you begin interacting with St. Jude Cloud Platform from the | ||
command line, you'll need to understand some details on the underlying | ||
architecture of the platform. The St. Jude Cloud Platform is built on | ||
top of a genomics cloud ecosystem provided by [DNAnexus](https://www.dnanexus.com/). | ||
|
||
## Overview | ||
|
||
Workspaces in DNAnexus are organized by projects, which are essentially | ||
folders in the cloud. Each data request and tool in St. Jude Cloud | ||
creates its own unique cloud workspace (DNAnexus project). For instance, | ||
a data request creates a DNAnexus project behind the scenes with the | ||
same name as the request name you specify when you request data. | ||
|
||
## Installation | ||
|
||
Open-source software provided by DNAnexus called the [dx-toolkit](https://github.com/dnanexus/dx-toolkit) is | ||
used to interact with the St. Jude Cloud Platform from the command line. | ||
You can use this to create these projects, upload and download data, and | ||
many other operations. You'll need to install that software on your | ||
computer by following [this guide](https://documentation.dnanexus.com/downloads#DNAnexus-Platform-SDK). | ||
|
||
!!!tip | ||
A quickstart to getting up and running with the dx-toolkit: | ||
|
||
1. Install Python 2.7.13+. Note that using the system-level Python is usually not a good idea (by default, system level Python is typically too old/does not support the latest security protocols required). You can install using [Anaconda](https://conda.io/docs/user-guide/getting-started.html) (recommended) or using the default [Python installer](https://www.python.org/downloads/). | ||
2. Run `pip install dxpy`. | ||
3. Type `dx --help` at the command line. | ||
!!! | ||
|
||
## A quick tour | ||
|
||
### Logging in | ||
|
||
To log in using the dx-toolkit, run the following command: | ||
|
||
```bash | ||
dx login --noprojects | ||
# enter username and password when prompted | ||
``` | ||
|
||
!!!note | ||
If you are a St. Jude employee, you'll need to follow [this | ||
guide](https://documentation.dnanexus.com/user/login-and-logout#generating-an-authentication-token) to log in instead. | ||
!!! | ||
|
||
### Selecting a project | ||
|
||
First, you'll need to choose which cloud workspace you would like to | ||
access. This depends on if you are downloading data from a request or | ||
working with input/output files from a tool. You can see the workspaces | ||
available to you by running the following command in your terminal: | ||
|
||
```bash | ||
dx select | ||
``` | ||
|
||
This will present with a prompt similar to the below screenshot. A list | ||
of your available cloud workspaces will be shown with a number out to | ||
the left of each. You should enter the number corresponding to the | ||
workspace you are wanting to interact with. In the example below, the | ||
user has selected the Rapid RNA-Seq tool. | ||
|
||
![](./select-project.png) | ||
|
||
### Some useful commands | ||
|
||
Moving data back and forth between the cloud and your local computer is | ||
simple once you have selected the correct project for your tool. | ||
|
||
You will find that many common Linux commands with `dx` prepended work as expected. | ||
|
||
```bash | ||
# list available files for the tool for the main folder | ||
dx ls | ||
|
||
# list all available files for the tool | ||
dx find . | ||
|
||
# list all commands | ||
dx --help | ||
``` | ||
|
||
## Uploading data | ||
|
||
You can use the following process to upload data to be used by St. Jude | ||
Cloud Platform tools: | ||
|
||
1. First, click "View" on the tool you'd like to run from [this page](https://stjude.cloud/tools.html). In this example, we will choose the Rapid RNA-Seq tool. | ||
|
||
2. If you have not already, click "Start" on the tool you'd like to run. This will create a cloud workspace for you to upload your data to with the same name as the tool. | ||
|
||
![](./RapidRNASeq_startPage.png) | ||
|
||
3. Open up your terminal application and select the cloud workspace with the same name as the tool you are trying to run. | ||
|
||
![](./select-rapid-rnaseq.png) | ||
|
||
4. Last, navigate to the local files you'd like to upload to the cloud and use the `dx upload` command as specified in [upload-download-data]{role="ref"} to upload your data to St. Jude Cloud. | ||
|
||
![](./rapid-rnaseq-upload-data.png) | ||
|
||
## Downloading data | ||
|
||
!!!warning | ||
To download data from a St. Jude Cloud data request, you must have | ||
indicated that you wished to download the data in your Data Access | ||
Agreement (DAA) during your submission. Any downloading of St. Jude data without completing this step is strictly PROHIBITED. | ||
!!! | ||
|
||
You can use the following steps to download data from a St. Jude Cloud | ||
data request: | ||
|
||
1. Complete a data request using the Genomics Platform application. See this [guide](../requesting-data/data-request.md) for instructions. | ||
|
||
2. Open up your terminal application and select the cloud workspace | ||
relevant to your data request. For instance, in this case we | ||
would type `#!bash dx select "Retinoblastoma Data"`. | ||
|
||
![](./select-retinoblastoma.png) | ||
|
||
3. You can use typical commands like `#!bash dx ls`, | ||
`#!bash dx pwd`, and `#!bash dx cd` to navigate around | ||
your cloud folder as you would a local folder. Your project may look | ||
different based on what data you requested and whether you were | ||
previously approved to access the data. Your data should either be | ||
in the **restricted** folder (if this is your first time | ||
requesting access) or the **immediate** folder (if you | ||
were previously granted access permission). | ||
|
||
![](./navigate-data-request.png) | ||
|
||
4. In the root of every data request is a file called | ||
`SAMPLE_INFO.txt`. This should contain all of the | ||
information about the samples you checked out as well as the | ||
associated metadata we provide. | ||
|
||
5. To download data from the cloud to local storage, use the | ||
`#!bash dx download` command as specified in | ||
[upload-download-data]{role="ref"}. For instance, if I wanted to | ||
download all of the BAM files to my local computer, I would type | ||
`#!bash dx download immediate/bam/*`. | ||
|
||
![](./download-bams.png) | ||
|
||
## Similar Topics | ||
|
||
[About our Data](../requesting-data/about-our-data.md) | ||
[Making a Data Request](../requesting-data/data-request.md) | ||
[Working with our Data Overview](../managing-data/working-with-our-data.md) | ||
[Upload/Download Data (local)](../managing-data/upload-local.md) |
Binary file added
BIN
+112 KB
docs/genomics-platform/analyzing-data/command-line/navigate-data-request.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+79.1 KB
docs/genomics-platform/analyzing-data/command-line/rapid-rnaseq-upload-data.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+67.7 KB
docs/genomics-platform/analyzing-data/command-line/select-project.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+140 KB
docs/genomics-platform/analyzing-data/command-line/select-rapid-rnaseq.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+63.1 KB
docs/genomics-platform/analyzing-data/command-line/select-retinoblastoma.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,145 @@ | ||
# Frequently Asked Questions | ||
|
||
[TOC] | ||
|
||
## Will I be charged for using St. Jude Cloud Genomics Platform? | ||
|
||
Within St. Jude Cloud Genomics Platform, any St. Jude data you receive through a | ||
data request is *sponsored*, meaning that you do not have to pay a fee to store | ||
this data in St. Jude Cloud. You **will not** incur any costs except in the | ||
following situations: | ||
|
||
* Any *other* files, such as input files uploaded to or results produced by St. | ||
Jude Cloud, will incur a monthly fee. See your [DNAnexus billing | ||
information](https://platform.dnanexus.com/profile/settings/billing) for the | ||
cost per GB. | ||
* Any analysis workflows, including those provided by St. Jude or your own that | ||
you have uploaded and packaged into the cloud, will incur a charge. The charge | ||
depends on the underlying compute resources used and the amount of time taken. | ||
Documentation for specific workflows we provide should contain guidance on how | ||
much the workflow costs. See your [DNAnexus billing | ||
information](https://platform.dnanexus.com/profile/settings/billing) for the | ||
price of each VM size per hour. | ||
* At the current time, downloading data is free to end-users. Note, however, | ||
that downloading is not without cost to St. Jude. All cloud environments | ||
charge an **egress fee** for anyone downloading data outside of the cloud. At | ||
the current time, St. Jude has chosen to sponsor any egress fees associated | ||
with downloading data. However, we reserve the right to alter this in the | ||
future. | ||
|
||
## How can I set up billing for my lab? | ||
|
||
Billing setup is different based on whether you are an internal user (you work | ||
at St. Jude) or an external user. If you are a St. Jude Employee, please refer | ||
to the dedicated [intranet | ||
page](https://home.stjude.org/computational-biology/Pages/bioinformatics-self-service-cloud.aspx) | ||
for instructions. If you are not a St. Jude Employee, please refer to our | ||
[Create an Account](./accounts-and-billing.md#billing-setup) guide. | ||
|
||
## Does St. Jude Cloud allow for-profit companies to access genomics data? | ||
|
||
We do not allow for-profit companies to access any of our restricted access genomics data. We are persistently working with our institution to create a path forward for companies. If you work for a for-profit company and would like to be notified if this rule changes, feel free to email us at | ||
[support@stjude.cloud](mailto:support@stjude.cloud). | ||
|
||
## Why do I need to sign the Data Access Agreement (DAA)? | ||
|
||
The [data access | ||
agreement](./requesting-data/glossary.md#data-access-agreement) | ||
serves many purposes. Ultimately, the terms included in the data access | ||
agreement are in place to protect our patients. We take patient security very | ||
seriously, and we require that requesters are committed to protecting that | ||
privacy to the fullest extent. | ||
|
||
## How do I submit edits/revisions to the DAA? | ||
|
||
We do not alter the terms of the data access agreement **for any reason** except | ||
when the terms are found to be directly in conflict with state or national law. | ||
In this case, please send a reference to law and a short description to | ||
[support@stjude.cloud](mailto:support@stjude.cloud). Otherwise, please | ||
understand that simply cannot manage the operational overhead of | ||
differing agreements with different parties. | ||
|
||
## Can I get a Microsoft Word version of the DAA? | ||
|
||
We do not provide any editable format of the DAA, as we do not accept edits or | ||
revisions from external parties. | ||
|
||
## Where can I find the latest version of the Data Access Agreement (DAA)? | ||
|
||
You can download the latest version of the DAA | ||
[here](https://platform.stjude.cloud/access_form). | ||
|
||
## Where do I submit the Data Access Agreement (DAA)? | ||
|
||
You can submit your Data Access Agreement in the drag and drop box on the [last | ||
step of the data request | ||
process](./requesting-data/data-request.md#requesting-data). | ||
|
||
## What if I did not fill out the Data Download Permission section of the original DAA, but now I want to download data? | ||
|
||
This would be a change in terms from the original agreement, you would need to | ||
fill out a new DAA (including the [Data Download Permission | ||
section](./requesting-data/how-to-fill-out-DAA.md#the-data-access-agreement) | ||
for any data sets you want to download. | ||
|
||
## What clinical information is available about samples in St. Jude Cloud? | ||
|
||
You can view the basic clinical and phenotypic information we currently provide | ||
[here](./requesting-data/about-our-data.md#metadata). | ||
|
||
## Can I gain access to further clinical information than what is currently available? | ||
|
||
We are working towards being able to provide additional clinical annotations | ||
such as treatment, outcome, and survival data in the future. Unfortunately, we | ||
do not offer it today and we do not have a timeline for when it will be | ||
available. | ||
|
||
## Can I get a copy of IRB consent forms? | ||
|
||
We do not provide individual consent forms or blank consent forms for any | ||
samples on St. Jude Cloud. We have chosen to remain consistent with the | ||
requirements of the other major genomic data repositories in that (1) there is | ||
an internal vetting process by the St. Jude IRB to ensure samples may be shared | ||
with the research community, but (2) we do not share the informed consents with | ||
data requesters. | ||
|
||
## Can I request FASTQ files on St. Jude Cloud? | ||
|
||
We do not store FASTQ files in St. Jude Cloud because it would double the | ||
storage cost without any benefit. Several tools exist that you can leverage to | ||
revert BAM to FASTQ files — we recommend using Picard SamToFastq to revert BAM | ||
files. You can efficiently revert BAMs to FASTQs in the cloud by wrapping the | ||
conversion tool of your choice into a [Cloud | ||
App](./analyzing-data/creating-a-cloud-app.md) | ||
|
||
## How can I work with genomics data in the cloud? | ||
|
||
You can view [this guide](./analyzing-data/creating-a-cloud-app.md) to learn how | ||
create a cloud application. | ||
|
||
## Why am I getting a connectivity error when connecting to DNAnexus API via SSH? | ||
|
||
If you are trying to run something like | ||
`$ dx run --ssh <executable> ` | ||
and are getting a connectivity error, it may be that your firewall is too | ||
restrictive. Are you able to perform the command from an unrestricted network | ||
(like a home network)? If yes, you can resolve this issue by asking your network | ||
administrator to whitelist connections to Azure US West. All subnets (Region | ||
Name="uswest") are provided | ||
[here](https://www.microsoft.com/en-us/download/details.aspx?id=41653). | ||
|
||
## How can I delete my account? | ||
|
||
Today, a St. Jude Cloud Genomics Platform account is simply a DNAnexus account. | ||
Thus, of you'd like to delete your account, you'll need to email DNAnexus asking | ||
for it to be removed. You can do so by contacting DNAnexus support at | ||
<support@dnanexus.com> with the following email. | ||
|
||
!!!note "Subject: St. Jude Cloud account deletion" | ||
|
||
Hi DNAnexus, | ||
|
||
Would you please assist me in deleting my St. Jude Cloud account? My username is _____. | ||
|
||
Thank you! | ||
!!! |
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.