Auto provision resources on AWS scicomp account. Cloudformation templates in this repo build on top of CF templates in Sage-Bionetworks/scicomp-infra repo.
Note - This project depends on CF templates from other accounts.
The workflow to provision AWS resources is done using pull requests. PRs provide history, gating, and a way to review and approve resource requests.
Instructions and workflow to auto provision and de-provision an EC2 is in Example PR: Auto provision an EC2 instance
Merging the above should create an EC2 instance and join the instance to a Sage Jumpcloud "system group" identified by $JcSystemsGroupId. Jumpcloud "User groups" that have access to $JcSystemsGroupId will have access to the provisioned instance.
We allow provisioning based on custom AMIs. List of Sage IT managed AMIs:
| AMI ID | Distribution | Volume | Comment |
|---|---|---|---|
| ami-082278746f893d99c | AWS linux 2 | 8GB encrypted boot volume | Default AMI |
We use a directory service Jumpcloud to manage user access to EC2 instances.
Find system groups by using the Jumpcloud API:
curl -X GET https://console.jumpcloud.com/api/v2/systemgroups \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-H 'x-api-key: abcd111122223333aaaabbbbccccddddeeeeffff'
Find systems by using the Jumpcloud API:
curl -X GET https://console.jumpcloud.com/api/systems \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-H 'x-api-key: abcd111122223333aaaabbbbccccddddeeeeffff'
Instructions and workflow to auto provision a Synapse external S3 bucket can be found in Example PR: Auto provision a synapse bucket
Merging the above should create a synapse bucket with the configurations defined in the documentation.
We have configured Travis to deploy CF template updates. Travis deploys using sceptre
- We use the AWS SSM to store secrets for this project. Sceptre retrieves the secrets using a sceptre ssm resolver and passes them to the cloudformation stack on deployment.