Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bsweger/update readme remove statementid #44

Merged
merged 2 commits into from
May 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 37 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,43 @@ Each cloud-enabled hub requires several dedicated AWS resources. These resources

## Onboarding a Hub

TODO: Add detailed instructions for onboarding a hub to the cloud.
1. Update the `hubs.yaml` file with the hub's name and the desired AWS region.
2. Submit a pull request
Currently, this infrastructure repository is not integrated with actual hub repositories. In other words, it doesn't pull information (_e.g._, the S3 bucket name) from a hub's `admin.json` configuration file. Thus, to host a hub in the Hubverse AWS account, we need to manually add its information to [`hubs.yaml`](src/hubverse_infrastructure/hubs/hubs.yaml).

1. Add a new `hub` entry to [`hubs.yaml`](src/hubverse_infrastructure/hubs/hubs.yaml):
- `hub` key: the name of the hub (this value will be used as the S3 bucket name, so make sure it's something unique)
- `org`: the GitHub organization that hosts the hub's repository
- `repo`: the name of the hub's repository

For example:
```yaml
- hub: flusight-forecast
org: cdcepi
repo: FluSight-forecast-hub
```

2. Submit the above changes as a pull request (PR) to this repository.
3. Shortly after the PR is opened, Pulumi will add a comment about the AWS changes it will make once the PR is merged.

For example:

```
Name Type Operation
+ flusight-forecast aws:iam/role:Role create
+ flusight-forecast-write-bucket-policy aws:iam/policy:Policy create
+ flusight-forecast-allow aws:lambda/permission:Permission create
+ flusight-forecast-transform-model-output-lambda aws:iam/rolePolicyAttachment:RolePolicyAttachment create
+ flusight-forecast aws:s3/bucket:Bucket create
+ flusight-forecast-read-bucket-policy aws:s3/bucketPolicy:BucketPolicy create
+ flusight-forecast aws:iam/rolePolicyAttachment:RolePolicyAttachment create
+ flusight-forecast-public-access-block aws:s3/bucketPublicAccessBlock:BucketPublicAccessBlock create
+ flusight-forecast-create-notification aws:s3/bucketNotification:BucketNotification create
```

4. If the Pulumi preview looks good, the PR can be merged after a code review. Once the PR is merged, Pulumi will apply the AWS changes.
5. The hub is now hosted in the Hubverse AWS account.

**Important:** The `org` and `repo` fields are used to create permissions that allow the hub's GitHub workflow to sync data to s3. If these values are not correct, the workflow will fail.


## Dev Setup

Expand Down
2 changes: 1 addition & 1 deletion src/hubverse_infrastructure/hubs/iam.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,9 @@ def create_model_output_lambda_trigger(
hub_name: str, hub_bucket: aws.s3.Bucket, model_output_lambda: aws.lambda_.Function
) -> aws.s3.BucketNotification:
"""Create the trigger that will invoke the model output lambda when a new file is written to the hub's S3 bucket."""

allow_bucket = aws.lambda_.Permission(
resource_name=f"{hub_name}-allow",
statement_id="AllowExecutionFromS3Bucket",
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternately, we can retain the statement_id but make it unique by incorporating the name of the hub

action="lambda:InvokeFunction",
function=model_output_lambda.arn.apply(lambda arn: f"{arn}"),
principal="s3.amazonaws.com",
Expand Down