Skip to content

Commit

Permalink
docs: Updates for Meltano Cloud GitHub App permissions and remove Alp…
Browse files Browse the repository at this point in the history
…ha references (meltano#7585)

* updates for GH App permissions and remove Alpha references

* update details

* update secrets verbiage

* Update docs/src/_cloud/security.md

Co-authored-by: Will Da Silva <will@willdasilva.xyz>

---------

Co-authored-by: Will Da Silva <will@willdasilva.xyz>
  • Loading branch information
magreenbaum and WillDaSilva committed Apr 21, 2023
1 parent ca136d4 commit 4ee87ca
Show file tree
Hide file tree
Showing 6 changed files with 33 additions and 151 deletions.
45 changes: 0 additions & 45 deletions docs/src/_cloud/api_reference.md

This file was deleted.

73 changes: 0 additions & 73 deletions docs/src/_cloud/encrypting_secrets.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,76 +35,3 @@ Secrets cannot be decrypted after they are set. If you need to change a secret,
### Reserved Variables

See the [reserved variables](/platform/#reserved-variables) docs for more details on variables that are reserved for use by Meltano Cloud.

## Alpha Phase Encryption Method (Deprecated)

<div class="notification is-warning">
<p><strong>Meltano Cloud is currently in Beta.</strong></p>
<p>These Alpha instructions are now obsolete but are maintained for legacy
Alpha users.</p>
</div>

### Public Key

During the on-boarding process, Meltano will provide you with the Public Key of your public/private encryption key pair.
For details on the encryption algorithms and other security related information, refer to the [Security page](/security).

Save your public key somewhere for use during encryption.

### Utility kms-ext

We have a utility extension for encrypting your secrets:
https://github.com/meltano/kms-ext

Recommended installation process:

```
pip install pipx
pipx install git+https://github.com/meltano/kms-ext.git@main
```

Once installed, you should be able to run `kms --help` to see usage, options, and commands available.

> **Note**
> Since the private key of your encryption key never leaves our AWS servers, you are not able to decrypt your secrets once you have encrypted them.
> If you need to change or confirm your values, you will need to re-encrypt your .env file.
> Each time encryption is performed, all contents in the .env file will update.
### Example

For this example, the following statements are true:

- I am in my meltano project root directory
- Public key file is named `meltano.pub`
- Environment variable secrets are set in a file called `.env`
- I want my `secrets.yml` encrypted file in the root of my project directory

The following command will encrypt your `.env` file with the `meltano.pub` key and save the encrypted secrets to a file called `secrets.yml`.

```
kms encrypt meltano.pub --dotenv-path .env --output-path secrets.yml
```

By default, our cloud runners will look for `secrets.yml` in the root of your project.
If you would like to change the location or name of the secrets file, please inform us during your on-boarding process.

Example `.env` file:

```
THESE_ARE_MY_SECRET_VARS=secret_contents
DATABASE_CREDENTIALS=secret_database_credentials_here
```

The default output file, `secrets.yml`, will look similar to this:

```
env:
- name: THESE_ARE_MY_SECRET_VARS
value:
ciphertext: J+msMhK53SFZhAU6EGYyGwwFytyi4QzEG7cYSGMluBK1Kgs31YpldzwrNvygcyRzUtZ2XD2UKGVjzCl72aF63W/EizRCpZmwBviMLPn+ifp2MPWTVuJb0uD5qdy6ByUYjwyZdah9CwZcCCn2T0fZv9F6CX+srB4a0pluRovK14InoNm9Tr1ssqnrPPIWYH0sTNHjR0dGzdjrvOjSUqwXuPnROGmKRlHUQjW2LOYzn/3FbFC+M9uHiAGK6/lQRVTMOufvIJ/DwEnFJ22BcDhZYMBNkiB/pSnuhtdyAZf0IEgB/bzljqeZeRdoSnkm3vylyY148u78hLl9TWonDzxsbYO6oPYn8sxrzjbf9cwrwhRdMPjqyZLbMrXvwMReFa0/5r5soULrSy3aEECRD9vxZ2Xij4jHICDmfmPHPoZVovLw7se9Pnxyir9HASnSKa657fZQTcNFIWgIisIkJV6fE/+3EWyYeAwmTZ+Mjp2p13BwWRSmwMS2pY/lNm4TGAR73FvFdpjkHMLYqluTK53EHqZhUxlkfTNlMX8vYlmoUzQFCFeeGKKGtYLlQUAt+oYpFO3aUG1diM0QdR2Fss9W1VfZiHDA0bz0yCcV/GRZki8T3ehDMxaVqYreJyitM9r9TQIz8HWny1JSupNGGCFwPto4e7veOTSVo0AozBiG5Do=
scheme: RSAES_OAEP_SHA_256
- name: DATABASE_CREDENTIALS
value:
ciphertext: 3A6IWdP4fZoiI+xjENaxGI3MSue6svKk5l3ecXCJt4+sbD5X9M2IcvN6sBooi9jjKyQf55ojhzSlC2Wzaaw/d4Y1Ulh2kH4lae1UHrpT+K5yvah3PqZ51xU+TZqV4+7pd2YGpoEpdNsw3C/ZfLg+tt2JjpK0nOXnbgVTLrcqqVQj7PpjNSXFXr2IJmULgybRgCBKmBoTJWcLasLVvhuTOqdU5ZCm0fp/RXRltlK3/pFE1YMOXrVOGbozNluoHS5b5JrdOGZuHZ+He56PmH3bh4d1pWmi970gI+BQ3GBkyLOxdYigK0d/z8mZCdsc0G93GRS35J3HSg2cHoACsXPvCxAFSwt73skBsNMKuRdplrBc0YSpld5lG9ccTISGKf1t0YtXlDYI5bT/jZH56DcU0Lyz54zBo+PjNEQN4nGOil3d/pjBSXi0UuDH3GWEIw7Tvb08N3GfYPMQd5rPVagVyQjrHwGBugMDQy1SEtlsTBXl76porBnurAcb0LCiGQv+Q1dJCMG+JM1PE+qLrj6cOANhDWj8lCsHD7Nyz5Q4wJehnznvHKobDwZ50bEde53grXCp5s4gLOmaG9JCa8pDUjlM7ppqEkZcERFjKfp2VVicJI9Skcd1NRB9yemJrtdUKlsD5OOawZER0piJCBfQewJmBBDvtU7K5lPSUWqshH8=
scheme: RSAES_OAEP_SHA_256
```
12 changes: 4 additions & 8 deletions docs/src/_cloud/known_issues.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Known Limitations"
description: Details the Alpha limitations for Meltano Cloud
description: Details Beta limitations for Meltano Cloud
layout: doc
hidden: true
---
Expand All @@ -10,17 +10,13 @@ hidden: true
<p>While in Beta, functionality is not guaranteed and subject to change. <br> If you're interested in using Meltano Cloud please join our <a href="https://meltano.com/cloud/">waitlist</a>.</p>
</div>

## Limitations during Alpha

### Limited secrets management features

Within the Alpha phase, there will only be limited support for secrets management. Future Meltano Core and Meltano Cloud features will enable additional secrets management options.
## Limitations during Beta

### Manually-submitted schedules lists

During the Alpha, Meltano Cloud will only run schedules which you have explicitly declared and requested.
During Beta, Meltano Cloud will only run schedules which you have explicitly declared and requested.

In future, Meltano Cloud will automatically run any schedules you have configured to run within the named environment.
In the future, Meltano Cloud will automatically run any schedules you have configured to run within the named environment.

See related issue: [#6853](https://github.com/meltano/meltano/issues/6853).

Expand Down
19 changes: 0 additions & 19 deletions docs/src/_cloud/reserved_variables.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,25 +18,6 @@ There are specific environment variables that are reserved for certain use-cases

`GIT_SSH_PRIVATE_KEY` is a reserved variable that should be set if you have private repository packages.

> The following instruction and example is secrets encryption for Alpha which will be deprecated in Beta.
> Current encryption instructions can be found in our [Encrypting Secrets guide](https://github.com/meltano/cloud-docs/blob/main/docs/encrypting_secrets.md#components-for-encryption).
To encrypt, set the ssh private key env variable into your `.env` file as-is in the private key file with single quotes
around them.

Example `.env` file to be encrypted:
```
GIT_SSH_PRIVATE_KEY='-----BEGIN OPENSSH PRIVATE KEY-----
therearelotsofprivatekeymaterialhere
onvariouslineslikethis
wecanjustcopypasteasitappearsinthefile
andusesinglequotesaroundthewholething
-----END OPENSSH PRIVATE KEY-----'
SOME_OTHER_SECRET=1234asdf
```

Then continue with encryption using the [kms-ext](https://github.com/meltano/kms-ext) utility.

## Job or Schedule Run Notifications via Webhook

`MELTANO_CLOUD_WEBHOOK_URL` can be set to receive notifications on success or fail of a job or schedule run.
Expand Down
2 changes: 0 additions & 2 deletions docs/src/_cloud/sandbox_environments.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,6 @@ If your data pipelines write data to a data warehouse or other production resour

For example, if your production data pipelines output data to `RAW_DB`, you may want to create a new database called `STAGING_RAW_DB` or `RAW_DB_STAGING` which can be the target for the Meltano Cloud workloads during testing.

The default environment name for Meltano Cloud Alpha is `'sandbox'`, but you can also submit a different environment name if you prefer.

Note:

- In the Beta phase, branch and environment names will be fully configurable without contacting Meltano Support.
33 changes: 29 additions & 4 deletions docs/src/_cloud/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ weight: 6
## Security Whitepaper

<div class="notification is-info">
<p>Meltano Cloud is currently in Alpha. Features and implementation details may change between Alpha and GA.</p>
<p>Meltano Cloud is currently in Beta. Features and implementation details may change between Beta and GA.</p>
</div>

## Securing Project Secrets
Expand All @@ -27,12 +27,37 @@ weight: 6
1. Meltano will never store your secrets in clear text.
1. Meltano engineers do not have access to directly decrypt your secrets.
1. The decryption key for project secrets will never leave AWS servers.
1. Our IAM policies only allow the `decrypt` action within containers that are running project workloads.
1. Our IAM policies only allow the `decrypt` action within containers that are running project workloads or for services that require specific secrets to perform tasks such as sending notifications to your webhook urls.
1. Meltano uses envelope encryption strategy with AWS KMS keys to encrypt your secrets.

##### What encryption algorithms are used?

The algorithm for encrypting secrets is an RSA assymetric encryption, using 4096-bit keys.
The algorithm for encrypting secrets is an AES symmetric encryption, using 256-bit AES-GCM encryption keys.

More information is available on the AWS website:

- [https://docs.aws.amazon.com/kms/latest/developerguide/asymmetric-key-specs.html#key-spec-rsa-encryption](https://docs.aws.amazon.com/kms/latest/developerguide/asymmetric-key-specs.html#key-spec-rsa-encryption)
- [Symmetric Encryption Keys](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#symmetric-cmks)

## Meltano Cloud GitHub App Permissions

### Meltano Cloud Login

When performing login to Meltano Cloud, either via the CLI or the web UI, the Meltano Cloud GitHub App will request only the following permissions:

- Read-only access to your email addresses

### Meltano Cloud Project Permissions

When adding a project to Meltano Cloud, you will need to install the GitHub App to your Organization in GitHub.
You may grant the GitHub App read-only permissions to only the repositories you require it to access.
The following permissions are provided to the Meltano Cloud GitHub App:

Repository Permissions
- Repository contents, commits, branches, downloads, releases, and merges (read-only)
- Search repositories, list collaborators, and access repository metadata (read-only)

Organization Permissions
- Organization members and teams (read-only)

Account Permissions
- Manage a user's email addresses (read-only)

0 comments on commit 4ee87ca

Please sign in to comment.