Skip to content

Commit

Permalink
docs: Update onboarding docs (meltano#7717)
Browse files Browse the repository at this point in the history
* Update onboarding docs

* Update docs/src/_cloud/onboarding.md

Co-authored-by: Taylor A. Murphy <taylor@meltano.com>

* Update docs/src/_cloud/onboarding.md

Co-authored-by: Taylor A. Murphy <taylor@meltano.com>

* Update docs/src/_cloud/onboarding.md

Co-authored-by: Taylor A. Murphy <taylor@meltano.com>

* Update docs/src/_cloud/onboarding.md

Co-authored-by: Taylor A. Murphy <taylor@meltano.com>

* Update docs/src/_cloud/onboarding.md

Co-authored-by: Taylor A. Murphy <taylor@meltano.com>

* [pre-commit.ci] auto fixes from pre-commit.ci hooks

for more information, see https://pre-commit.ci

* Update docs/src/_cloud/onboarding.md

Co-authored-by: Taylor A. Murphy <taylor@meltano.com>

* Update docs/src/_cloud/onboarding.md

Co-authored-by: Taylor A. Murphy <taylor@meltano.com>

* Update docs/src/_cloud/onboarding.md

Co-authored-by: Taylor A. Murphy <taylor@meltano.com>

* PR feedback

* Add link to Getting Started guide

* Add deployments step

* Update GitHub app internal link

* PR feedback

* Update docs/src/_cloud/onboarding.md

Co-authored-by: Edgar R. M. <edgar@meltano.com>

* More PR feedback

* Update docs/src/_cloud/onboarding.md

Co-authored-by: Pat Nadolny <patnadolny@gmail.com>

* Update docs/src/_cloud/onboarding.md

Co-authored-by: Pat Nadolny <patnadolny@gmail.com>

* Update docs/src/_cloud/onboarding.md

Co-authored-by: Pat Nadolny <patnadolny@gmail.com>

* Update docs/src/_cloud/onboarding.md

Co-authored-by: Pat Nadolny <patnadolny@gmail.com>

---------

Co-authored-by: Taylor A. Murphy <taylor@meltano.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Edgar R. M. <edgar@meltano.com>
Co-authored-by: Pat Nadolny <patnadolny@gmail.com>
  • Loading branch information
5 people committed May 31, 2023
1 parent a8e848c commit cc97bd6
Show file tree
Hide file tree
Showing 3 changed files with 64 additions and 48 deletions.
3 changes: 3 additions & 0 deletions docs/src/_cloud/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,11 @@ title: Meltano Cloud
description: Information about Meltano Cloud Concepts.
layout: doc
toc: false
hidden: false
---



<div class="notification is-info">
<p><strong>Meltano Cloud is currently in Beta.</strong></p>
<p>While in Beta, functionality is not guaranteed and subject to change. <br> If you're interested in using Meltano Cloud please join our <a href="https://meltano.com/cloud/">waitlist</a>.</p>
Expand Down
107 changes: 61 additions & 46 deletions docs/src/_cloud/onboarding.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,29 +12,23 @@ weight: 2

## Prereqs

### Prereq #1: Provide access to your repo
### Prereq #1: Sign up for the waitlist

Click the following link to install the [Meltano Cloud GitHub App](https://github.com/apps/meltano-cloud) to your organization.
When asked "Where do you want to install this app?" you _must_ select your GitHub organization, not your personal account.
When asked which repositories to provide the app access to, select your GitHub project repo.
While Meltano Cloud is in Beta, you must sign up via [the waitlist](https://meltano.com/cloud/) in order to gain access.

### Prereq #2: Creating a test or sandbox environment
Waitlist members are added to Meltano Cloud on a rolling basis. You'll receive an email once you've been added.

For onboarding and debugging purposes, we recommend that teams create a [Meltano Environment](/concepts/environments) named `'sandbox'`, which can be used by Meltano Cloud to test that all jobs are working as expected.
### Prereq #2: Provide access to your repo

For more information, please see our guide: [Creating a Sandbox Environment for Meltano Cloud](/cloud/sandbox_environments)
Click the following link to install the [Meltano Cloud GitHub App](https://github.com/apps/meltano-cloud) to your organization.
When asked "Where do you want to install this app?" you _must_ select your GitHub organization, not your personal account.
When asked which repositories to provide the app access to, select your GitHub project repo.

If you aren't yet declaring 'environments' in `meltano.yml`, you can get started quickly by copy-pasting this snippet into your `meltano.yml` file:
If you do not yet have a Meltano project in GitHub, you can follow the steps in our [Getting Started](/getting-started) guide.

```yml
environments:
- dev
- staging
- prod
default_environment: dev
```
### Prereq #3: Define your schedules

### Prereq #3: Create schedules if needed
Meltano Cloud uses the [schedules](https://docs.meltano.com/concepts/project#schedules) defined in your meltano.yml project file to run workloads.

If your project does not yet have schedules defined - for instance, if you are running workloads via an external orchestrator - you'll want to create new schedules for use by Meltano Cloud.

Expand All @@ -59,25 +53,7 @@ jobs:

## Onboarding Steps

### Step 1: Submit Project Onboarding Information

Once you've completed the above, you are ready to onboard your project to Meltano Cloud.

To onboard your project, Meltano will need the following project information:

1. Company name: `___________`
- E.g. "ABC Company Inc."
1. Primary contact name and email: `___________`
1. GitHub repo information:
1. Repo URL: `___________`
2. Initial Meltano Environment Name: `___________` (e.g. 'prod' or 'staging')
3. Git branch name to use for the above environment: `___________` (e.g. `main`)
1. List of GitHub User IDs, separated by commas: `___________`
1. List of Meltano schedule names and intervals: `___________`
- This can be copy-pasted from `meltano.yml`.
- During Beta, you'll need to reach out to Meltano support in order to create additional schedules.

### Step 2: Install (or update) the Cloud CLI
### Step 1: Install (or update) the Cloud CLI

To install from scratch using [pipx](https://pypa.github.io/pipx/installation/#install-pipx):

Expand All @@ -93,32 +69,66 @@ pipx reinstall meltano-cloud-cli

The above commands will install a `meltano-cloud` CLI command into your workstation.

### Step 3: User creation and authorization
### Step 2: Login to Meltano Cloud

First, create your user account by logging in via: `meltano-cloud login`.
Login to Meltano Cloud by running the `meltano-cloud login` [command](/cloud/cloud-cli#login) in your local terminal.
You'll have to sign in with the same GitHub user you provided when you signed up via the waitlist.

The login command will open a browser window which you can use to create your account. In the Beta, your identification and authorization will be driven by your GitHub login identity. No Meltano-specific passwords or usernames are needed, and Meltano Cloud does not have access to your personal GitHub credentials.
The login command will open a browser window which you can use to access your account. In the Beta, your identification and authorization will be driven by your GitHub login identity. No Meltano-specific passwords or usernames are needed, and Meltano Cloud does not have access to your personal GitHub credentials.

After you create your account, within one business day, your account will be authorized according to the onboarding information you'll see be able to access your project within the CLI.
While Meltano Cloud is in Beta, it may take up to one business day for your user to be fully provisioned and added to your Meltano Cloud organization. You'll be notified via email once your newly created user has been added to your organization, at which time you can logout via `meltano-cloud logout` and then log back in to have full CLI access to your organization's Meltano Cloud resources.

<div class="notification is-info">
<p>As of now, installing the Meltano Cloud <a href="#prereq-1-provide-access-to-your-repo">GitHub App</a> to your organization is a separate process from granting Meltano Cloud access to use your GitHub profile for login purposes. Both the GitHub App installation for your organization <em>and</em> the OAuth grant flow for your profile must be performed in order to have full access to Meltano Cloud functionality.</p>
<p>Installing the Meltano Cloud <a href="#prereq-2-provide-access-to-your-repo">GitHub App</a> to your organization is a separate process from granting Meltano Cloud access to use your GitHub profile for login purposes. Both the GitHub App installation for your organization <em>and</em> the OAuth grant flow for your profile must be performed in order to have full access to Meltano Cloud functionality.</p>
</div>

### Step 4: Test basic functionality

After login, and after your project is onboarded, you can explore the interface with a few different commands:
### Step 3: Set default project and validate access and functionality

After logging in you can explore the interface with a few different commands.
The full list of CLI commands is in the [Cloud Docs](https://docs.meltano.com/cloud/cloud-cli).

To see Meltano Cloud projects for your organizations, run:
```console
meltano-cloud history
meltano-cloud project list
```
```console
Running this command will verify that your project is connected and you're properly authenticated with Meltano Cloud.
```

You should select a project to use as default for all commands.
You can do this by running:
```console
meltano-cloud schedule list
meltano-cloud project use <project name>
```

### Step 4: Create deployments

In order for pipelines to run, they must have a [deployment](/cloud/concepts#meltano-cloud-deployments) to run in.

To deploy a named [Meltano Environment](/concepts/environments) to Meltano Cloud, run the following [command](https://docs.meltano.com/cloud/cloud-cli#deployment):

```console
meltano-cloud deployment create --name <deployment name> --environment-name <Meltano Environment name> --git-rev <the git revision to use for this deployment>
```

For example, if you wanted to deploy the `prod` Meltano Environment as defined in your `meltano.yml` in the `main` branch of your git repo and you wanted the Meltano Cloud deployment to be named `production`, you would run:

```console
meltano-cloud deployment create --name production --environment-name prod --git-rev main
```

To confirm that your deployment was created, you can view all of your Meltano Cloud deployments by running:
```console
meltano-cloud deployments list
```

### Step 5: Initialize secrets

Secrets allow you to pass environment variables to your workloads without needing to expose them within your `meltano.yml`.
Setting a secret in Meltano Cloud is equivalent to using a `.env` file to store environment variables at runtime.
Secrets are shared across all deployments in your project and can be referenced in your `meltano.yml` file just as you would reference environment variables locally.

Secrets are configured using the Cloud CLI.

```console
Expand All @@ -132,9 +142,9 @@ meltano-cloud config env list
> TAP_GITLAB_PASSWORD
```

### Step 6: Run and debug your workloads
### Step 6: Run your workloads

You can invoke a schedule on-demand with the `meltano-cloud run` command:
You can invoke a schedule on-demand with the `meltano-cloud run` [command](/cloud/cloud-cli#run):

```console
meltano-cloud run --deployment=staging SCHEDULE_NAME
Expand All @@ -151,3 +161,8 @@ To view logs for any completed or still-running job, you can use:
```console
meltano-cloud logs print --execution-id=ASDF1234...
```

### Step 7: Enable schedules

Your Meltano Cloud project's schedules are disabled by default.
Use the `meltano-cloud schedule enable <SCHEDULE NAME>` [command](/cloud/cloud-cli#schedule) to enable schedules.
2 changes: 0 additions & 2 deletions docs/src/_cloud/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,6 @@ weight: 6
<p>While in Beta, functionality is not guaranteed and subject to change. <br> If you're interested in using Meltano Cloud please join our <a href="https://meltano.com/cloud/">waitlist</a>.</p>
</div>

## Security Whitepaper

<div class="notification is-info">
<p>Meltano Cloud is currently in Beta. Features and implementation details may change between Beta and GA.</p>
</div>
Expand Down

0 comments on commit cc97bd6

Please sign in to comment.