feat: Cloud CLI for adding projects (#7620)

* wip * first pass * await response * tests and pr feedback * tidy up * Update src/cloud-cli/meltano/cloud/cli/project.py Co-authored-by: Edgar R. M. <edgar@meltano.com> * Update src/cloud-cli/tests/cli/test_project.py Co-authored-by: Edgar R. M. <edgar@meltano.com> * poetry update * Project add command * Use new provisioner API projects endpoint in project add command * Use yaspin * [pre-commit.ci] auto fixes from pre-commit.ci hooks for more information, see https://pre-commit.ci * Add -> create * Add -> create in tests * [pre-commit.ci] auto fixes from pre-commit.ci hooks for more information, see https://pre-commit.ci * feat: Deploy project stack on creation (#7925) * Project add command * Use new provisioner API projects endpoint in project add command * Use yaspin * [pre-commit.ci] auto fixes from pre-commit.ci hooks for more information, see https://pre-commit.ci * Add -> create * Add -> create in tests * [pre-commit.ci] auto fixes from pre-commit.ci hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update src/cloud-cli/meltano/cloud/cli/project.py Co-authored-by: Will Da Silva <will@willdasilva.xyz> * Update src/cloud-cli/meltano/cloud/cli/project.py Co-authored-by: Will Da Silva <will@willdasilva.xyz> * Update src/cloud-cli/meltano/cloud/cli/project.py Co-authored-by: Will Da Silva <will@willdasilva.xyz> * Update src/cloud-cli/meltano/cloud/cli/project.py Co-authored-by: Will Da Silva <will@willdasilva.xyz> * Update src/cloud-cli/tests/cli/test_project.py Co-authored-by: Will Da Silva <will@willdasilva.xyz> * Update src/cloud-cli/tests/cli/test_project.py Co-authored-by: Will Da Silva <will@willdasilva.xyz> * Don't print created project json * project_name -> name * repository-url -> git-repository * correct project create args in test * correct project create args in test * project-root-path -> root-path * [pre-commit.ci] auto fixes from pre-commit.ci hooks for more information, see https://pre-commit.ci * Update project create test * Update src/meltano/cloud/cli/project.py Co-authored-by: Edgar R. M. <edgar@meltano.com> * Update src/meltano/cloud/cli/project.py Co-authored-by: Edgar R. M. <edgar@meltano.com> * Update src/meltano/cloud/cli/project.py Co-authored-by: Edgar R. M. <edgar@meltano.com> * PR feedback * Add project create to cloud-cli docs. * More flexible expected stdout output for project create test * ignore type checks for yaspin * [pre-commit.ci] auto fixes from pre-commit.ci hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.ci hooks for more information, see https://pre-commit.ci * Addressing pr feedback for docs. --------- Co-authored-by: Edgar R. M. <edgar@meltano.com> Co-authored-by: Cody J. Hanson <cody@meltano.com> Co-authored-by: Cody J. Hanson <cody@codyjhanson.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Will Da Silva <will@willdasilva.xyz>
meltano · Aug 22, 2023 · 5da0e84 · 5da0e84
1 parent 5a300bf
commit 5da0e84
Show file tree

Hide file tree

Showing 4 changed files with 168 additions and 11 deletions.
diff --git a/docs/docs/cloud/cloud-cli.mdx b/docs/docs/cloud/cloud-cli.mdx
@@ -7,8 +7,14 @@ sidebar_position: 5
 
 :::info
 
-<p><strong>Meltano Cloud is currently in Beta.</strong></p>
-<p>While in Beta, functionality is not guaranteed and subject to change. <br /> If you're interested in using Meltano Cloud please join our <a href="https://meltano.com/cloud/">waitlist</a>.</p>
+<p>
+  <strong>Meltano Cloud is currently in Beta.</strong>
+</p>
+<p>
+  While in Beta, functionality is not guaranteed and subject to change. <br />{" "}
+  If you're interested in using Meltano Cloud please join our{" "}
+  <a href="https://meltano.com/cloud/">waitlist</a>.
+</p>
 
 :::
 
@@ -68,8 +74,12 @@ The above example creates a new deployment named `my-dev-deployment` for the Mel
 
 :::info
 
-<p>If your deployment is failing you can try running <a href="/reference/command-line-interface#compile">`meltano compile`</a> to confirm that your configuration files are valid.
-Also double check that you have schedules configured, otherwise the deployment will throw an error.</p>
+<p>
+  If your deployment is failing you can try running{" "}
+  <a href="/reference/command-line-interface#compile">`meltano compile`</a> to
+  confirm that your configuration files are valid. Also double check that you
+  have schedules configured, otherwise the deployment will throw an error.
+</p>
 
 :::
 
@@ -219,6 +229,16 @@ meltano cloud project list
 ╰───────────┴───────────────────────────────┴──────────────────────────────────────────────────────────╯
 ```
 
+The `create` subcommand creates a new Meltano Cloud project.
+
+In order to create a project, you'll first need to grant access to the Meltano Cloud GitHub app as described in [our onboarding documentation](/cloud/onboarding#prereq-2-provide-access-to-your-repo).
+
+```sh
+meltano-cloud project create --name example-project --repo-url https://github.com/meltano/squared.git --root-path "data/"
+⠋ Creating project - this may take several minutes...
+Project 'example-project' created successfully.
+```
+
 When you run the `login` command, if you only have a single project, it will be set as the default project to use for future commands. Otherwise, you will need to run `meltano cloud project use` to specify which Meltano Cloud project the other `meltano cloud` commands should operate on.
 
 When `meltano cloud project use` is not provided any argument, it will list the available projects, and have you select one interactively using the arrow keys. To select a project as the default non-interactively, use the `--name` argument:
@@ -260,7 +280,16 @@ meltano cloud job stop --execution-id 15e1cbbde6b2424f86c04b237291d652
 
 :::info
 
-  <p>At the moment, executions stopped manually will be marked as <i>Failed</i> in the output of <a href="#history"><code>meltano cloud history</code></a>. See <a href="https://github.com/meltano/meltano/issues/7697">#7697</a> for more details.</p>
+{" "}
+<p>
+  At the moment, executions stopped manually will be marked as <i>Failed</i> in
+  the output of{" "}
+  <a href="#history">
+    <code>meltano cloud history</code>
+  </a>
+  . See <a href="https://github.com/meltano/meltano/issues/7697">#7697</a> for
+  more details.
+</p>
 :::
 
 ## `schedule`

diff --git a/docs/docs/cloud/onboarding.md b/docs/docs/cloud/onboarding.md
@@ -85,11 +85,22 @@ While Meltano Cloud is in Beta, it may take up to one business day for your user
   <p>Installing the Meltano Cloud <a href="#prereq-2-provide-access-to-your-repo">GitHub App</a> to your organization is a separate process from granting Meltano Cloud access to use your GitHub profile for login purposes. Both the GitHub App installation for your organization <em>and</em> the OAuth grant flow for your profile must be performed in order to have full access to Meltano Cloud functionality.</p>
 :::
 
-### Step 3: Set default project and validate access and functionality
+### Step 3: Create a Project
 
 After logging in you can explore the interface with a few different commands.
 The full list of CLI commands is in the [Cloud Docs](https://docs.meltano.com/cloud/cloud-cli).
 
+To create a project, use the `meltano-cloud project create` command:
+
+```sh
+meltano-cloud project create --name example-project --repo-url https://github.com/meltano/squared.git --root-path "data/"
+⠋ Creating project - this may take several minutes...
+Project 'example-project' created successfully.
+```
+
+### Step 4: Set default project and validate access and functionality
+
+
 To see Meltano Cloud projects for your organizations, run:
 
 ```console
@@ -107,7 +118,7 @@ You can do this by running:
 meltano cloud project use --name <project name>
 ```
 
-### Step 4: Create deployments
+### Step 5: Create deployments
 
 In order for pipelines to run, they must have a [deployment](/cloud/concepts#meltano-cloud-deployments) to run in.
 
@@ -135,7 +146,7 @@ To confirm that your deployment was created, you can view all of your Meltano Cl
 meltano cloud deployment list
 ```
 
-### Step 5: Initialize secrets
+### Step 6: Initialize secrets
 
 Secrets allow you to pass environment variables to your workloads without needing to expose them within your `meltano.yml`.
 Setting a secret in Meltano Cloud is equivalent to using a `.env` file to store environment variables at runtime.
@@ -154,7 +165,7 @@ meltano cloud config env list
 > TAP_GITLAB_PASSWORD
 ```
 
-### Step 6: Run your workloads
+### Step 7: Run your workloads
 
 You can invoke a schedule on-demand with the `meltano cloud run` [command](/cloud/cloud-cli#run):
 
@@ -174,7 +185,7 @@ To view logs for any completed or still-running job, you can use:
 meltano cloud logs print --execution-id=ASDF1234...
 ```
 
-### Step 7: Enable schedules
+### Step 8: Enable schedules
 
 Your Meltano Cloud project's schedules are disabled by default.
 Use the `meltano cloud schedule enable <SCHEDULE NAME>` [command](/cloud/cloud-cli#schedule) to enable schedules.
diff --git a/src/meltano/cloud/cli/project.py b/src/meltano/cloud/cli/project.py
@@ -6,12 +6,16 @@
 import logging
 import sys
 import typing as t
+from http import HTTPStatus
 
 import click
 import questionary
+import requests
+from slugify import slugify
 from ulid import ULID
+from yaspin import yaspin  # type: ignore
 
-from meltano.cloud.api.client import MeltanoCloudClient
+from meltano.cloud.api.client import MeltanoCloudClient, MeltanoCloudError
 from meltano.cloud.cli.base import (
     LimitedResult,
     get_paginated,
@@ -93,6 +97,26 @@ async def get_projects(
                 ),
             )
 
+    async def create_project(
+        self,
+        project_name: str,
+        git_repository: str,
+        project_root_path: str | None = None,
+    ):
+        """Use POST to create new Meltano Cloud project."""
+        async with self.authenticated():
+            payload = {"project_name": project_name, "git_repository": git_repository}
+            if project_root_path:
+                payload["project_root_path"] = project_root_path
+            prepared_request = await self._json_request(
+                "POST",
+                f"/projects/v1/{self.config.tenant_resource_key}",
+                json=payload,
+            )
+            response = requests.request(**t.cast(t.Dict[str, t.Any], prepared_request))
+            response.raise_for_status()
+            return response
+
 
 @click.group("project")
 def project_group() -> None:
@@ -334,3 +358,41 @@ async def use_project(
         ),
         fg="green",
     )
+
+
+@project_group.command("create")
+@click.option("--name", type=str, prompt=True)
+@click.option("--repo-url", type=str, prompt=True)
+@click.option("--root-path", type=str, required=False)
+@pass_context
+@run_async
+async def create_project(
+    context: MeltanoCloudCLIContext,
+    name: str,
+    repo_url: str,
+    root_path: str | None = None,
+):
+    """Create a project to your Meltano Cloud."""
+    async with ProjectsCloudClient(config=context.config) as client:
+        try:
+            with yaspin(
+                text="Creating project - this may take several minutes...",
+            ):
+                response = await client.create_project(
+                    project_name=name,
+                    git_repository=repo_url,
+                    project_root_path=root_path,
+                )
+        except MeltanoCloudError as e:
+            if e.response.status == HTTPStatus.CONFLICT:
+                click.secho(
+                    (
+                        f"A project named {name!r} (normalized to "
+                        f"{slugify(name)!r}) already exists."
+                    ),
+                    fg="yellow",
+                )
+            return None
+        click.echo(f"Project {name!r} created successfully.")
+        if response.status_code == HTTPStatus.NO_CONTENT:
+            return None
diff --git a/tests/meltano/cloud/cli/test_project.py b/tests/meltano/cloud/cli/test_project.py
@@ -17,6 +17,7 @@
 
 if t.TYPE_CHECKING:
     from pytest_httpserver import HTTPServer
+    from requests_mock import Mocker as RequestsMocker
 
     from meltano.cloud.api.config import MeltanoCloudConfig
     from meltano.cloud.api.types import CloudProject
@@ -74,6 +75,21 @@ def projects(self, tenant_resource_key: str) -> list[CloudProject]:
     def path(self, tenant_resource_key: str) -> str:
         return f"/projects/v1/{tenant_resource_key}"
 
+    @pytest.fixture()
+    def prepared_request(self, request: pytest.FixtureRequest):
+        return {
+            "method": request.param["method"],
+            "url": "https://asdf.lambda-url.us-west-2.on.aws.example.com/",
+            "params": {},
+            "headers": {
+                "Content-Type": "application/json",
+                "X-Amz-Date": "20230525T185058Z",
+                "X-Amz-Security-Token": "IQoJb3JpZ2luX2VjEOP/////////wEaCXV==",
+                "Authorization": "AWS4-HMAC-SHA256 Credential=ASI3AV1SM3BH...",
+            },
+            "data": '{"environment_name": "staging", "git_rev": "main"}',
+        }
+
     def test_project_list_table(
         self,
         config: MeltanoCloudConfig,
@@ -392,6 +408,45 @@ def test_project_use_by_name_and_id_error(self, config: MeltanoCloudConfig):
         assert result.exit_code == 2
         assert "The '--name' and '--id' options are mutually exclusive" in result.output
 
+    @pytest.mark.parametrize("prepared_request", ({"method": "POST"},), indirect=True)
+    def test_project_create(
+        self,
+        projects: list[CloudProject],
+        httpserver: HTTPServer,
+        config: MeltanoCloudConfig,
+        prepared_request,
+        requests_mock: RequestsMocker,
+    ):
+        for project in projects[:3]:
+            httpserver.expect_oneshot_request(
+                f"/projects/v1/{project['tenant_resource_key']}",
+                method="POST",
+            ).respond_with_json(prepared_request)
+            requests_mock.post(prepared_request["url"], json=project)  # noqa: S113
+            result = CliRunner().invoke(
+                cli,
+                (
+                    "--config-path",
+                    config.config_path,
+                    "project",
+                    "create",
+                    "--name",
+                    project["project_name"],
+                    "--repo-url",
+                    project["git_repository"],
+                    "--root-path",
+                    project["project_root_path"],
+                ),
+            )
+            assert result.exit_code == 0, result.output
+            assert (
+                "Creating project - this may take several minutes..." in result.output
+            )
+            assert (
+                f"Project {project['project_name']!r} created successfully.\n"
+                in result.output
+            )
+
     def test_project_invalid_ulid(self, config: MeltanoCloudConfig):
         result = CliRunner().invoke(
             cli,