Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Cloud CLI for adding projects #7620

Merged
merged 58 commits into from
Aug 22, 2023
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
c5c7476
wip
Apr 27, 2023
f4414e3
first pass
Apr 27, 2023
7028745
Merge branch 'cloud' into kgpayne/issue7412
Apr 27, 2023
3aaf5e8
await response
Apr 27, 2023
c510bbe
Merge branch 'cloud' into kgpayne/issue7412
May 3, 2023
c473da8
tests and pr feedback
May 3, 2023
4e89668
tidy up
May 4, 2023
3a391f3
Merge branch 'cloud' into kgpayne/issue7412
May 4, 2023
3693eab
Update src/cloud-cli/meltano/cloud/cli/project.py
May 4, 2023
938f616
Update src/cloud-cli/tests/cli/test_project.py
May 4, 2023
26bfa27
Merge branch 'cloud' into kgpayne/issue7412
May 5, 2023
ca3341b
poetry update
May 5, 2023
f2edfc9
Merge branch 'cloud' into kgpayne/issue7412
May 15, 2023
cf67cfd
Merge branch 'cloud' into kgpayne/issue7412
cjohnhanson Jul 17, 2023
e35b699
Project add command
cjohnhanson Jul 18, 2023
49f4639
Use new provisioner API projects endpoint in project add command
cjohnhanson Jul 19, 2023
a2fa7da
Use yaspin
cjohnhanson Jul 19, 2023
770919e
[pre-commit.ci] auto fixes from pre-commit.ci hooks
pre-commit-ci[bot] Jul 19, 2023
30c655f
Add -> create
cjohnhanson Jul 19, 2023
861dcbb
Merge branch 'cloud' into kgpayne/issue7412
cjohnhanson Jul 19, 2023
eb616a4
Merge branch 'kgpayne/issue7412' into INFRA-988-deploy-project-stack-…
cjohnhanson Jul 19, 2023
dcf6816
Add -> create in tests
cjohnhanson Jul 20, 2023
3340adc
[pre-commit.ci] auto fixes from pre-commit.ci hooks
pre-commit-ci[bot] Jul 20, 2023
93e74cb
Merge branch 'cloud' into kgpayne/issue7412
cjohnhanson Jul 28, 2023
9280990
Merge branch 'kgpayne/issue7412' into INFRA-988-deploy-project-stack-…
cjohnhanson Jul 28, 2023
8d68cbe
feat: Deploy project stack on creation (#7925)
cjohnhanson Jul 28, 2023
6fab957
merge cloud
cjohnhanson Aug 3, 2023
7431f5e
Merge branch 'cloud' into INFRA-988-deploy-project-stack-on-creation
cjohnhanson Aug 7, 2023
e073299
Update src/cloud-cli/meltano/cloud/cli/project.py
cjohnhanson Aug 7, 2023
ab367b3
Update src/cloud-cli/meltano/cloud/cli/project.py
cjohnhanson Aug 7, 2023
f443689
Update src/cloud-cli/meltano/cloud/cli/project.py
cjohnhanson Aug 7, 2023
5be269f
Update src/cloud-cli/meltano/cloud/cli/project.py
cjohnhanson Aug 7, 2023
6745784
Update src/cloud-cli/tests/cli/test_project.py
cjohnhanson Aug 7, 2023
bbce161
Update src/cloud-cli/tests/cli/test_project.py
cjohnhanson Aug 7, 2023
b00c218
merging cloud
cjohnhanson Aug 7, 2023
a4b8503
Don't print created project json
cjohnhanson Aug 7, 2023
8db1246
project_name -> name
cjohnhanson Aug 7, 2023
4b9e567
repository-url -> git-repository
cjohnhanson Aug 7, 2023
7b6d3d2
correct project create args in test
cjohnhanson Aug 7, 2023
297bec9
correct project create args in test
cjohnhanson Aug 7, 2023
03dbb64
project-root-path -> root-path
cjohnhanson Aug 8, 2023
2ec325d
[pre-commit.ci] auto fixes from pre-commit.ci hooks
pre-commit-ci[bot] Aug 8, 2023
11b9119
Merge branch 'cloud' into kgpayne/issue7412
cjohnhanson Aug 11, 2023
daaa92a
Update project create test
cjohnhanson Aug 11, 2023
ec7bfe6
Update src/meltano/cloud/cli/project.py
cjohnhanson Aug 11, 2023
cbdb557
Update src/meltano/cloud/cli/project.py
cjohnhanson Aug 11, 2023
537e2e5
Update src/meltano/cloud/cli/project.py
cjohnhanson Aug 11, 2023
e4a7d56
PR feedback
cjohnhanson Aug 11, 2023
f13e507
Add project create to cloud-cli docs.
cjohnhanson Aug 11, 2023
f511f1d
More flexible expected stdout output for project create test
cjohnhanson Aug 11, 2023
f199c9b
ignore type checks for yaspin
cjohnhanson Aug 11, 2023
9f7554b
Merge branch 'cloud' into kgpayne/issue7412
cjohnhanson Aug 21, 2023
4e8ae89
re-add signed request
cjohnhanson Aug 21, 2023
96444ec
[pre-commit.ci] auto fixes from pre-commit.ci hooks
pre-commit-ci[bot] Aug 21, 2023
2172f13
Merge branch 'cloud' into kgpayne/issue7412
cjohnhanson Aug 21, 2023
b9e79f5
[pre-commit.ci] auto fixes from pre-commit.ci hooks
pre-commit-ci[bot] Aug 21, 2023
da30a4f
Addressing pr feedback for docs.
cjohnhanson Aug 22, 2023
421362b
Merge branch 'cloud' into kgpayne/issue7412
cjohnhanson Aug 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
37 changes: 32 additions & 5 deletions docs/docs/cloud/cloud-cli.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,14 @@ sidebar_position: 5

:::info

<p><strong>Meltano Cloud is currently in Beta.</strong></p>
<p>While in Beta, functionality is not guaranteed and subject to change. <br /> If you're interested in using Meltano Cloud please join our <a href="https://meltano.com/cloud/">waitlist</a>.</p>
<p>
<strong>Meltano Cloud is currently in Beta.</strong>
</p>
<p>
While in Beta, functionality is not guaranteed and subject to change. <br />{" "}
If you're interested in using Meltano Cloud please join our{" "}
<a href="https://meltano.com/cloud/">waitlist</a>.
</p>

:::

Expand Down Expand Up @@ -68,8 +74,12 @@ The above example creates a new deployment named `my-dev-deployment` for the Mel

:::info

<p>If your deployment is failing you can try running <a href="/reference/command-line-interface#compile">`meltano compile`</a> to confirm that your configuration files are valid.
Also double check that you have schedules configured, otherwise the deployment will throw an error.</p>
<p>
If your deployment is failing you can try running{" "}
<a href="/reference/command-line-interface#compile">`meltano compile`</a> to
confirm that your configuration files are valid. Also double check that you
have schedules configured, otherwise the deployment will throw an error.
</p>

:::

Expand Down Expand Up @@ -219,6 +229,14 @@ meltano cloud project list
╰───────────┴───────────────────────────────┴──────────────────────────────────────────────────────────╯
```

The `create` subcommand creates a new Meltano Cloud project.

```sh
meltano-cloud project create --name example-project --repo-url https://github.com/meltano/squared.git --root-path "data/"
⠋ Creating project - this may take several minutes...
Project 'example-project' created successfully.
cjohnhanson marked this conversation as resolved.
Show resolved Hide resolved
```

When you run the `login` command, if you only have a single project, it will be set as the default project to use for future commands. Otherwise, you will need to run `meltano cloud project use` to specify which Meltano Cloud project the other `meltano cloud` commands should operate on.

When `meltano cloud project use` is not provided any argument, it will list the available projects, and have you select one interactively using the arrow keys. To select a project as the default non-interactively, use the `--name` argument:
Expand Down Expand Up @@ -260,7 +278,16 @@ meltano cloud job stop --execution-id 15e1cbbde6b2424f86c04b237291d652

:::info

<p>At the moment, executions stopped manually will be marked as <i>Failed</i> in the output of <a href="#history"><code>meltano cloud history</code></a>. See <a href="https://github.com/meltano/meltano/issues/7697">#7697</a> for more details.</p>
{" "}
<p>
At the moment, executions stopped manually will be marked as <i>Failed</i> in
the output of{" "}
<a href="#history">
<code>meltano cloud history</code>
</a>
. See <a href="https://github.com/meltano/meltano/issues/7697">#7697</a> for
more details.
</p>
:::

## `schedule`
Expand Down
62 changes: 61 additions & 1 deletion src/meltano/cloud/cli/project.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,14 @@
import logging
import sys
import typing as t
from http import HTTPStatus

import click
import questionary
from slugify import slugify
from yaspin import yaspin # type: ignore

from meltano.cloud.api.client import MeltanoCloudClient
from meltano.cloud.api.client import MeltanoCloudClient, MeltanoCloudError
from meltano.cloud.cli.base import (
LimitedResult,
get_paginated,
Expand Down Expand Up @@ -63,6 +66,25 @@
),
)

async def create_project(
self,
project_name: str,
git_repository: str,
project_root_path: str | None = None,
):
"""Use POST to create new Meltano Cloud project."""
async with self.authenticated():
payload = {"project_name": project_name, "git_repository": git_repository}
if project_root_path:
payload["project_root_path"] = project_root_path
async with self._raw_request(
"POST",
f"/projects/v1/{self.config.tenant_resource_key}",
json=payload,
) as response:
response.raise_for_status()
return response


@click.group("project")
def project_group() -> None:
Expand Down Expand Up @@ -282,3 +304,41 @@
),
fg="green",
)


@project_group.command("create")
@click.option("--name", type=str, prompt=True)
@click.option("--repo-url", type=str, prompt=True)
@click.option("--root-path", type=str, required=False)
@pass_context
@run_async
async def create_project(
context: MeltanoCloudCLIContext,
name: str,
repo_url: str,
root_path: str | None = None,
):
"""Create a project to your Meltano Cloud."""
async with ProjectsCloudClient(config=context.config) as client:
try:
with yaspin(
text="Creating project - this may take several minutes...",
):
response = await client.create_project(
project_name=name,
git_repository=repo_url,
project_root_path=root_path,
)
except MeltanoCloudError as e:

Check warning on line 332 in src/meltano/cloud/cli/project.py

View check run for this annotation

Codecov / codecov/patch

src/meltano/cloud/cli/project.py#L332

Added line #L332 was not covered by tests
if e.response.status == HTTPStatus.CONFLICT:
click.secho(

Check warning on line 334 in src/meltano/cloud/cli/project.py

View check run for this annotation

Codecov / codecov/patch

src/meltano/cloud/cli/project.py#L334

Added line #L334 was not covered by tests
(
f"A project named {name!r} (normalized to "
f"{slugify(name)!r}) already exists."
),
fg="yellow",
)
return None

Check warning on line 341 in src/meltano/cloud/cli/project.py

View check run for this annotation

Codecov / codecov/patch

src/meltano/cloud/cli/project.py#L341

Added line #L341 was not covered by tests
click.echo(f"Project {name!r} created successfully.")
if response.status == HTTPStatus.NO_CONTENT:
return None

Check warning on line 344 in src/meltano/cloud/cli/project.py

View check run for this annotation

Codecov / codecov/patch

src/meltano/cloud/cli/project.py#L344

Added line #L344 was not covered by tests
55 changes: 55 additions & 0 deletions tests/meltano/cloud/cli/test_project.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@

if t.TYPE_CHECKING:
from pytest_httpserver import HTTPServer
from requests_mock import Mocker as RequestsMocker

from meltano.cloud.api.config import MeltanoCloudConfig
from meltano.cloud.api.types import CloudProject
Expand Down Expand Up @@ -74,6 +75,21 @@ def projects(self, tenant_resource_key: str) -> list[CloudProject]:
def path(self, tenant_resource_key: str) -> str:
return f"/projects/v1/{tenant_resource_key}"

@pytest.fixture()
def prepared_request(self, request: pytest.FixtureRequest):
return {
"method": request.param["method"],
"url": "https://asdf.lambda-url.us-west-2.on.aws.example.com/",
"params": {},
"headers": {
"Content-Type": "application/json",
"X-Amz-Date": "20230525T185058Z",
"X-Amz-Security-Token": "IQoJb3JpZ2luX2VjEOP/////////wEaCXV==",
"Authorization": "AWS4-HMAC-SHA256 Credential=ASI3AV1SM3BH...",
},
"data": '{"environment_name": "staging", "git_rev": "main"}',
}

def test_project_list_table(
self,
config: MeltanoCloudConfig,
Expand Down Expand Up @@ -391,3 +407,42 @@ def test_project_use_by_name_and_id_error(self, config: MeltanoCloudConfig):
)
assert result.exit_code == 2
assert "The '--name' and '--id' options are mutually exclusive" in result.output

@pytest.mark.parametrize("prepared_request", ({"method": "POST"},), indirect=True)
def test_project_create(
self,
projects: list[CloudProject],
httpserver: HTTPServer,
config: MeltanoCloudConfig,
prepared_request,
requests_mock: RequestsMocker,
):
for project in projects[:3]:
httpserver.expect_oneshot_request(
f"/projects/v1/{project['tenant_resource_key']}",
method="POST",
).respond_with_json(prepared_request)
requests_mock.post(prepared_request["url"], json=project) # noqa: S113
result = CliRunner().invoke(
cli,
(
"--config-path",
config.config_path,
"project",
"create",
"--name",
project["project_name"],
"--repo-url",
project["git_repository"],
"--root-path",
project["project_root_path"],
),
)
assert result.exit_code == 0, result.output
assert (
"Creating project - this may take several minutes..." in result.output
)
assert (
f"Project {project['project_name']!r} created successfully.\n"
in result.output
)