Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

controller: add storcon-cli #7114

Merged
merged 13 commits into from Apr 3, 2024
Merged

controller: add storcon-cli #7114

merged 13 commits into from Apr 3, 2024

Conversation

jcsp
Copy link
Contributor

@jcsp jcsp commented Mar 13, 2024 •
edited

Problem

During incidents, we may need to quickly access the storage controller's API without trying API client code or crafting curl CLIs on the fly. A basic CLI client is needed for this.

Summary of changes

  • Update storage controller node listing API to only use public types in controller_api.rs
  • Add a storage controller API for listing tenants
  • Add a basic test that the CLI can list and modify nodes and tenants.

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 13, 2024 •
edited

2754 tests run: 2615 passed, 0 failed, 139 skipped (full report)

Flaky tests (1)

Postgres 16

  • test_empty_branch_remote_storage_upload: debug

Code coverage* (full report)

  • functions: 28.1% (6380 of 22709 functions)
  • lines: 47.0% (44976 of 95727 lines)

* collected from Rust tests only

The comment gets automatically updated with the latest test results
3108d3c at 2024-04-03T10:22:04.622Z :recycle:

@jcsp jcsp mentioned this pull request Mar 18, 2024
Merged
5 tasks
jcsp added a commit that referenced this pull request Mar 19, 2024
@jcsp
storage controller: tech debt (#7165)
a5d5c2a 
This is a mixed bag of changes split out for separate review while
working on other things, and batched together to reduce load on CI
runners. Each commits stands alone for review purposes:
- do_tenant_shard_split was a long function and had a synchronous
validation phase at the start that could readily be pulled out into a
separate function. This also avoids the special casing of
ApiError::BadRequest when deciding whether an abort is needed on errors
- Add a 'describe' API (GET on tenant ID) that will enable storcon-cli
to see what's going on with a tenant
- the 'locate' API wasn't really meant for use in the field. It's for
tests: demote it to the /debug/ prefix
- The `Single` placement policy was a redundant duplicate of Double(0),
and Double was a bad name. Rename it Attached.
(#7107)
- Some neon_local commands were added for debug/demos, which are now
replaced by commands in storcon-cli (#7114 ). Even though that's not
merged yet, we don't need the neon_local ones any more.

Closes #7107

## Backward compat of Single/Double -> `Attached(n)` change

A database migration is used to convert any existing values.
@jcsp jcsp force-pushed the jcsp/storcon-cli branch 2 times, most recently from 8fb5907 to 0702e93 Compare April 2, 2024 13:41
@jcsp jcsp added t/feature Issue type: feature, for new features or requests c/storage/controller Component: Storage Controller labels Apr 2, 2024
@jcsp jcsp requested a review from VladLazar April 2, 2024 15:11
@jcsp jcsp marked this pull request as ready for review April 2, 2024 15:11
@jcsp jcsp requested a review from a team as a code owner April 2, 2024 15:11
@jcsp jcsp mentioned this pull request Apr 2, 2024
Open
problame
problame approved these changes Apr 2, 2024
View reviewed changes
Copy link
Contributor

@problame problame left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nit-pickery, dismiss at your discretion.

Nice to have a CLI, it helps illustrate the rationale behind these APIs a bit.

It would be nice to have some one-liner doc comments for each Command variant.

control_plane/storcon_cli/src/main.rs Outdated Show resolved Hide resolved
control_plane/storcon_cli/src/main.rs Outdated Show resolved Hide resolved
control_plane/storcon_cli/src/main.rs Show resolved Hide resolved
control_plane/storcon_cli/src/main.rs Outdated Show resolved Hide resolved
control_plane/storcon_cli/src/main.rs Outdated Show resolved Hide resolved
libs/pageserver_api/src/controller_api.rs Outdated Show resolved Hide resolved
@jcsp jcsp enabled auto-merge (squash) April 3, 2024 08:24
@jcsp jcsp merged commit 6e3834d into main Apr 3, 2024
49 of 50 checks passed
@jcsp jcsp deleted the jcsp/storcon-cli branch April 3, 2024 10:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@problame problame problame approved these changes

@VladLazar VladLazar Awaiting requested review from VladLazar

Assignees
No one assigned
Labels
c/storage/controller Component: Storage Controller t/feature Issue type: feature, for new features or requests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@jcsp @problame