Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow API Keys to be managed for remote clusters #142400

Closed
legrego opened this issue Sep 30, 2022 · 12 comments · Fixed by #162363
Closed

Allow API Keys to be managed for remote clusters #142400

legrego opened this issue Sep 30, 2022 · 12 comments · Fixed by #162363
Assignees
Labels
enhancement New value added to drive a business result Feature:Security/Authorization Platform Security - Authorization Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! v8.10.0

Comments

@legrego
Copy link
Member

legrego commented Sep 30, 2022

The updated remote cluster security model requires changes to the API Key management screen.
RCS remote clusters will require specialized API keys to work. The specialized API key will be created with a new API endpoint. The request payload will be a simplified version of existing CreateApiKey payload to focus only on the necessary information. The specialized API key will be differentiated from regular API keys by its type. The type information will be returned in Get and Query responses. This is the only way to tell whether an API key is specialized.

High-level requirements

  1. The API Key management screen should be updated to allow the different API Key types to be distinguished from one another.
  2. Users with the elevated manage_security cluster privilege shall have the ability to create & update RCS API Keys, as described in the technical document provided by ES. This is a different privilege than what the rest of the screen requires today.
  3. API Key Invalidation will require the existing manage_api_keys cluster privilege.
    4) RCS API Keys cannot have their privileges limited, so this ability should be hidden when creating/updating these key types.
  4. RCS API Keys are (initially?) not available in the serverless offering, so these changes should only be visible for the current offering.
  5. RCS API Keys require an enterprise license, and should not be available for deployments with a lesser license level.
  6. Any new APIs introduced on the Kibana side should be marked as internal, for consistency with the rest of our API Key endpoints.
  7. These RCS API Key changes must not be visible in the UI until support for this is enabled in ES.

Nice-to-haves

  1. Elasticsearch can expose the role_descriptors and limited_by_role_descriptors as described here: Support viewing API key privileges elasticsearch#89058. It would be nice to update our screens to show these fields when viewing/editing an existing key, so that users can better understand the capabilities of each of their keys.
@legrego legrego added Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! enhancement New value added to drive a business result Feature:Security/Authorization Platform Security - Authorization labels Sep 30, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-security (Team:Security)

@legrego
Copy link
Member Author

legrego commented Apr 5, 2023

This work is no longer required

@legrego legrego closed this as not planned Won't fix, can't repro, duplicate, stale Apr 5, 2023
@legrego
Copy link
Member Author

legrego commented Apr 28, 2023

Reopening, as the design for RCS 2.0 has changed to include custom API Keys.

@legrego legrego reopened this Apr 28, 2023
@bytebilly
Copy link
Contributor

RCS API Keys cannot have their privileges limited, so this ability should be hidden when creating/updating these key types.

This sounds wrong. Users can limit what these API keys grant access to, it's actually one of the key features of the new security model 🙂

@bytebilly
Copy link
Contributor

The API Key management screen should be updated to allow the different API Key types to be distinguished from one another.

I'm not sure if this has been discussed or not, but there is also the option to provide this functionality in the "Remote cluster" section instead of the existing API key management panel.
It probably depends if we will list RCS API keys in the same list, or if we introduce a dedicated view.

@legrego
Copy link
Member Author

legrego commented May 16, 2023

RCS API Keys cannot have their privileges limited, so this ability should be hidden when creating/updating these key types.

This sounds wrong. Users can limit what these API keys grant access to, it's actually one of the key features of the new security model 🙂

Oops I think you're right, thanks for calling this out. I was confused by this statement:
"The specialized API keys will not use the existing limited-by model. This is a corollary of the manage_security privilege requirement. The ownership model still applies (owner means the user who creates these API keys)."

@legrego
Copy link
Member Author

legrego commented May 16, 2023

The API Key management screen should be updated to allow the different API Key types to be distinguished from one another.

I'm not sure if this has been discussed or not, but there is also the option to provide this functionality in the "Remote cluster" section instead of the existing API key management panel.

It probably depends if we will list RCS API keys in the same list, or if we introduce a dedicated view.

Ah that's an interesting thought, I think @thomheymann was trying to tell me this earlier today, too. @alisonelizabeth & @sabarasaba, is this something you've explored?

@bytebilly
Copy link
Contributor

Oops I think you're right, thanks for calling this out. I was confused by this statement:
"The specialized API keys will not use the existing limited-by model.

Oh, I see 🙂 That actually means that they are not limited by privileges of the user creating the API key, that are enforced for the "regular" creation flow, but only by what is specified in the API call payload.

@sabarasaba
Copy link
Member

Ah that's an interesting thought, I think @thomheymann was trying to tell me this earlier today, too. @alisonelizabeth & @sabarasaba, is this something you've explored?

We havent explored this idea as just recently we found out that we were going to include custom API Keys. I do like the thought of keeping things closer together, but it feels like the UIs would be quite similar to eachother. What do you think about this @kyrspl?

@thomheymann
Copy link
Contributor

The API Key management screen should be updated to allow the different API Key types to be distinguished from one another.

I'm not sure if this has been discussed or not, but there is also the option to provide this functionality in the "Remote cluster" section instead of the existing API key management panel. It probably depends if we will list RCS API keys in the same list, or if we introduce a dedicated view.

I think we would ideally provide both, the ability to view and manage RCS API keys from within the API Keys screen but also a dedicated experience for configuring remote clusters using RCS API keys from within the Remote Clusters screen.

We could create a few shared components if any of the forms / screens are the same to avoid duplication.

@bytebilly
Copy link
Contributor

Just to clarify further, the API key is created on one cluster and added to the keystore of the other one, so if we make it part of the remote cluster UI it will not interact with the setup flow anyway.

So, under remote cluster it would likely be splitting the view in two areas:

  • remote clusters that are configured locally
  • api keys that are created to allow other clusters to access

@thomheymann thomheymann self-assigned this Jun 27, 2023
@thomheymann
Copy link
Contributor

Depends on elastic/eui#6900

thomheymann added a commit that referenced this issue Aug 7, 2023
Resolves #142400

## Release notes

Added ability to manage Cross-Cluster API keys. 

## Summary

- Redesigned API keys page to cater for different API key types: 
- **Personal API Key** - Allows external services to access the Elastic
Stack on behalf of a user.
- **Cross-Cluster API key** - Allows remote clusters to connect to your
local cluster.
- **Managed API key** - Created and managed by Kibana to correctly run
background tasks. (e.g. alerting / fleet)
- Redesigned Create/Update API key popup to allow adding Cross-Cluster
API keys.
- Redesigned View API key popup showing more information and added
feedback when API keys cannot be edited.

## Technical notes

- Refactored API key schemas and types throughout all API key related
routes, services and clients to create a single source of truth and stop
types going out of sync.
- Consolidated internal endpoints to simplify usage and reduce
unnecessary HTTP roundtrips.
- Migrated API Key form to Formik to more reliably manage state and to
simplify validation logic when toggling visibility of form elements.
- Broke out API key table and related primitives into separate reusable
components for a more cohesive design.

## Screenshots

### API keys page

<details>
  <summary>Before</summary>

<img width="1249" alt="Screenshot 2023-07-24 at 10 25 04"
src="https://github.com/elastic/kibana/assets/190132/eb122597-f138-4658-9141-fd76b3291751">
</details>

<details open>
  <summary>After</summary>

<img width="1159" alt="Screenshot 2023-07-23 at 17 01 24"
src="https://github.com/elastic/kibana/assets/190132/42be5002-235e-4785-83e3-eb4063ca75ba">
</details>

### Create Cross-Cluster API key flyover

<details>
  <summary>Before</summary>

<img width="1261" alt="Screenshot 2023-07-24 at 10 25 21"
src="https://github.com/elastic/kibana/assets/190132/9e3ddffc-e6ec-4c9a-aaa4-a20b0ecf4d51">
</details>

<details open>
  <summary>After</summary>

<img width="1172" alt="Screenshot 2023-07-23 at 17 06 41"
src="https://github.com/elastic/kibana/assets/190132/e6823c07-2154-4777-820f-a3bab9cabe0f">
</details>

### API key details flyover

<details>
  <summary>Before</summary>

<img width="1262" alt="Screenshot 2023-07-24 at 10 25 35"
src="https://github.com/elastic/kibana/assets/190132/212293f3-355b-40d3-a1d6-5eea9c61c1cf">
</details>

<details open>
  <summary>After</summary>

<img width="1260" alt="Screenshot 2023-07-24 at 10 34 01"
src="https://github.com/elastic/kibana/assets/190132/a5f561af-e415-49dd-9f3f-70250eb32ef7">
</details>

## Testing

### Conditions of satisfaction

1) The API Key management screen should be updated to allow the
different API Key types to be distinguished from one another.
2) Users with the elevated `manage_security` cluster privilege shall
have the ability to create & update RCS API Keys, as described in the
technical document provided by ES. This is a different privilege than
what the rest of the screen requires today.
3) API Key Invalidation will require the existing `manage_api_keys`
cluster privilege.
4) RCS API Keys are not available in the serverless offering, so these
changes should only be visible for the current offering.

Note: This functionality requires a true serverless ES instance - The
feature will not be hidden simply by running Kibana using serverless
flag

5) RCS API Keys require an `enterprise` license, and should not be
available for deployments with a lesser license level.
6) Any new APIs introduced on the Kibana side should be marked as
`internal`, for consistency with the rest of our API Key endpoints.
7) These RCS API Key changes must not be visible in the UI until support
for this is enabled in ES.
crespocarlos pushed a commit to crespocarlos/kibana that referenced this issue Aug 8, 2023
Resolves elastic#142400

## Release notes

Added ability to manage Cross-Cluster API keys. 

## Summary

- Redesigned API keys page to cater for different API key types: 
- **Personal API Key** - Allows external services to access the Elastic
Stack on behalf of a user.
- **Cross-Cluster API key** - Allows remote clusters to connect to your
local cluster.
- **Managed API key** - Created and managed by Kibana to correctly run
background tasks. (e.g. alerting / fleet)
- Redesigned Create/Update API key popup to allow adding Cross-Cluster
API keys.
- Redesigned View API key popup showing more information and added
feedback when API keys cannot be edited.

## Technical notes

- Refactored API key schemas and types throughout all API key related
routes, services and clients to create a single source of truth and stop
types going out of sync.
- Consolidated internal endpoints to simplify usage and reduce
unnecessary HTTP roundtrips.
- Migrated API Key form to Formik to more reliably manage state and to
simplify validation logic when toggling visibility of form elements.
- Broke out API key table and related primitives into separate reusable
components for a more cohesive design.

## Screenshots

### API keys page

<details>
  <summary>Before</summary>

<img width="1249" alt="Screenshot 2023-07-24 at 10 25 04"
src="https://github.com/elastic/kibana/assets/190132/eb122597-f138-4658-9141-fd76b3291751">
</details>

<details open>
  <summary>After</summary>

<img width="1159" alt="Screenshot 2023-07-23 at 17 01 24"
src="https://github.com/elastic/kibana/assets/190132/42be5002-235e-4785-83e3-eb4063ca75ba">
</details>

### Create Cross-Cluster API key flyover

<details>
  <summary>Before</summary>

<img width="1261" alt="Screenshot 2023-07-24 at 10 25 21"
src="https://github.com/elastic/kibana/assets/190132/9e3ddffc-e6ec-4c9a-aaa4-a20b0ecf4d51">
</details>

<details open>
  <summary>After</summary>

<img width="1172" alt="Screenshot 2023-07-23 at 17 06 41"
src="https://github.com/elastic/kibana/assets/190132/e6823c07-2154-4777-820f-a3bab9cabe0f">
</details>

### API key details flyover

<details>
  <summary>Before</summary>

<img width="1262" alt="Screenshot 2023-07-24 at 10 25 35"
src="https://github.com/elastic/kibana/assets/190132/212293f3-355b-40d3-a1d6-5eea9c61c1cf">
</details>

<details open>
  <summary>After</summary>

<img width="1260" alt="Screenshot 2023-07-24 at 10 34 01"
src="https://github.com/elastic/kibana/assets/190132/a5f561af-e415-49dd-9f3f-70250eb32ef7">
</details>

## Testing

### Conditions of satisfaction

1) The API Key management screen should be updated to allow the
different API Key types to be distinguished from one another.
2) Users with the elevated `manage_security` cluster privilege shall
have the ability to create & update RCS API Keys, as described in the
technical document provided by ES. This is a different privilege than
what the rest of the screen requires today.
3) API Key Invalidation will require the existing `manage_api_keys`
cluster privilege.
4) RCS API Keys are not available in the serverless offering, so these
changes should only be visible for the current offering.

Note: This functionality requires a true serverless ES instance - The
feature will not be hidden simply by running Kibana using serverless
flag

5) RCS API Keys require an `enterprise` license, and should not be
available for deployments with a lesser license level.
6) Any new APIs introduced on the Kibana side should be marked as
`internal`, for consistency with the rest of our API Key endpoints.
7) These RCS API Key changes must not be visible in the UI until support
for this is enabled in ES.
bryce-b pushed a commit to bryce-b/kibana that referenced this issue Aug 9, 2023
Resolves elastic#142400

## Release notes

Added ability to manage Cross-Cluster API keys. 

## Summary

- Redesigned API keys page to cater for different API key types: 
- **Personal API Key** - Allows external services to access the Elastic
Stack on behalf of a user.
- **Cross-Cluster API key** - Allows remote clusters to connect to your
local cluster.
- **Managed API key** - Created and managed by Kibana to correctly run
background tasks. (e.g. alerting / fleet)
- Redesigned Create/Update API key popup to allow adding Cross-Cluster
API keys.
- Redesigned View API key popup showing more information and added
feedback when API keys cannot be edited.

## Technical notes

- Refactored API key schemas and types throughout all API key related
routes, services and clients to create a single source of truth and stop
types going out of sync.
- Consolidated internal endpoints to simplify usage and reduce
unnecessary HTTP roundtrips.
- Migrated API Key form to Formik to more reliably manage state and to
simplify validation logic when toggling visibility of form elements.
- Broke out API key table and related primitives into separate reusable
components for a more cohesive design.

## Screenshots

### API keys page

<details>
  <summary>Before</summary>

<img width="1249" alt="Screenshot 2023-07-24 at 10 25 04"
src="https://github.com/elastic/kibana/assets/190132/eb122597-f138-4658-9141-fd76b3291751">
</details>

<details open>
  <summary>After</summary>

<img width="1159" alt="Screenshot 2023-07-23 at 17 01 24"
src="https://github.com/elastic/kibana/assets/190132/42be5002-235e-4785-83e3-eb4063ca75ba">
</details>

### Create Cross-Cluster API key flyover

<details>
  <summary>Before</summary>

<img width="1261" alt="Screenshot 2023-07-24 at 10 25 21"
src="https://github.com/elastic/kibana/assets/190132/9e3ddffc-e6ec-4c9a-aaa4-a20b0ecf4d51">
</details>

<details open>
  <summary>After</summary>

<img width="1172" alt="Screenshot 2023-07-23 at 17 06 41"
src="https://github.com/elastic/kibana/assets/190132/e6823c07-2154-4777-820f-a3bab9cabe0f">
</details>

### API key details flyover

<details>
  <summary>Before</summary>

<img width="1262" alt="Screenshot 2023-07-24 at 10 25 35"
src="https://github.com/elastic/kibana/assets/190132/212293f3-355b-40d3-a1d6-5eea9c61c1cf">
</details>

<details open>
  <summary>After</summary>

<img width="1260" alt="Screenshot 2023-07-24 at 10 34 01"
src="https://github.com/elastic/kibana/assets/190132/a5f561af-e415-49dd-9f3f-70250eb32ef7">
</details>

## Testing

### Conditions of satisfaction

1) The API Key management screen should be updated to allow the
different API Key types to be distinguished from one another.
2) Users with the elevated `manage_security` cluster privilege shall
have the ability to create & update RCS API Keys, as described in the
technical document provided by ES. This is a different privilege than
what the rest of the screen requires today.
3) API Key Invalidation will require the existing `manage_api_keys`
cluster privilege.
4) RCS API Keys are not available in the serverless offering, so these
changes should only be visible for the current offering.

Note: This functionality requires a true serverless ES instance - The
feature will not be hidden simply by running Kibana using serverless
flag

5) RCS API Keys require an `enterprise` license, and should not be
available for deployments with a lesser license level.
6) Any new APIs introduced on the Kibana side should be marked as
`internal`, for consistency with the rest of our API Key endpoints.
7) These RCS API Key changes must not be visible in the UI until support
for this is enabled in ES.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New value added to drive a business result Feature:Security/Authorization Platform Security - Authorization Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! v8.10.0
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants