Personal Access Tokens

[kobelb]

In a recent RFC regarding how Kibana handles Authentication headers provided by proxies, the following statement was made

The Kibana and Elasticsearch security teams posit that an API Key is not a user, and that an API Key is not necessarily under the control of the user that created it. In other words, these are not designed to be used as "personal access tokens".

While API keys have not been designed to be used as "personal access tokens", we have seen them used in this manner. For example, when a user authenticates to Kibana using an SSO realm, we've commonly recommended that they create an API key if they want to use a tool like curl to communicate with either Elasticsearch or Kibana HTTP APIs.

This will likely only get more common over time in situations like Elastic Cloud (ESS) as we continue to support additional different types of users who can authenticate using the preconfigured Elastic Cloud SSO realm.

As we begin adding Kibana features that are "per user" and the API Keys that a user has created are seen as a different user, we will introduce some interesting behaviors for the situations where we have historically been using API keys like "personal access tokens" and will eventually need personal access tokens.

/cc @elastic/kibana-security

[elasticmachine]

Pinging @elastic/es-security (Team:Security)

[bytebilly]

I feel that we are mixing multiple topics here, so I'll try to clarify a few details about our existing plans for API keys and PAT.

API keys can still be used to directly access Elasticsearch API in the case of SSO realms. However, API keys are not PAT and don't "follow the user". This means that their use may have unexpected behaviors in some cases. They also require a specific privilege to be created.

For Cloud SSO, we agreed that API keys are enough to guarantee access to Elasticsearch API at this stage, since SSO users are always superusers, their permissions don't change, and they can create API keys.
PAT have been discussed in this context several times, and we decided to further follow up in the future when Cloud SSO will possibly evolve to support multiple users with different roles, and use of native users will be drastically reduced.

The problem for Kibana API is that API keys work in most of the cases, but not for tasks requiring the creation of other API keys (e.g. Alerting) because an API key cannot create another API key. This is a technical constraint that affects users in any realm.

If there is an urgency to allow programmatic access to all Kibana API, this could be addressed independently and PAT may or may not be the right answer.

[kobelb]

Thank you for the detailed response, @bytebilly. Do you mind checking my understanding?

We will continue to recommend that users create API keys to access both ES and Kibana HTTP APIs, for example in Cloud.

These API Keys will not work for programmatic access to Kibana HTTP APIs that should only be used by people because we should be using personal access tokens for this.

Additionally, these API Keys will not work for programmatic access to Kibana HTTP APIs that internally create API Keys because API Keys can't create API Keys. This is potentially a separate problem from what would be solved by personal access tokens.

[bytebilly]

We will continue to recommend that users create API keys to access both ES and Kibana HTTP APIs, for example in Cloud.

👍

These API Keys will not work for programmatic access to Kibana HTTP APIs that should only be used by people because we should be using personal access tokens for this.

I'm not sure that they will not work. But there will be limitations in some cases.

Additionally, these API Keys will not work for programmatic access to Kibana HTTP APIs that internally create API Keys because API Keys can't create API Keys. This is potentially a separate problem from what would be solved by personal access tokens.

👍

[kobelb]

Thank you for correcting my prior thinking, @bytebilly, I'll update the issue description accordingly as the situation is not as dire as I feared.

[tvernum]

The Kibana and Elasticsearch security teams posit that an API Key is not a user, and that an API Key is not necessarily under the control of the user that created it. In other words, these are not designed to be used as "personal access tokens".

The last part of that description might be a bit of over-editorialisation.

Given the range of ways in which ES API Keys are used today, it is definitely not safe to treat an API Key as being the user that owns it. For example

• Fleet enrollment keys are not PATs for the individual that created the fleet policy. We should definitely not allow an enrollment key to act on that user's Kibana preferences.
• An API Key used for Kibana Alerting is closer to being a PAT, but it's still not "the user". For example, we should not update the user's "last login time" every time an alerting job runs.

Whether API keys are designed to be used as "personal access tokens" probably depends on what definition we want to use for a PAT. Our view is that it's perfectly fine for them to be used as PATs if their behaviour fits the needs of the people who want to use them.
However it's definitely not fine for Elastic products to assume that an API Key owned by user1 is functionally equivalent to a user authenticating as user1. Questions about access to personal preferences, notifications, etc require a more in-depth consideration than simply being based on the user-id.

[kobelb]

Whether API keys are designed to be used as "personal access tokens" probably depends on what definition we want to use for a PAT. Our view is that it's perfectly fine for them to be used as PATs if their behaviour fits the needs of the people who want to use them.
However it's definitely not fine for Elastic products to assume that an API Key owned by user1 is functionally equivalent to a user authenticating as user1. Questions about access to personal preferences, notifications, etc require a more in-depth consideration than simply being based on the user-id.

To be honest, this is what concerns me. API Keys don't currently work well for Kibana HTTP APIs. We already have the issue that API Keys can't be used to create alerting rules because API Keys can't create API Keys. @bytebilly has brought up that the solution to this problem potentially isn't personal access tokens, so I'm happy to discuss this complication elsewhere.

And while I totally get that it's not fine for Elastic products to assume that an API Key owned by user1 is functionally equivalent to a user authenticating as user1 in all situations, I'm almost certain some people will want for it to behave this way in some situations. Particularly because we're recommending that they create API Keys to interact with Kibana HTTP APIs when they've authenticated using SSO. If the solution to this is Personal Access Tokens, we will need them sooner rather than later.