Support for security remote_cluster and associated privileges

[jakelandis]

This commit adds support for the remote_cluster in the role and role descriptors.
Additionally:

• adds missing references to remote_indices
• add new cluster privilege monitor_stats
• adds related version information where applicable
• updates references to cluster from string[] to proper enumeration

===

Note - this PR replaces #3123 since PR from forks won't pass CI.

[jakelandis]

Follow ups from comments on original PR:

#3123 (comment)

Can you please run make spec-format-fix and reopen this pull request from a branch?

done in 1d5d03a

#3123 (comment)

RemoteClusterPrivilege (enum) -> RemoteClusterPrivileges (class)

done in 9a70a07

#3123 (comment)

I've opened a pull request to remove it and required a review from you.

Thank you !

#3123 (comment) and #3123 (comment)

For me it would be ok to introduce a breaking change by changing the return type to the correct enum type. What about the rest @elastic/devtools-team ?

Still pending...

[github-actions]

Following you can find the validation results for the APIs you have changed.

API	Status	Request	Response
security.activate_user_profile	🟢	9/9	9/9
security.authenticate	🟢	30/30	30/30
security.bulk_delete_role	🟢	1/1	1/1
security.bulk_put_role	🔴	0/1	1/1
security.bulk_update_api_keys	🟠	Missing type	Missing type
security.change_password	🟢	9/9	9/9
security.clear_api_key_cache	🟢	13/13	13/13
security.clear_cached_privileges	🟢	3/3	3/3
security.clear_cached_realms	🟢	1/1	1/1
security.clear_cached_roles	🟢	2/2	2/2
security.clear_cached_service_tokens	🟢	4/4	4/4
security.create_api_key	🔴	66/68	59/59
security.create_cross_cluster_api_key	🔴	2/3	3/3
security.create_service_token	🟢	3/3	3/3
security.delete_privileges	🟢	6/6	6/6
security.delete_role_mapping	🟢	9/9	9/9
security.delete_role	🟢	8/8	8/8
security.delete_service_token	⚪	Missing test	Missing test
security.delete_user	🟢	9/9	9/9
security.disable_user_profile	🟢	1/1	1/1
security.disable_user	🟢	3/3	3/3
security.enable_user_profile	🟢	1/1	1/1
security.enable_user	🟢	4/4	4/4
security.enroll_kibana	⚪	Missing test	Missing test
security.enroll_node	⚪	Missing test	Missing test
security.get_api_key	🔴	38/38	15/38
security.get_builtin_privileges	🟢	1/1	1/1
security.get_privileges	🟢	12/12	12/12
security.get_role_mapping	🔴	18/18	10/18
security.get_role	🔴	24/24	22/24
security.get_service_accounts	⚪	Missing test	Missing test
security.get_service_credentials	🟢	1/1	1/1
security.get_settings	🟠	Missing type	Missing type
security.get_token	🟢	25/25	24/24
security.get_user_privileges	🔴	8/8	7/8
security.get_user_profile	🟢	8/8	8/8
security.get_user	🟢	25/25	25/25
security.grant_api_key	🟢	7/7	7/7
security.has_privileges_user_profile	🟢	3/3	3/3
security.has_privileges	🟢	24/24	24/24
security.invalidate_api_key	🟢	12/12	12/12
security.invalidate_token	🟢	11/11	11/11
security.oidc_authenticate	🟠	Missing type	Missing type
security.oidc_logout	🟠	Missing type	Missing type
security.oidc_prepare_authentication	🟠	Missing type	Missing type
security.put_privileges	🟢	10/10	10/10
security.put_role_mapping	🔴	2/11	11/11
security.put_role	🔴	32/39	38/38
security.put_user	🟢	48/48	47/47
security.query_api_keys	🔴	14/14	1/14
security.query_role	🟢	2/2	2/2
security.query_user	🟢	4/4	4/4
security.saml_authenticate	⚪	Missing test	Missing test
security.saml_complete_logout	⚪	Missing test	Missing test
security.saml_invalidate	⚪	Missing test	Missing test
security.saml_logout	⚪	Missing test	Missing test
security.saml_prepare_authentication	⚪	Missing test	Missing test
security.saml_service_provider_metadata	⚪	Missing test	Missing test
security.suggest_user_profiles	🟢	1/1	1/1
security.update_api_key	🟢	5/5	5/5
security.update_cross_cluster_api_key	🟢	2/2	2/2
security.update_settings	🟠	Missing type	Missing type
security.update_user_profile_data	🟢	1/1	1/1

You can validate these APIs yourself by using the make validate target.

[n1v0lg]

LGTM from an ES security perspective -- deferring to @flobernd on the final recommendation around whether we want to go from string to enum for privilege values in the read APIs.

[n1v0lg]

Not for this PR, but I think this is wrong: IndexName should either be a string or IndexPrivilege -- IndexName is something else entirely

[github-actions]

Following you can find the validation results for the APIs you have changed.

API	Status	Request	Response
security.activate_user_profile	🟢	9/9	9/9
security.authenticate	🟢	30/30	30/30
security.bulk_delete_role	🟢	1/1	1/1
security.bulk_put_role	🔴	0/1	1/1
security.bulk_update_api_keys	🟠	Missing type	Missing type
security.change_password	🟢	9/9	9/9
security.clear_api_key_cache	🟢	13/13	13/13
security.clear_cached_privileges	🟢	3/3	3/3
security.clear_cached_realms	🟢	1/1	1/1
security.clear_cached_roles	🟢	2/2	2/2
security.clear_cached_service_tokens	🟢	4/4	4/4
security.create_api_key	🔴	66/68	59/59
security.create_cross_cluster_api_key	🔴	2/3	3/3
security.create_service_token	🟢	3/3	3/3
security.delete_privileges	🟢	6/6	6/6
security.delete_role_mapping	🟢	9/9	9/9
security.delete_role	🟢	8/8	8/8
security.delete_service_token	⚪	Missing test	Missing test
security.delete_user	🟢	9/9	9/9
security.disable_user_profile	🟢	1/1	1/1
security.disable_user	🟢	3/3	3/3
security.enable_user_profile	🟢	1/1	1/1
security.enable_user	🟢	4/4	4/4
security.enroll_kibana	⚪	Missing test	Missing test
security.enroll_node	⚪	Missing test	Missing test
security.get_api_key	🔴	38/38	15/38
security.get_builtin_privileges	🟢	1/1	1/1
security.get_privileges	🟢	12/12	12/12
security.get_role_mapping	🔴	18/18	10/18
security.get_role	🔴	24/24	22/24
security.get_service_accounts	⚪	Missing test	Missing test
security.get_service_credentials	🟢	1/1	1/1
security.get_settings	🟠	Missing type	Missing type
security.get_token	🟢	25/25	24/24
security.get_user_privileges	🔴	8/8	7/8
security.get_user_profile	🟢	8/8	8/8
security.get_user	🟢	25/25	25/25
security.grant_api_key	🟢	7/7	7/7
security.has_privileges_user_profile	🟢	3/3	3/3
security.has_privileges	🟢	24/24	24/24
security.invalidate_api_key	🟢	12/12	12/12
security.invalidate_token	🟢	11/11	11/11
security.oidc_authenticate	🟠	Missing type	Missing type
security.oidc_logout	🟠	Missing type	Missing type
security.oidc_prepare_authentication	🟠	Missing type	Missing type
security.put_privileges	🟢	10/10	10/10
security.put_role_mapping	🔴	2/11	11/11
security.put_role	🔴	32/39	38/38
security.put_user	🟢	48/48	47/47
security.query_api_keys	🔴	14/14	1/14
security.query_role	🟢	2/2	2/2
security.query_user	🟢	4/4	4/4
security.saml_authenticate	⚪	Missing test	Missing test
security.saml_complete_logout	⚪	Missing test	Missing test
security.saml_invalidate	⚪	Missing test	Missing test
security.saml_logout	⚪	Missing test	Missing test
security.saml_prepare_authentication	⚪	Missing test	Missing test
security.saml_service_provider_metadata	⚪	Missing test	Missing test
security.suggest_user_profiles	🟢	1/1	1/1
security.update_api_key	🟢	5/5	5/5
security.update_cross_cluster_api_key	🟢	2/2	2/2
security.update_settings	🟠	Missing type	Missing type
security.update_user_profile_data	🟢	1/1	1/1

You can validate these APIs yourself by using the make validate target.

[github-actions]

Following you can find the validation results for the APIs you have changed.

API	Status	Request	Response
security.activate_user_profile	🟢	9/9	9/9
security.authenticate	🟢	30/30	30/30
security.bulk_delete_role	🟢	1/1	1/1
security.bulk_put_role	🔴	0/1	1/1
security.bulk_update_api_keys	🟠	Missing type	Missing type
security.change_password	🟢	9/9	9/9
security.clear_api_key_cache	🟢	13/13	13/13
security.clear_cached_privileges	🟢	3/3	3/3
security.clear_cached_realms	🟢	1/1	1/1
security.clear_cached_roles	🟢	2/2	2/2
security.clear_cached_service_tokens	🟢	4/4	4/4
security.create_api_key	🔴	66/68	59/59
security.create_cross_cluster_api_key	🔴	2/3	3/3
security.create_service_token	🟢	3/3	3/3
security.delete_privileges	🟢	6/6	6/6
security.delete_role_mapping	🟢	9/9	9/9
security.delete_role	🟢	8/8	8/8
security.delete_service_token	⚪	Missing test	Missing test
security.delete_user	🟢	9/9	9/9
security.disable_user_profile	🟢	1/1	1/1
security.disable_user	🟢	3/3	3/3
security.enable_user_profile	🟢	1/1	1/1
security.enable_user	🟢	4/4	4/4
security.enroll_kibana	⚪	Missing test	Missing test
security.enroll_node	⚪	Missing test	Missing test
security.get_api_key	🔴	38/38	15/38
security.get_builtin_privileges	🟢	1/1	1/1
security.get_privileges	🟢	12/12	12/12
security.get_role_mapping	🔴	18/18	10/18
security.get_role	🔴	24/24	22/24
security.get_service_accounts	⚪	Missing test	Missing test
security.get_service_credentials	🟢	1/1	1/1
security.get_settings	🟠	Missing type	Missing type
security.get_token	🟢	25/25	24/24
security.get_user_privileges	🔴	8/8	7/8
security.get_user_profile	🟢	8/8	8/8
security.get_user	🟢	25/25	25/25
security.grant_api_key	🟢	7/7	7/7
security.has_privileges_user_profile	🟢	3/3	3/3
security.has_privileges	🟢	24/24	24/24
security.invalidate_api_key	🟢	12/12	12/12
security.invalidate_token	🟢	11/11	11/11
security.oidc_authenticate	🟠	Missing type	Missing type
security.oidc_logout	🟠	Missing type	Missing type
security.oidc_prepare_authentication	🟠	Missing type	Missing type
security.put_privileges	🟢	10/10	10/10
security.put_role_mapping	🔴	2/11	11/11
security.put_role	🔴	32/39	38/38
security.put_user	🟢	48/48	47/47
security.query_api_keys	🔴	14/14	1/14
security.query_role	🟢	2/2	2/2
security.query_user	🟢	4/4	4/4
security.saml_authenticate	⚪	Missing test	Missing test
security.saml_complete_logout	⚪	Missing test	Missing test
security.saml_invalidate	⚪	Missing test	Missing test
security.saml_logout	⚪	Missing test	Missing test
security.saml_prepare_authentication	⚪	Missing test	Missing test
security.saml_service_provider_metadata	⚪	Missing test	Missing test
security.suggest_user_profiles	🟢	1/1	1/1
security.update_api_key	🟢	5/5	5/5
security.update_cross_cluster_api_key	🟢	2/2	2/2
security.update_settings	🟠	Missing type	Missing type
security.update_user_profile_data	🟢	1/1	1/1

You can validate these APIs yourself by using the make validate target.

[pquentin]

Thank you for iterating! LGTM.

[jakelandis]

@pquentin / @flobernd - thanks for the guidance. Should I merge this back to the 8.x branch ? The changes here are relevant for 8.17. (not sure how the branching strategy works for this repo)

[github-actions]

The backport to 8.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-8.x 8.x
# Navigate to the new working tree
cd .worktrees/backport-8.x
# Create a new branch
git switch --create backport-3125-to-8.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 dac7201aa2a52228954f70195c478647caeae7c1
# Push it to GitHub
git push --set-upstream origin backport-3125-to-8.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-8.x

Then, create a pull request where the base branch is 8.x and the compare/head branch is backport-3125-to-8.x.

[pquentin]

Thanks! We use the same branching strategy but backporting is a bit different. I applied the "backport 8.x" label.

[pquentin]

Since the automatic backport failed due to a conflict, I created it manually at #3140 and would appreciate a review.