new feature: cluster list/resource fetcher

[tmishina]

Overview

Kubernetes resources (e.g., kubectl get pod) can be used as evidence. For example, spec of Pod, custom resource of an operator, and ConfigMap shows whether applications (pod) and kubernetes infrastructure (operator) run with correct (expected) configuration. An enterprise often uses multiple clusters operated by multiple cloud service platform (e.g., EKS of AWS, GKE of GCP, OpenShift of IBM Cloud) for its IT infrastructure. In that situation, it is not straightforward to fetch resources from the multiple clusters because their authentication/authorization mechanisms and cluster management mechanisms differ over the providers.

This issue focuses on fetching resources from multiple clusters of multiple cloud service providers. We plan to implement two fetchers; one is cluster list fetcher (per cloud service provider) and another is cluster resource fetcher.

Requirements

• The cluster list fetcher(s) should fetch a list of cluster for vanilla kube clusters and at least one cloud service provider clusters
• The cluster resource fetcher should fetch resources from the listed clusters

Approach

To support multiple cloud service providers,

1. cluster list fetcher is implemented per each provider
2. cluster resource fetcher is implemented once, but login mechanism for provider is delegated to provider-specific login function

Cluster List Fetchers

• For vanilla kube, BOM (Bill of Materials) written in an auditree config file and it will be stored into evidence locker
• For each public cluster, the provider-specific CLI tool (e.g, eksutil for EKS, gcloud for GKE, ibmcloud for IBM Cloud) will be used to login each cloud provider, and then fetch cluster list from the provider's cluster admin API

Cluster Resource Fetcher

• For vanilla kube, kubectl get RESOURCE_TYPE --kubeconfig path/to/kubeconfig is used to fetch resources
• For each public cluster, the provider-specific CLI tool will be used to login each cluster, and then kubectl get RESOURCE_TYPE (neither --kubeconfig nor --token is specified because authorization token is already configured by the login command) is used to fetch resources

Security and Privacy

Cluster List fetchers

• For vanilla kube clusters, no authorization token is used because the BOM is directly specified in an auditree config file, and thus no security concern exists.
• For each public cluster, API key for the provider should be stored in ~/.credential to login the cluster management API of each provider. User needs to manage ~/.credentials in secure manner.

Cluster Resource fetcher

• For vanilla kube cluster, existing kubeconfig file specified in an auditree config file is used to access the cluster. User needs to manage kubeconfig file as usual.
• For each public cluster, kubeconfig file which is configured by login command of provider's CLI tool is used to access the cluster as similar to the list fetcher behaviour.

Test Plan

The test will be done against one public cluster service both for vanilla kube logic and public cloud logic.

[alfinkel]

Which fetcher category folders do you expect these fetchers to live in or will we need to add new categories?

[tmishina]

@alfinkel

• The cluster resource fetcher will go into kubernetes category.
• Regarding the cluster list fetchers, I will implement two fetchers.
  • One is BOM (Bill of Materials) fetcher in kubernetes category
  • Another is IBM Cloud cluster list fetcher in ibm_cloud category.

[alfinkel]

@tmishina can we close this?

[tmishina]

@alfinkel I have a plan to contribute an IBM Cloud resource fetcher, which is based on the Kubernetes fetcher (#32), and therefore I'd like to close this issue when I complete the IBM Cloud resource fetcher.