Ability to deploy only the selected API groups of a provider

[muvaf]

What problem are you facing?

When you deploy a provider like provider-aws, CRDs of all groups are deployed and their controllers are started. But if you'd like to use only IAM resources like IAMRole, IAMPolicy etc., you can't really do it.

One of the use cases that needs this granularity could be the use of granular IRSA. Today, you can choose how controller should authenticate to provider API by using spec.providerConfigRef, however, the controller pod itself has ability to use every auth mechanism that's defined, including IRSA, if it wants to. So, I wanted to open this issue to see how folks feel about this in terms of security.

How could Crossplane help solve your problem?

We could deploy a separate provider instance for each group. For example:

kubectl crossplane install crossplane/provider-aws:v0.15.0 \
  --groups rds.aws.crossplane.io,someothergroup.aws.crossplane.io
  # if --groups is not given, assume all

Then you can configure IRSA of this provider pod specifically using ControllerConfig and then if you'd like to deploy other groups with different IRSA configuration, you can do so. The resulting pods won't share the same cache and each controller manager would have a bit of overhead, so in terms of CPU/memory usage I'd expect higher numbers in total.

[bassam]

it might be helpful to have a whitelist and blacklist of supported groups. This would enable folks to disable IAM for example.

[muvaf]

We had a quick sync with @ulucinar about this issue. I believe the use cases (partition, hence stronger IRSA story) described in the issue will be mostly covered by #2411 . So, it would make sense to limit this to just being able to install a provider with limited set of resources with no guarantee about whether you can install another provider with another set or conflicting sets. The rough design is to have a new field under Provider resource:

apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws
spec:
  image: crossplane/provider-aws:v0.19.2
  apiGroups: # optional. if empty, it will work just like today.
  - ec2.aws.crossplane.io
  - rds.aws.crossplane.io

The spec.apiGroups would propagate to ProviderRevision and provider revision controller would install only allowed set and let the provider binary know about the list via a command line flag.

One important aspect is that if the user adds or removes a group from the list, we should be able to account for that and take necessary steps, which could be creating a new ProviderRevision, treating a change in spec.apiGroups just like a change in spec.image.

cc @negz @hasheddan

[ulucinar]

Thanks @muvaf. One question: Why do we treat a change in the set of reconciled API groups by the provider as a new provider revision? It will correspond to a change in the command-line options we pass to the provider. Is it because the change is in the provider spec? If so, what about making this part of the ControllerConfig maybe?

[muvaf]

It will correspond to a change in the command-line options we pass to the provider.

Unfortunately, it's not only that. Package manager applies CRDs and then creates the Deployment object. Then RBAC manager reconciles the active ProviderRevision to create the managed roles. All three is affected by the list of CRDs a provider brings in. So, the significance of the change is as high as updating the provider. Even if the image tag doesn't change, we actually do update everything deployed for that provider. So, we can use the same mechanism, i.e. creating a new ProviderRevision, that we use in update so that we don't have to reimplement that functionality.

If so, what about making this part of the ControllerConfig maybe?

If it was only a flag change, we could consider this. But since it affects other stuff, we'd have to treat ControllerConfig like a configuration of the whole provider, i.e. not only the deployment, which I'm not sure if we'd like to do.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.