Support for Multiarch Manager Operator

[Prashanth684]

This enhancement proposes a multiarch manager operator which will be a secondary OLM managed operator installable through OperatorHub. The operator will expose functionality that when configured, would allow pods to be scheduled on the appropriate architecture nodes based on the architecture of the container images.

The operator's name is chosen as such in case future functionality would need to be added related to multi-architectural workings.

xref: https://issues.redhat.com/browse/MIXEDARCH-215

Co-authored-by: Alessandro Di Stefano aleskandro@redhat.com

[Prashanth684]

/cc @aleskandro

[mtrmac]

(BTW callers of the containers/image library can point it at other locations, so the code does not necessarily need root inside the container to be able to overwrite files in /etc.)

[aleskandro]

We will not leverage a privileged/rooted pod for changing this folder and will address the actual location of the files at implementation time.

[JoelSpeed]

I think identifying the outcome of the question I had about ExcludedNamespaces vs a MatchExpressions is currently blocking any further API review, read through the rest of the enhancement though and only found a couple of things to comment on

[Prashanth684]

@JoelSpeed @ingvagabund could you PTAL the updates and let us know if you have more concerns or if all have been addressed?

[bparees]

@Prashanth684 this mostly looks ready for approval, but:

1. please go through and resolve all the open conversations assuming you have incorporated the feedback into the EP itself (and if not, then the feedback should be added to the EP, not just addressed in the comment thread).

2. there are a number of questions in the "open questions" section for which the answer seems to have significant implications on delivering the implementation/feature. They may not block you getting to a "phase 1" tech preview type implementation, but they will definitely need to be addressed prior to having a GA-ready solution imho.

e.g.:

Do we need to be aware of any unique ways to access pull secrets for
  disconnected clusters?

and

 Should the `SchedulingGateMutatingWebhook` and the `PodPlacementConfig` CR also consider a Pod Selector to filter
  pods that should be gated and inspected?

[Prashanth684]

@Prashanth684 this mostly looks ready for approval, but:

1. please go through and resolve all the open conversations assuming you have incorporated the feedback into the EP itself (and if not, then the feedback should be added to the EP, not just addressed in the comment thread).
2. there are a number of questions in the "open questions" section for which the answer seems to have significant implications on delivering the implementation/feature. They may not block you getting to a "phase 1" tech preview type implementation, but they will definitely need to be addressed prior to having a GA-ready solution imho.

e.g.:

Do we need to be aware of any unique ways to access pull secrets for
  disconnected clusters?

and

 Should the `SchedulingGateMutatingWebhook` and the `PodPlacementConfig` CR also consider a Pod Selector to filter
  pods that should be gated and inspected?

@bparees most open conversations have been resolved except for the one about the deletion of the CR which I believe can be resolved as well.

We went through the open questions section and updated it to remove some of them which we have found to not be necessary - for example the one you highlighted around investigating if disconnected installs have a special way of accessing pull secrets. After experimenting with the poc code, we found that to not be the case. The questions in the open questions section relate mainly to performance which we will only find as we experiment with the feature and collect data as it is being used. I believe there are no "blocker" questions which still need to be addressed.

As for

Should the `SchedulingGateMutatingWebhook` and the `PodPlacementConfig` CR also consider a Pod Selector to filter
   pods that should be gated and inspected?

I don't think we need this in the open questions as it can be seen as an "add-on" to the base feature which we can consider when the feature evolves and if we see a need when users start adopting it.

[bparees]

All discussions appear to be resolved with the only remaining objection/question being about the global secrets permission requirement. I don't think there's a way for this to move forward w/o read permission on global secrets, so i think we need to accept that, but the fact that the operator will have those permissions definitely needs to be taken into consideration during implementation to ensure the permission cannot be abused by users to access/use secrets they can't normally use(including using this operator to fetch metadata about images they do not have access to themselves)

/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bparees

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [bparees]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[JoelSpeed]

/lgtm
/hold are we waiting for anyone else to approve this before merge?

[Prashanth684]

/lgtm /hold are we waiting for anyone else to approve this before merge?

the only other person i'd like to take a look is @joelanford from the OLM perspective to see if there are any concerns (I don't expect any). I've asked him to take a look.

[joelanford]

Left a few comments, but nothing blocking.

/lgtm
from an OLM perspective.

[openshift-ci]

@Prashanth684: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[joelanford]

/lgtm

[Prashanth684]

/hold cancel