forklift: introduce forklift controller

[bennyz]

What this PR does / why we need it:
A controller to run the forklift populators, it will handle the population for the OvirtVolumePopulator and OpenstackVolumePopulator kinds

Example DVs:

apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata:
  name: my-populator-dv
  annotations:
spec:
  storage:
    dataSourceRef:
        apiGroup: forklift.konveyor.io
        kind: OvirtVolumePopulator
        name: test
    resources:
      requests:
        storage: 1G
    accessMode:
      - ReadWriteOnce
    storageClassName: nfs-csi

apiVersion: forklift.konveyor.io/v1beta1
kind: OvirtVolumePopulator
metadata:
  name: test
  namespace: default
spec:
  engineUrl: https://ovirt.fqdn
  engineSecretName: ovirt-secret
  diskId: 134db723-7ba2-47c4-b19a-975383ff1491

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

Special notes for your reviewer:
depends on #2947
Release note:

Introduce a controller to handle forklift's volume populators (ovirt, openstack)

TODO:

• Make sure all annotations are properly set
• Handle pod cleanup
• Complete tests

[kubevirt-bot]

Hi @bennyz. Thanks for your PR.

I'm waiting for a kubevirt member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[bennyz]

@akalenyu could you have a preliminary look?

[akalenyu]

@akalenyu could you have a preliminary look?

So this is fine for review, regardless of the ovirt populator still not being in?

[bennyz]

@akalenyu could you have a preliminary look?

So this is fine for review, regardless of the ovirt populator still not being in?

This fetches the ovirt populator from our repo, as this is seems the direction we'll be going for, and even if not, switching back to the original approach will not be difficult and won't affect the core controller code too much, just the boilerplate code around

[akalenyu]

cc @alromeros @mhenriks
my worry is productizing this in downstream

[akalenyu]

WDYT about including & using the sources under CDI? at least in our scripts
On other distributions (downstream) the env var will be set to something else and that will get tested instead.

[bennyz]

no problem

[akalenyu]

this is not implemented yet, right?

[bennyz]

not yet, just to clarify, we want to build it in CDI?
something like was done here: #2895 ?

[akalenyu]

yeah, any drawbacks? lets us stay completely in charge upstream if anything breaks down

[akalenyu]

not sure why you linked the closed PR, the one that merged also had the ovirt populator sources in it:
#2947

[bennyz]

in the closed one ovirt-populator is built separately which is what I assume we want here, while in #2947 it's part of cdi-importer

[akalenyu]

ah, I see. maybe for consistency let's keep it non separate (part of importer)? wdyt?

[bennyz]

published, only did a basic sanity check but looks ok

[awels]

/test all

[awels]

/test pull-cdi-goveralls

[awels]

/test pull-cdi-generate-verify

[coveralls]

coverage: 60.687% (-0.8%) from 61.503%
when pulling 84dc967 on bennyz:forklift-controller
into 6f7809e on kubevirt:main.

[awels]

/test pull-containerized-data-importer-fossa

[awels]

/test pull-containerized-data-importer-e2e-nfs

[awels]

/test pull-containerized-data-importer-e2e-ceph

[awels]

/retest-required

[akalenyu]

/cherrypick release-v1.59

[kubevirt-bot]

@akalenyu: #2983 failed to apply on top of branch "release-v1.59":

Applying: forklift: add types for forklift populators
Applying: forklift: introduce forklift controller
Applying: forklift: add CRD missing CRD suffix
Applying: forklift: fix tests
Applying: forklift: start support for mutated PVC
Applying: forklift: fix linter issues
Applying: forklift: update vendor and generated code
Applying: forklift: remove unnecessary argument
Applying: forklift: add ovirt-populator to cdi-importer
Using index info to reconstruct a base tree...
M	WORKSPACE
M	hack/build/rpm-deps.sh
Falling back to patching base and 3-way merge...
Auto-merging hack/build/rpm-deps.sh
Auto-merging WORKSPACE
CONFLICT (content): Merge conflict in WORKSPACE
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0009 forklift: add ovirt-populator to cdi-importer
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

/cherrypick release-v1.59

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.