Consolidate the katib-cert-generator to the katib-controller

[tenzen-y]

What this PR does / why we need it:
I modified the katib-controller and removed the katib-cert-generator to consolidate the katib-cert-generator to the katib-controller.

By this PR, katib is started in the following steps:

1. All probe servers are registered to the manager.
2. Start the manager.
3. Start cert-generator as a part of the manager.
4. Controllers and webhooks are registered to the manager after the certs are generated (certsReady channel is closed)

In the katib-controller, I mainly worked on the below tasks:

1. Expand the KatibConfig API like this:

---
apiVersion: config.kubeflow.org/v1beta1
kind: KatibConfig
init:
+ certGenerator:
+   enable: true
+   webhookServiceName: test-svc
+   webhookSecretName: test-secret
  controller:
    webhookPort: 8443
...

5. Add the cert-generator to the manager as a Runnable:

katib/pkg/cert-generator/v1beta1/generator.go

Lines 50 to 87 in 3fe9369

	// CertGenerator is the manager to generate certs.
	type CertGenerator struct {
	namespace string
	serviceName string
	secretName string
	kubeClient client.Client
	certsReady chan struct{}
	certs *certificates
	fullServiceDomain string
	}
	var _ manager.Runnable = &CertGenerator{}
	var _ manager.LeaderElectionRunnable = &CertGenerator{}
	func (c *CertGenerator) Start(ctx context.Context) error {
	if err := c.generate(ctx); err != nil {
	return err
	}
	// Close a certsReady means start to register controllers to the manager.
	close(c.certsReady)
	return nil
	}
	func (c *CertGenerator) NeedLeaderElection() bool {
	return true
	}
	// AddToManager adds the cert-generator to the manager.
	func AddToManager(mgr manager.Manager, config configv1beta1.CertGeneratorConfig, certsReady chan struct{}) error {
	return mgr.Add(&CertGenerator{
	namespace: consts.DefaultKatibNamespace,
	serviceName: config.WebhookServiceName,
	secretName: config.WebhookSecretName,
	kubeClient: mgr.GetClient(),
	certsReady: certsReady,
	})
	}

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #2149

/hold
Blocked on #2176

Checklist:

• Docs included if any changes are user facing

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tenzen-y

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [tenzen-y]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[tenzen-y]

I removed this health check since this is duplicated with L154-L157.

	if err = mgr.AddHealthzCheck("healthz", hookServer.StartedChecker()); err != nil {
		log.Error(err, "Add webhook server health checker to the manager failed")
		os.Exit(1)
	}

[andreyvelich]

Thank you for this great contribution @tenzen-y!
Please can you rebase your PR so I can review it this week.

[tenzen-y]

Thank you for this great contribution @tenzen-y!
Please can you rebase your PR so I can review it this week.

Sure.

[tenzen-y]

I will rebase this PR once #2176 is merged into the master branch.

[tenzen-y]

@andreyvelich This PR is ready for review. Please take a look.

[andreyvelich]

Thank you for this great improvement @tenzen-y!
I left a few initial comments.

[tenzen-y]

Rebased.

[johnugeorge]

/lgtm

[johnugeorge]

/hold cancel

[andreyvelich]

Thank you for this great improvement @tenzen-y 🎉 .
Let me rebase my PR: #2018

[tenzen-y]

Thanks, everyone!