[Feature Track] Active/passive HA

[pmorie]

This issue is for the active/passive HA feature track, document here.

/kind feature

[pmorie]

Haha, we're not done. Next steps here:

• Determine how to horizontally scale webhooks
• Determine whether to separate reconcilers and webhooks for the same resource
• Adopt health checks in sharedmain #1048

[vagababov]

Do I miss something about Webhooks just being naturally Horizontally scalable by virtue of being stateless (well modulo certs)

[pmorie]

@vagababov The webhook parts are naturally horizontally scalable. We have adopted an idiosyncratic architecture of running the reconcilers and webhooks for resources in the same process. We decided to introduce the additional main methods to insulate webhook binaries from the leader election changes for now. Personally, I would rather migrate the reconcilers into the main controllers and disentangle them from the webhooks; we skipped doing it initially because we didn't easily align in a few beats of discussion.

[vagababov]

Ah, I didn't know you run them combined :)