Allowing adding arbitrary flags to kubernetes core components through plugins

[omar-nahhas]

This PR Is to allow adding specific configuration flags to the kube-proxy, kubelet and kube-scheduler through plugins, it is an extension of the currently existing functionality to add fags to the kubeapi and kube-controller-manager.

In the plugin.yaml you can add flags like follows:

Then they will be injected on the configuration of the specific components:

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: redbaron

If they are not already assigned, you can assign the PR to them by writing /assign @redbaron in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov-io]

Codecov Report

Merging #1564 into master will increase coverage by 0.05%.
The diff coverage is 88.88%.

@@            Coverage Diff             @@
##           master    #1564      +/-   ##
==========================================
+ Coverage    25.8%   25.85%   +0.05%     
==========================================
  Files          98       98              
  Lines        5057     5062       +5     
==========================================
+ Hits         1305     1309       +4     
- Misses       3605     3606       +1     
  Partials      147      147

Impacted Files	Coverage Δ
pkg/api/types.go	0% <ø> (ø)	⬆️
pkg/model/node_pool_config.go	34.88% <ø> (ø)	⬆️
pkg/model/config.go	54.76% <ø> (ø)	⬆️
pkg/model/compiler.go	55.69% <100%> (+0.56%)	⬆️
pkg/model/stack_new.go	33.33% <75%> (+1.03%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5f14e23...f0c7bba. Read the comment docs.

[omar-nahhas]

/assign @davidmccormick

[omar-nahhas]

I have tested this by adding the flag metricsBindAddress 0.0.0.0 to the kube-proxy (through a plugin) and then spun the cluster. The cluster spun successfully and my external Prometheus could finally scrape all the kube-proxy(ies)

[mumoshu]

A few comments on naming. This seems like a great idea though!

[omar-nahhas]

@mumoshu I have refactored it and test it again by spinning a cluster.

[davidmccormick]

/lgtm

[davidmccormick]

Thanks @omar-nahhas for your continued support! 🙏

[paalkr]

I'm not sure if I understand how to include a plugin that will allow me to configure the kube-controller-manager flags. My goal is to add this flag
--horizontal-pod-autoscaler-downscale-stabilization-window=10m

Any guidance or examples would be highly appreciated.