fix addons map race condition

[prezha]

fixes #15705

from the issue's logs, we see that we can hit a race condition over concurrent reading and writing to the addons map - eg, we have addons.Start() called asynchronously, that might try to update the addons map (eg, here and here), while we also have multiple places where we potentially try to read the addons map at the same time (as being a part of the cluster config)

now, because it makes sense to try to enable addons concurrently, this pr tries to guard against that situation by splitting the addons.Start() into addons.Enable() (that is thread-safe and does the actual work of concurrently enabling addons) and addons.ToEnable() and addons.UpdateConfig() that are not thread-safe and therefore should be called synchronously before and after the addons.Enable is started as a goroutine
this way, we do not need to try to cover with locks all the places where we concurrently read the map - currently and in the future

attached are the logs from local test runs for TestAddons and TestMultiNode on ubuntu 20.04, ubuntu 22.04 vms and opensuse with docker & kvm2 driver and containerd container runtime - all passed:

• TestAddons - docker@ubuntu2004 - PASS.log
• TestAddons - docker@ubuntu2204 - PASS.log
• TestAddons - kvm2@opensuse - PASS.log
• TestMultiNode - docker@ubuntu2004 - PASS.log
• TestMultiNode - docker@ubuntu2204 - PASS.log
• TestMultiNode - kvm2@opensuse - PASS.log

[spowelljr]

/ok-to-test

[minikube-pr-bot]

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15706) |
+----------------+----------+---------------------+
| minikube start | 54.7s    | 54.1s               |
| enable ingress | 27.1s    | 27.4s               |
+----------------+----------+---------------------+

Times for minikube start: 54.3s 53.5s 55.0s 56.3s 54.5s
Times for minikube (PR 15706) start: 53.3s 54.5s 54.1s 54.7s 53.7s

Times for minikube ingress: 27.7s 27.8s 27.3s 24.2s 28.7s
Times for minikube (PR 15706) ingress: 30.3s 28.2s 25.2s 27.7s 25.7s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 15706) |
+----------------+----------+---------------------+
| minikube start | 26.2s    | 28.2s               |
| enable ingress | 34.2s    | 21.9s               |
+----------------+----------+---------------------+

Times for minikube start: 26.3s 25.3s 26.6s 26.5s 26.3s
Times for minikube (PR 15706) start: 26.5s 28.9s 29.5s 29.2s 27.0s

Times for minikube ingress: 22.0s 22.5s 21.5s 82.0s 23.1s
Times for minikube (PR 15706) ingress: 23.0s 20.0s 22.0s 22.0s 22.5s

docker driver with containerd runtime

+-------------------+----------+---------------------+
|      COMMAND      | MINIKUBE | MINIKUBE (PR 15706) |
+-------------------+----------+---------------------+
| minikube start    | 22.0s    | 22.1s               |
| ⚠️  enable ingress | 29.7s    | 37.4s ⚠️             |
+-------------------+----------+---------------------+

Times for minikube start: 21.3s 20.8s 21.6s 23.8s 22.4s
Times for minikube (PR 15706) start: 21.8s 21.4s 23.9s 21.3s 22.0s

Times for minikube ingress: 31.5s 20.5s 32.5s 32.5s 31.5s
Times for minikube (PR 15706) ingress: 80.5s 32.7s 21.5s 20.5s 31.5s

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: prezha, spowelljr

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [prezha,spowelljr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[minikube-pr-bot]

These are the flake rates of all failed tests.

Environment	Failed Tests	Flake Rate (%)
Hyperkit_macOS	TestFunctional/parallel/ImageCommands/ImageLoadDaemon (gopogh)	0.00 (chart)
Hyperkit_macOS	TestFunctional/parallel/ImageCommands/ImageLoadFromFile (gopogh)	0.00 (chart)
Hyperkit_macOS	TestFunctional/parallel/ImageCommands/ImageReloadDaemon (gopogh)	0.00 (chart)
Hyperkit_macOS	TestFunctional/parallel/ImageCommands/ImageSaveDaemon (gopogh)	0.00 (chart)
Hyperkit_macOS	TestFunctional/parallel/ImageCommands/ImageSaveToFile (gopogh)	0.00 (chart)
Hyperkit_macOS	TestFunctional/parallel/ImageCommands/ImageTagAndLoadDaemon (gopogh)	0.00 (chart)
Hyperkit_macOS	TestFunctional/parallel/ImageCommands/Setup (gopogh)	0.00 (chart)
Hyperkit_macOS	TestFunctional/serial/CacheCmd/cache/add_local (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/CertSync (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/DockerEnv/powershell (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/ImageCommands/ImageBuild (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/ImageCommands/ImageListJson (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/ImageCommands/ImageListShort (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/ImageCommands/ImageListTable (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/ImageCommands/ImageListYaml (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/ImageCommands/ImageLoadDaemon (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/ImageCommands/ImageLoadFromFile (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/ImageCommands/ImageReloadDaemon (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/ImageCommands/ImageRemove (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/ImageCommands/ImageSaveDaemon (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/ImageCommands/ImageSaveToFile (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/ImageCommands/ImageTagAndLoadDaemon (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/MySQL (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/NodeLabels (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/PersistentVolumeClaim (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/ServiceCmd (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/ServiceCmdConnect (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/StatusCmd (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/TunnelCmd/serial/WaitService/Setup (gopogh)	0.00 (chart)
Hyper-V_Windows	TestFunctional/parallel/UpdateContextCmd/no_changes (gopogh)	0.00 (chart)
More tests...	Continued...

Too many tests failed - See test logs for more details.

To see the flake rates of all tests by environment, click here.