fix: Mount /usr/local/share/ca-certificates folder into kube-controller-manager

[ericsuhong]

Reason for Change:
To support airgap clouds, we need to mount /usr/local/share/ca-certificates folder into kube-controller-manager

Credit Where Due:

Does this change contain code from or inspired by another project?

• No
• Yes

If "Yes," did you notify that project's maintainers and provide attribution?

• No
• Yes

Requirements:

• uses conventional commit messages
• includes documentation
• adds unit tests
• tested upgrade from previous version

Notes:
kube-controller-manager was working fine in airgap clouds on K8s 1.16 without requiring this change, so I assume this change is only needed for K8s 1.17+

[jackfrancis]

/lgtm

[acs-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ericsuhong, jackfrancis

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [jackfrancis]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[jackfrancis]

/azp run pr-e2e

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[codecov]

Codecov Report

Merging #4001 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #4001   +/-   ##
=======================================
  Coverage   73.68%   73.68%           
=======================================
  Files         147      147           
  Lines       23161    23161           
=======================================
  Hits        17067    17067           
  Misses       4979     4979           
  Partials     1115     1115           

Impacted Files	Coverage Δ
pkg/engine/templates_generated.go	48.97% <ø> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4262e4b...dc7fab1. Read the comment docs.

[jadarsie]

aks-engine/parts/k8s/cloud-init/artifacts/cse_customcloud.sh

Lines 16 to 23 in 2e2dfd7

	local i=1
	for cert in $(echo $CUSTOM_CLOUD_ROOT_CERTIFICATES | tr ',' '\n')
	do
	echo $cert | base64 -d > "/usr/local/share/ca-certificates/customCloudRootCertificate$i.crt"
	((i++))
	done
	update-ca-certificates

@ericsuhong, why is this ^^^ not enough? The man page for update-ca-certificates seems to indicates that the extra mount should not be required: http://manpages.ubuntu.com/manpages/xenial/man8/update-ca-certificates.8.html

[ericsuhong]

@jadarsie We encountered this issue on Ubuntu 18.04 with K8s 1.18 in airgap cloud where if you download and update root certificates, cloud-controller won't recognize new root certs until we added that additional mount point.

[jadarsie]

right, I understand the symptom, I just wonder why that's the case.
Is update-ca-certificates printing out a warning or error message?
Please let me know if eventually this becomes clear. Thx.

[ericsuhong]

I will be honest, I could not pinpoint the root cause exactly. I also faced an issue where it sometimes required explicit node reboot for kube-controller-manager to reflect the cert update as well... (does not happen for other services running on agent nodes).

update-ca-certificates command did not print out any warning or error messages. It should not require node reboots neither...

For now, since this is a pretty safe change, I've decided to make this change and move on due to tight airgap deadlines.