feat(chart): add container security context

[mikaeld]

This is a follow up to #24588 which was closed due to a fork sync'ing issue. From the original PR by @ChrisFraun:

What: Deployments can have security settings in their manifest on two levels: pod and container. However, there are some capabilities only configurable in one of the respective levels(https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/#securitycontext-v1-core). This PR sets a default configuration for container securityContext, which denies privilege escalation and drops all POSIX capabilities. These are and should be standard settings in the context of Kubernetes. It also adds the possibility of running Airflow in an Kubernetes environment without PSP (to be removed in v1.25 https://kubernetes.io/docs/concepts/security/pod-security-policy/), but with OpenPolicyAgent (a or possibly the PSP substitute) with the same capabilities as a restricted PSP instead.
Why: This missing configuration restricts Airflow from being used with the simple upstream helm chart without modifications/unnecessary maintenance. This especially applies to the restricted policy use in OPA. The specific setting in this PR is not inherited from podSecurityContext(pod level) in securityContext(container level).
Problem: There is already a securityContext in the values.yaml, however, this should also be be called podSecurityContext since it's on pod level, but it isn't. To not break backwards compatibility of Airflow, this PR hardcodes the respective capabilities on container level for statsd, scheduler and webserver.
The other possibility would be to introduce a containerSecurityContext in the values.yaml, which is a made up word since it is commonly called scurityContext.
Benefit in either case would be a more secure deployment.

closes: #27612

---

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[mikaeld]

I've fixed most of the unit test issues in tests/charts. However, there are failing tests left in tests/charts/airflow_aux/test_pod_template_file.py which I haven't been able to fix on my own. I tried replicating the errors from running the unit tests in breeze by using helm locally but I do not see any errors. I would appreciate another set of eyes because all the remaining failing tests seem to fail for the same reason.

@potiuk @jedcunningham @ChrisFraun Thanks!

[potiuk]

I rebased it to latest main to check if it was not a temporary issue but if it does not help, I think the only way is to follow the exact steps in https://github.com/apache/airflow/blob/main/TESTING.rst#helm-unit-tests and see if it can be reproduced (including rebuilding the latest image) - this will run the tests in the same conditions as they are run in CI and you can also enter breeze image and run the tests from there - looks like for some reason, trying to render those helm templates failed, but what was the reason is hard to say until you manually run the exact helm command with all the same parameters that were used in the tests or run the tests.

It can also be (though not very likely) that this is a side effect of runnning the tests one after another and then the reason for the side effects should be investigated.

[mikaeld]

@potiuk I've fixed the tests!

[potiuk]

great. I think some comments on that from a few people involved in chart more than me would be great, let's wait for them - now that the tests are green it;s good for review.

[potiuk]

@jedcunningham @dstandish @ephraimbuddy - it seems like a good change - do you see some problems with it?

[mikaeld]

@jedcunningham Thanks for the feedback, I've addressed the issues. This is ready for re-review!

[jedcunningham]

Last round 🤞

[mikaeld]

@jedcunningham ready for review!

[jedcunningham]

#protm

Thanks @ChrisFraun and @mikaeld! Really appreciate all the effort and persistence to get this change in 🍺