CASMPET-6915: use newer plugin image and change it to daemonset

[bo-quan]

Summary and Scope

Due to limitations of server-side load balancing in kubernetes, especially with OPA as it uses GRPC protocol leveraging persistent connections, we often run into situations where only 1 or 2 OPA ingressgateway pods are used. This has exposed OPA memory leakage bug found in older OPA envoy plugin versions. This PR attempts to address the issue by changing OPA deployment to daemonset and uses a kubernetes beta feature that improves load balancing, in addition to using a newer OPA envoy plugin version v0.62.0 that has fixes for a memory leakage issue (open-policy-agent/opa#5320).

Issues and Related PRs

List and characterize relationship to Jira/Github issues and other pull requests. Be sure to list dependencies.

• Resolves CASMPET-6915
• Change will also be needed in <insert branch name here>
• Future work required by [issue id](issue link)
• Documentation changes required in [issue id](issue link)
• Merge with/before/after <insert PR URL here>

Testing

List the environments in which these changes were tested.

Tested on:

• fanta
• Local development environment
• Virtual Shasta

Test description:

Upgraded the chart, and verified that OPA has been changed to daemonset and is functioning as expected.

• Were the install/upgrade-based validation checks/tests run (goss tests/install-validation doc)?
• Were continuous integration tests run? If not, why?
• Was upgrade tested? If not, why? Y
• Was downgrade tested? If not, why? Y
• Were new tests (or test issues/Jiras) created for this change?

Risks and Mitigations

Low.

Pull Request Checklist

• Version number(s) incremented, if applicable
• Copyrights updated
• License file intact
• Target branch correct
• CHANGELOG.md updated
• Testing is appropriate and complete, if applicable
• HPC Product Announcement prepared, if applicable