Unable to deployment Ora Operator with limit access user.

[alinaqvion-hash]

We tried deploying the Oracle Database Operator on the Customer Kubernetes platform, we have observed multiple issues related to platform policies, RBAC restrictions, and security constraints.

The operator YAML expect cluster-wide privileges that are not permitted under our current ngnp-nokia-power user group thru which we are doing the deployment.
Several cluster policies (Kyverno, RBAC, cert-manager restrictions) prevent applying the manifests as-is.

Note that we have limit access account for doing all the deployment limited to namespace only, Cluster level roles not possible. Also, many roles created by operator we don't have access.
We need to adapt to deploy in customer environment where your support and input required.

Observed Issues:

1. Namespace Restrictions (Kyverno Policy)
   Policy: restrict-system-namespaces
   Error: Namespaces with the suffix -system cannot be created.
   Impact: Operator manifests default to using namespaces with -system suffix, which are blocked by cluster admission policies.
   Fix Options: We can use different namespace, don't' see any issues

2. RBAC Privilege Escalation Errors
   Errors such as:
   user "mohd_ali.naqvi.ext@nokia.com" is attempting to grant RBAC permissions not currently held
   Operator attempt to create ClusterRoles, RoleBindings, and rules with privileged verbs (e.g., pods/exec, configmaps/status, /metrics endpoint, etc.), which are not allowed.
   Only Cluster Admins can apply these RBAC rules.
   Fix Options:
   We can apply only namespace-scoped resources (Deployment, Service, ConfigMaps, Secrets) instead of ClusterRole, use Role (for our namespace). Any issue we deploy operator, oracle db and other application pods in same namespace then ClusterRole no longer needed?

Operator Specific failure ->> How to handle this via operator, can be adapted?
oracle-database-operator-metrics-reader
        Fails due to non-resource RBAC (/metrics). → Not permitted.
ngnp-oracleop-ngnp-oracleop-helm-test-role
        Invalid RBAC: references apps/pods (pods are core API, not apps)
oracle-database-operator-leader-election-role
        Requests configmaps/status subresource. Likely over-specified; configmaps should suffice.
oracle-database-operator-manager-role
    Over-privileged:
containers subresource (rarely granted).
pods/exec with patch/update/delete.
secrets/status.
Incorrect apps group for pods/configmaps.

3. Cert-Manager Resources Blocked
   Errors:
   certificates.cert-manager.io is forbidden
   issuers.cert-manager.io is forbidden
   Operator requires cert-manager for webhook TLS.
   Fix Options:

4. Security Policy (Kyverno) Violations
   While attempting helm upgrade --install, multiple Kyverno policies blocked the Operator Pods:
   Root / Privileged execution not allowed
   runAsNonRoot must be true
   runAsGroup must be > 0
   allowPrivilegeEscalation must be false
   Filesystem & Capabilities
   Root filesystem must be read-only
   ALL Linux capabilities must be dropped
   CAP_NET_RAW must be dropped
   Probes & Seccomp
   Readiness/Liveness probes missing
   Seccomp profile must be set to RuntimeDefault or Localhost

Events captured:
policy require-run-as-nonroot → fail
policy require-ro-rootfs → fail
policy require-non-root-groups → fail
policy disallow-privilege-escalation → fail
policy drop-all-capabilities → fail
policy require-pod-probes → fail
policy restrict-seccomp-strict → fail
policy drop-cap-net-raw → fail
Impact: Even if RBAC issues are resolved, Operator Pods are denied at admission due to non-compliance with cluster’s security policies.

[z2349211@oa89599c7091 (prod) ~]$ helm upgrade --install ngnp-oracleop ngnp-oracleop -n ngnp-nokia-test
Release "ngnp-oracleop" does not exist. Installing it now.
W0910 14:36:39.490168 54630 warnings.go:70] spec.privateKey.rotationPolicy: In cert-manager >= v1.18.0, the default value changed from Never to Always.
Error: 4 errors occurred:

• roles.rbac.authorization.k8s.io "oracle-database-operator-metrics-reader" is forbidden: user "mohd_ali.naqvi.ext@nokia.com" (groups=["ngnp-nokia-poweruser" "offline_access"
  "uma_authorization" "default-roles-schiff" "system:authenticated"]) is attempting to grant RBAC permissions not currently held:
  {NonResourceURLs:["/metrics"], Verbs:["get"]}
• roles.rbac.authorization.k8s.io "ngnp-oracleop-ngnp-oracleop-helm-test-role" is forbidden: user "mohd_ali.naqvi.ext@nokia.com" (groups=["ngnp-nokia-poweruser" "offline_acce
  ss" "uma_authorization" "default-roles-schiff" "system:authenticated"]) is attempting to grant RBAC permissions not currently held:
  {APIGroups:["apps"], Resources:["pods"], Verbs:["get" "list"]}
• roles.rbac.authorization.k8s.io "oracle-database-operator-leader-election-role" is forbidden: user "mohd_ali.naqvi.ext@nokia.com" (groups=["ngnp-nokia-poweruser" "offline_a
  ccess" "uma_authorization" "default-roles-schiff" "system:authenticated"]) is attempting to grant RBAC permissions not currently held:
  {APIGroups:[""], Resources:["configmaps/status"], Verbs:["get" "update" "patch"]}
• roles.rbac.authorization.k8s.io "oracle-database-operator-manager-role" is forbidden: user "mohd_ali.naqvi.ext@nokia.com" (groups=["ngnp-nokia-poweruser" "offline_access" "

Details of above issues:
oracle-database-operator-metrics-reader
Fails because of:
nonResourceURLs: ["/metrics"]
verbs: ["get"]

Don’t have rights to grant non-resource RBAC.
ngnp-oracleop-ngnp-oracleop-helm-test-role
Fails because it tries to grant:
apiGroups: ["apps"]
resources: ["pods"]
verbs: ["get","list"]

Pods are not in apps API group, only in "" (core).
oracle-database-operator-leader-election-role
Fails because of:
apiGroups: [""]
resources: ["configmaps/status"]
verbs: ["get","update","patch"]

configmaps/status is a subresource.

oracle-database-operator-manager-role
Fails because it tries to grant too much:
"containers" subresource
"pods/exec" with update/patch/delete
"secrets/status"
"apps" group for pods, configmaps

Many verbs don’t have rights to grant

Pod Security Compliance:
Operator manifests must be updated to comply with Kyverno:
Add probes (liveness/readiness)
Set runAsNonRoot: true, runAsGroup > 0
allowPrivilegeEscalation: false
Drop ALL capabilities including CAP_NET_RAW
Enable readOnlyRootFilesystem: true
Set seccompProfile: RuntimeDefault

[psaini79]

@alinaqvion-hash thanks Did you explore the namespace deployment mentioned under the namespace-scaope-deployment under https://github.com/oracle/oracle-database-operator/tree/main?

You do not need to run it as cluster scope, you can use namespace scope deployment.