Use the bundle API

[drewwells]

OPA has a readiness probe that leverages bundle API. This is the best way to ensure OPA is ready on initial start and guarantees it consistently keeps up to date with policy changes. Let me describe an example we are faced with the push approach.

acme service is deployed with a opa sidecar. acme becomes ready and accepts requests. opa does not have a readiness probe b/c it is unaware of what data needs to be loaded into it. acme service fails every request with 500/403 b/c opa is unable to process authorization requests to acme.

We have added a workaround to this approach by using an authorization request as a readiness probe, but this does not guarantee all types of policy data has been loaded into OPA. I would like to see other approaches to solve this problem, but it seems to us that the bundle API is the preferred way of handling it in OPA

Ref: https://www.openpolicyagent.org/docs/v0.11.0/rest-api/#health-api

[tsandall]

@drewwells can you clarify what the issue or feature request is? I'm not sure what you're looking for exactly.

kube-mgmt just replicates things from the Kubernetes API into OPA. It can replicate policies (stored in configmaps), arbitrary JSON data (also stored in configmaps), or arbitrary Kubernetes resources. The Bundle API provides another mechanism for loading policy & data into OPA. That API is not tied to Kubernetes. Bundles and kube-mgmt are therefore somewhat overlapped. You would typically use one or the other. There are some cases where you want both, e.g., relying on bundles for policy & non-k8s data distribution and then kube-mgmt for cluster-local data replication.

The Health API can be instructed to check for bundle readiness. It can't be instructed to wait for readiness on data that's being pushed into OPA though; this is somewhat fundamental since it's hard to know when the push is "done".

[drewwells]

It can't be instructed to wait for readiness on data that's being pushed into OPA though; this is somewhat fundamental since it's hard to know when the push is "done".

That is my use case here. I want to know when data is loaded, which is only currently possible from bundles. We are internally working on an operator pattern based on CRs. The CR would name any configmaps, policies, and data to load. It would package all of these assets up then push them to OPA. In this way, you can tie the OPA readiness probe to the state of the bundle.

[tsandall]

The CR would name any configmaps, policies, and data to load.

That makes sense--but I think it would need to be implemented outside of OPA--either inside your operator or inside kube-mgmt. The readiness check could then be added on the operator or kube-mgmt. I'm going to close this issue because it sounds like you've solved this inside your own operator.

EDIT: If you think we should re-open this to add more functionality into kube-mgmt, I'm open to that.

[drewwells]

Yes I do think it's useful to kube-mgmt to add readiness checks for policy + data. Our closed source operator won't be of much help to the larger community looking for declarative configuration of OPA. It should be a feature that kube-mgmt offers both declarative policy configuration and readiness checks.