Non-deterministic builtin functions can break policies run through partial evaluation

[philipaconrad]

Short description

When building a bundle with:

• An optimization level > 0 and
• opa.runtime() (or other builtin functions) used in the policy

The resulting optimized policy can wind up missing critical rules entirely. This occurs because the optimizer attempts partial evaluation at bundle build time, which can have nonsensical results for some non-deterministic Rego builtins that query the environment. (http.send, opa.runtime, etc.)

Short example

An easy example of this is the following policy:
test.rego:

package test

allow {
	opa.runtime().env.MY_APP_ID == input.details.id
}

Steps to reproduce

1. opa build ./test/ -b -O=2 -e test/allow
2. Check bundle, and see the following:

bundle.tar.gz/optimized/test.rego:

package test

default allow = false

Expected behavior

The optimizer should not be attempting to partially evaluate non-deterministic builtin functions like opa.runtime. Adding that one builtin to the "unsafe for partial eval" list would be enough on its own for solving some immediate issues, but it's probably best just to remove all of them from partial eval for safety.

Workaround techniques

You can currently force the optimizer to leave the opa.runtime calls alone by wrapping them in constructs that are opaque to it, like aggregate comprehensions, and else rules. (Thanks @tsandall for figuring this out!)

Example of else rule wrapping:

package test

allow {
	my_app_id == input.details.id
}

# HACK: The else rule here allows us to hide the opa.runtime call from partial eval in this policy.
my_app_id := x {
  x := opa.runtime().env.MY_APP_ID
} else := ""

Additional context

• Non-deterministic builtins are now marked as such, as of PR Support Caching Output of Non-Deterministic Builtins #4926.