OPA Compliance Test

Generator of compliance tests for OPA IR evaluators. Consumes OPA yaml test-cases, and produces a corresponding set of json test-cases that includes compiled IR plans.

How to Use

1. Initialize/update the OPA submodule
   1. In the project root: git submodule update --init
   2. (optional) Check out wanted OPA version: cd opa && git checkout <commit|branch|tag>
2. go mod download
3. go run . <TARGET_PATH>

On completion, TARGET_PATH will contain a set of folders, roughly corresponding to test categories. Each folder will contain a set of .json files, with the following structure:

• cases (list of dictionaries): a list of test-case definitions
  • note (string): the name of the test-case
  • plan (dictionary): the compiled IR for the test-case
  • entrypoints (list of string): a list of entrypoints in the IR to execute
  • data (dictionary): the data document to execute each entrypoint with
  • input (dictionary): the input document to execute each entrypoint with
  • want_plan_result (dictionary): a dictionary with the expected output, mapped by entrypoint name. (mutually exclusive with want_error_code and want_error)
  • want_error_code (string): if present, the error code of an expected error (mutually exclusive with want_plan_result)
  • want_error (string): if present, the error message of an expected error (mutually exclusive with want_plan_result)
  • strict_error (boolean): if present, errors generated by built-in functions should not produce an undefined result, instead, execution should be halted with that error.

Note: when an error is expected, the test-case definition doesn't indicate which entrypoint will spawn it. Therefore, when multiple entrypoints are present, all must be executed, but not all may spawn the expected error (but at least one must).