Learning Rego - Confusion regarding input behavior when writing unit tests

[mooneeb]

Hello! Hope you're all doing well.

TL;DR: Confused on how the custom imports in Rego work and Not sure why unit test cases are passing when it should fail.

---

I'm trying to write unit tests for certain policies that I have written, but I'm confused on how to write it.

My directory structure where i'm storing my input, my policies, and unit tests is as follows:

.
└── hub/
    └── pkg/
        └── policies/
            ├── rego_test_jsons/
             │   └── Internal_deployment.json.    <------- This contains the input JSON
            └── sources/
                ├── policy_detector.json. <----- The policy file I want to unit test
               └── policy_detector_test.json   <---- The unit test file

And these are how each file is defined

policy_detector.rego

package policy.special

import future.keywords.contains
import future.keywords.if
import future.keywords.in

resources contains msg if {
	input.kind == "DaemonSet"
	some container in input.spec.template.spec.containers
	contains(container.image, "internalstorage")
  msg := {"kind": input.kind, "name": input.metadata.name, "namespace": input.metadata.namespace, "detail": {"image": container.image}}
} {
	input.kind == "DaemonSet"
	some container in input.spec.template.spec.containers
	containsRegistryURL(container.image)
    msg := {"kind": input.kind, "name": input.metadata.name, "namespace": input.metadata.namespace, "detail": {"image": container.image}}
}

containsRegistryURL(image) {
        data.public_registries_urls[_] == getRegistryURL(image)
}

getRegistryURL(image) = registry_url {
        url := split(image, ":")
        registry_url = url[0]
}

found if count(resources) > 0

policy_detector_test.rego

package policy.special

import data.policy_detector.resources

test_resources_found{
	found with input as data.rego_test_jsons.Internal_deployment.Deployment
}

Inter_deployment.json

{
    "Internal_deployment": {
          "Deployment" : { 
                // deployment manifest here 
           }
           "public_registry_urls": ["false"]
    }
}

And i'm running opa test . -v at the parent folder i.e. /hub/pkg/policies.

I've used the print() command to verify that the unit test is receiving the correct input via the data store but I don't understand why the unit test case is still passing when it should fail because both the conditions are false (image name is actually just storage and public_registry_urls array does not contain the registry_url work/storage

Any help would be greatly appreciated.

[johanfylling]

Hi @mooneeb , replicating your setup, but cleaning up the input to valid json

{
    "Internal_deployment": {
        "Deployment": { 
        },
        "public_registry_urls": ["false"]
    }
}

I get:

$ opa test . -v
FAILURES
--------------------------------------------------------------------------------
data.policy.special.test_resources_found: FAIL (2.360375ms)

  query:1                                 Enter data.policy.special.test_resources_found = _
  sources/policy_detector_test.rego:5     | Enter data.policy.special.test_resources_found
  sources/policy_detector.rego:28         | | Enter data.policy.special.found
  sources/policy_detector.rego:28         | | | Fail gt(__local13__, 0)
  sources/policy_detector_test.rego:6     | | Fail data.policy.special.found with input as __local14__
  query:1                                 | Fail data.policy.special.test_resources_found = _

SUMMARY
--------------------------------------------------------------------------------
sources/policy_detector_test.rego:
data.policy.special.test_resources_found: FAIL (2.360375ms)
--------------------------------------------------------------------------------
FAIL: 1/1

Updating the input json to:

{
    "Internal_deployment": {
          "Deployment" : { 
            "kind": "DaemonSet",
            "spec": {
                "template": {
                    "spec": {
                        "containers": [
                            {
                                "image": "internalstorage"
                            }
                        ]
                    }
                }
            },
            "metadata": {
                "name": "foo",
                "namespace": "bar"
            }
           },
           "public_registry_urls": ["false"]
    }
}

I get:

$ opa test . -v
sources/policy_detector_test.rego:
data.policy.special.test_resources_found: PASS (790.708µs)
--------------------------------------------------------------------------------
PASS: 1/1

Could there be something at play in your local setup that isn't part of your problem description here?

[mooneeb]

Firstly, thank you for such a prompt response!

And yes, the mistake was in my understanding and not the code itself, which is slightly embarrassing. But thank you for helping me clear it up!

I did have two follow up questions though if you could help me with:

1. in policy_detector_test.rego even after omitting the import statement, the test continued to work as expected. I'm confused as to Rego's behavior regarding this.

2. Let me set the scene. In policy_detector.rego we are using data.public_registries_urls[_] for verification within one of the rules

containsRegistryURL(image) {
        data.public_registries_urls[_] == getRegistryURL(image)
}

I modified policy_detector.rego to the following:

package policy.special

import future.keywords.contains
import future.keywords.if
import future.keywords.in

resources contains msg if {
	input.kind == "DaemonSet"
	some container in input.spec.template.spec.containers
	containsRegistryURL(container.image)
    msg := {"kind": input.kind, "name": input.metadata.name, "namespace": input.metadata.namespace, "detail": {"image": container.image}}
}

containsRegistryURL(image) {
	data.public_registries_urls[_] == getRegistryURL(image)
}

getRegistryURL(image) = registry_url {
	url := split(image, ":")
	registry_url = url[0]
}

found if count(resources) > 0

I modified policy_detector_test.rego to the following:

package policy.special

test_resources_found_DaemonSet{
	found with input as data.rego_test_jsons.Internal_deployment.Deployment
           with data.public_registry_urls as data.rego_test_jsons.Internal_deployment.public_registry_urls
}

And when I used the following JSON as input (which you kindly expanded for me, I apologize for not providing a detailed JSON beforehand)

{
    "Internal_deployment": {
          "Deployment" : { 
            "kind": "DaemonSet",
            "spec": {
                "template": {
                    "spec": {
                        "containers": [
                            {
                                "image": "docker.io/internalstorage:latest"
                            }
                        ]
                    }
                }
            },
            "metadata": {
                "name": "foo",
                "namespace": "bar"
            }
           },
           "public_registry_urls": ["docker.io/internalstorage"]
    }
}

The OPA Test case passes.

My question is, why does it pass when in the policy_detector.rego we're using data.public_regestries_url while in policy_detector_test.rego we've passed with data.public_registry_urls as data.rego_test_jsons.Internal_deployment.public_registry_urls?

[johanfylling]

1. An import statement is creating an alias for another package or rule. So when you're importing data.policy_detector.resources you're saying that for every usage of resources in this file, expand it to data.policy_detector.resources. Unless you have other rego files containing the policy_detector.* package, your import is actually not referring to an existing package. For an import to have an effect, the aliased name (resources in this case) must be used somewhere in the module containing the import statement, and the referenced package/rule/function must be defined (i.e. exist). (Looks like your import is erroneously referring to a file name, and not an existing package). Another thing about packages and imports: if two modules (files) are declaring the same package, no imports are necessary. In fact, you can completely omit imports in favor of spelling out the complete path of a reference; e.g. data.policy.special.resources; but imports make your code less verbose (and if what you're referencing is within the same package as the call-site, you don't need to spell out the complete package either, as it's assumed within the context).
2. I might be misunderstanding your question here, but in policy_detector_test.rego you're calling rule data.policy.special.found, and the with modifier is telling the evaluator that for every instance of data.public_registry_urls encountered while evaluating said found rule, replace it with a reference to data.rego_test_jsons.Internal_deployment.public_registry_urls. Which is, effectively, ["docker.io/internalstorage"] as declared in the json file.

[mooneeb]

1. Ahhhhh that makes so much more sense. So since both the test and the detector exist in the same package, i don't need any import since rego searches for and detects the existence of found within other modules and runs the relevant test. Makes sense. Thank you for such a detailed explanation!

2. The evaluator is saying that for every instance of data.public_registry_urls replace it with the reference that I've specified in the JSON. That is my understanding as well. But if you look at the policy_detector.rego in the above comment, it doesn't have data.public_registry_urls. It instead has data.public_registries_urls so the query should not work right? But from my testing, the unit test is still passing when I provide it with the correct reference ["docker.io/internalstorage"] and failing when I provide it with an incorrect reference, which should not be the case. The query should not be working at all because data.public_registry_urls != data.public_registries_urls

[johanfylling]

data.public_registry_urls != data.public_registries_urls

Sorry, I completely missed this when looking at your code 😄 .

When running the latest set of code you present, I see the test failing, but if I change the with statement to

with data.public_registries_urls as data.rego_test_jsons.Internal_deployment.public_registry_urls

then the test passes. Is there perhaps some crud in your local env, not visible here, that's messing with your results?