genpolicy: support insecure registries and hermetic environments

[burgerdev]

Which feature do you think can be improved?

Policy generation with genpolicy. Specifically, image pulling for dm-verity calculation.

How can it be improved?

Add support for insecure registries, at least local ones. Many build systems restrict network access for security and reproducibility, and hermetic build rules are encouraged (e.g. Bazel, Nix). Local insecure registries would be an option for this use case, and they are supported by OCI tooling like docker, podman, crane, skopeo.

Additional Information

What I'm trying to do:

crane registry serve --address localhost:8080 &

curl -sLO https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/pods/simple-pod.yaml
curl -sLO https://raw.githubusercontent.com/kata-containers/kata-containers/main/src/tools/genpolicy/rules.rego
curl -sLO https://raw.githubusercontent.com/kata-containers/kata-containers/main/src/tools/genpolicy/genpolicy-settings.json

genpolicy -y simple-pod.yaml

sed -i 's|nginx:|localhost:8080/nginx:|' simple-pod.yaml

genpolicy -y simple-pod.yaml. # <--- FAILS

Error message:

thread 'main' panicked at src/registry.rs:112:17:
Failed to pull container image manifest and config - error: RequestError(
    reqwest::Error {
        kind: Request,
        url: Url {
            scheme: "https",
            cannot_be_a_base: false,
            username: "",
            password: None,
            host: Some(
                Domain(
                    "localhost",
                ),
            ),
            port: Some(
                8080,
            ),
            path: "/v2/",
            query: None,
            fragment: None,
        },
        source: hyper::Error(
            Connect,
            Ssl(
                Error {
                    code: ErrorCode(
                        1,
                    ),
                    cause: Some(
                        Ssl(
                            ErrorStack(
                                [
                                    Error {
                                        code: 167772427,
                                        library: "SSL routines",
                                        function: "ssl3_get_record",
                                        reason: "wrong version number",
                                        file: "ssl/record/ssl3_record.c",
                                        line: 354,
                                    },
                                ],
                            ),
                        ),
                    ),
                },
                X509VerifyResult {
                    code: 0,
                    error: "ok",
                },
            ),
        ),
    },
)

Before raising this enhancement request

Have you looked at the limitations document?

Yes, I did not find a relevant limitation.

[burgerdev]

Looking at the configuration options of oci-distribution, this use case seems expected - we could do

diff --git a/src/tools/genpolicy/src/registry.rs b/src/tools/genpolicy/src/registry.rs
index 6009dad6e..2ac31b7e9 100644
--- a/src/tools/genpolicy/src/registry.rs
+++ b/src/tools/genpolicy/src/registry.rs
@@ -13,7 +13,7 @@ use anyhow::{anyhow, bail, Result};
 use docker_credential::{CredentialRetrievalError, DockerCredential};
 use log::warn;
 use log::{debug, info, LevelFilter};
-use oci_distribution::client::{linux_amd64_resolver, ClientConfig};
+use oci_distribution::client::{linux_amd64_resolver, ClientConfig, ClientProtocol};
 use oci_distribution::{manifest, secrets::RegistryAuth, Client, Reference};
 use serde::{Deserialize, Serialize};
 use sha2::{digest::typenum::Unsigned, digest::OutputSizeUser, Sha256};
@@ -62,13 +62,14 @@ pub struct ImageLayer {
 }
 
 impl Container {
-    pub async fn new(use_cached_files: bool, image: &str) -> Result<Self> {
+    pub async fn new(use_cached_files: bool, insecure_registries: Vec<String>, image: &str) -> Result<Self> {
         info!("============================================");
         info!("Pulling manifest and config for {:?}", image);
         let reference: Reference = image.to_string().parse().unwrap();
         let auth = build_auth(&reference);
 
         let mut client = Client::new(ClientConfig {
+            protocol: ClientProtocol::HttpsExcept(insecure_registries),
             platform_resolver: Some(Box::new(linux_amd64_resolver)),
             ..Default::default()
         });

However, this additional argument would need to be added to quite a few intermediate function calls. Maybe we could pass a config struct with both use_cached_files and insecure_registries?

[burgerdev]

I found another source of problems when using genpolicy in hermetic environments: the pause container needs to be configurable, too.

https://github.com/kata-containers/kata-containers/blob/0174568/src/tools/genpolicy/src/pod.rs#L832-L833

[danmihai1]

I found another source of problems when using genpolicy in hermetic environments: the pause container needs to be configurable, too.

https://github.com/kata-containers/kata-containers/blob/0174568/src/tools/genpolicy/src/pod.rs#L832-L833

I agree. We're in the process of adding CI test coverage for genpolicy in the main branch - e.g., see #8922. As we work through these tests, we uncover and address these kinds of dark corners.