b1_02_policy_definition___sources_

Chapter 2: Policy Definition & Sources

In the previous chapter, we learned how the Application Snapshot tells ec-cli which container images need to be checked. That's like having a list of ingredients for our software recipe.

But just knowing the ingredients isn't enough! We need the actual recipe instructions – the rules that say what makes an ingredient "good" or "bad". For example, a rule might say "the flour must be organic" or "the eggs must not be expired".

This is where Policy Definition & Sources come in. They define the rulebook that ec-cli uses to validate the container images.

What is a Policy Definition?

Think of the Policy Definition as the master configuration for your validation checks. It's like the cover page and table of contents for your rulebook. It tells ec-cli things like:

What rules to apply: Where can ec-cli find the actual rule files?
What data the rules need: Sometimes rules need extra information (like a list of allowed licenses). Where is this data located?
How to verify signatures: How can we trust that the rules themselves haven't been tampered with? (We'll cover this more in Chapter 4: Sigstore Verification).

The main goal of the Policy Definition is to specify how the validation should be performed. It points ec-cli to the necessary components – the actual rule code and any supporting data.

Sources: Where Do the Rules and Data Come From?

The Policy Definition itself doesn't usually contain the entire set of rules directly. Instead, it points to Sources where the rules and data can be found. This makes policies reusable and easier to manage.

Common sources include:

Git Repositories: Rules and data files stored in a Git repository (like GitHub or GitLab).
OCI Artifacts: Rules packaged and stored as container images (or other artifacts) in an OCI registry (like Quay.io or Docker Hub).
Kubernetes Resources: The policy definition itself might be stored as a resource within your Kubernetes cluster.

Imagine your company has a standard set of security rules stored in a central Git repository. Your Policy Definition would simply point to that repository URL. ec-cli then knows to fetch the rules from there.

The rules themselves are typically written in a language called Rego. You don't need to be a Rego expert to use ec-cli, but it's helpful to know that's the language under the hood defining the logic (like "check for critical vulnerabilities" or "ensure the base image is approved").

Data files are often in simple formats like YAML or JSON.

How to Use Policies with `ec-cli`

Remember our ec validate image command from Chapter 1? We used the --images flag for the Application Snapshot. To specify the rulebook, we use the --policy flag.

ec validate image --images snapshot.yaml --policy my-policy-config.yaml

Here, my-policy-config.yaml is a file containing our Policy Definition. Let's look at a simple example of what this file might contain:

# my-policy-config.yaml
description: "Basic policy checking for critical vulns and base images"

# How to verify the image signatures (More in Chapter 4!)
publicKey: "k8s://openshift-pipelines/public-key"
rekorUrl: "https://rekor.sigstore.dev"

# List of locations for rules (Rego) and data (YAML/JSON)
sources:
  - name: "Standard Company Rules"
    # ---- Where the actual rule files (.rego) are located ----
    policy:
      # Fetch rules from the 'policy/release' directory in this Git repo
      - github.com/enterprise-contract/ec-policies//policy/release?ref=main
      # You can also use OCI artifacts
      # - oci::quay.io/my-company/my-rules:latest

    # ---- Where supporting data files are located ----
    data:
      # Fetch data from the 'data' directory in this Git repo
      - github.com/enterprise-contract/ec-policies//data?ref=main

Explanation:

description: A human-friendly description.
publicKey, rekorUrl: Parameters used for Sigstore Verification to ensure the image signatures are valid.
sources: A list of groups of rule/data locations.
policy: An array of URLs pointing to the directories or artifacts containing the actual Rego rule files. ec-cli will fetch these. The // and ?ref=main syntax is specific to go-getter (the tool ec-cli uses) for specifying subdirectories and branches/tags in Git.
data: An array of URLs pointing to directories or artifacts containing data files (often YAML or JSON) that the Rego rules might need.

When you run the command:

ec-cli reads snapshot.yaml to know which images to check (from Chapter 1).
ec-cli reads my-policy-config.yaml (specified by --policy).
It parses the policy config and sees the sources.
It fetches the Rego files from github.com/enterprise-contract/ec-policies//policy/release.
It fetches the data files from github.com/enterprise-contract/ec-policies//data.
It then uses these downloaded rules and data to validate the images listed in the snapshot. (We'll see how in the next chapter).

You can also provide the policy configuration in other ways:

Inline JSON/YAML:

ec validate image --images snapshot.yaml --policy '{"publicKey": "...", "sources": [{"policy": ["github..."], "data": ["github..."]}]}'

Kubernetes Resource: If your policy is defined as an EnterpriseContractPolicy custom resource in Kubernetes:
```
ec validate image --images snapshot.yaml --policy k8s://my-namespace/my-policy-name
```

Git URL directly:

ec validate image --images snapshot.yaml --policy github.com/my-org/my-policies//config.yaml?ref=v1.0

Under the Hood: Loading and Fetching Policies

How does ec-cli handle these different policy inputs and get the rules ready? Let's trace the steps:

sequenceDiagram
    participant User
    participant ec_cli as ec validate image
    participant PolicyLoader as Policy Loader
    participant SourceFetcher as Source Fetcher (go-getter)
    participant Validator as Validation Engine

    User->>ec_cli: Runs `ec validate image --policy <policy_ref> ...`
    Note over ec_cli: Receives policy reference (file, k8s, git, inline)
    ec_cli->>PolicyLoader: Load policy config from `<policy_ref>`
    PolicyLoader->>PolicyLoader: Detects type (file, k8s, git, inline)
    PolicyLoader->>PolicyLoader: Fetches/Reads the policy YAML/JSON content
    PolicyLoader->>PolicyLoader: Parses the content into an internal structure
    PolicyLoader-->>ec_cli: Returns Policy Config Object (contains source URLs)
    Note over ec_cli: Now knows the list of source URLs
    ec_cli->>SourceFetcher: Fetch rules from 'source_url_1' (e.g., Git repo)
    SourceFetcher->>SourceFetcher: Clones Git repo / Pulls OCI artifact
    SourceFetcher-->>ec_cli: Returns local path to downloaded rules
    ec_cli->>SourceFetcher: Fetch data from 'source_url_2' (e.g., Git repo)
    SourceFetcher->>SourceFetcher: Clones Git repo / Pulls OCI artifact
    SourceFetcher-->>ec_cli: Returns local path to downloaded data
    Note over ec_cli: Rules and data are now locally available
    ec_cli->>Validator: Use fetched rules/data at local paths to validate images
    Validator-->>ec_cli: Validation results
    ec_cli-->>User: Shows final report

Key Code Areas:

Parsing the Policy Reference (internal/policy/policy.go)

The ec-cli command first needs to understand what the --policy argument actually is. Is it a file path? Inline JSON? A Kubernetes reference? Code like loadPolicy handles this detection and initial loading.

// Simplified view from internal/policy/policy.go - loadPolicy method
func (p *policy) loadPolicy(ctx context.Context, policyRef string) error {
    // ... preliminary checks ...

    // Is it inline JSON/YAML (heuristically checking for ':')?
    if strings.Contains(policyRef, "{") || strings.Contains(policyRef, ": ") {
        log.Debug("Attempting to parse policyRef as inline YAML/JSON spec")
        // Try unmarshalling directly into the policy structure
        // 'yaml.Unmarshal' handles both YAML and JSON
        if err := yaml.Unmarshal([]byte(policyRef), &p.EnterpriseContractPolicySpec); err != nil {
            // Handle error... could be a file path or K8s ref instead
            log.Debugf("Not inline YAML/JSON: %v", err)
            // Fall through to other checks (file, k8s, git - logic simplified here)
        } else {
             // Successfully parsed as inline!
             log.Debug("Parsed policyRef as inline spec.")
             // Could also validate against schema here (omitted for brevity)
             return nil // Policy spec loaded
        }
    }

    // If not inline, check if it's a Kubernetes reference ("namespace/name")
    // Or check if it's a file path or Git URL etc. (details omitted)
    // Simplified K8s example:
    if isK8sRef(policyRef) { // Fictional helper function
         log.Debug("Read EnterpriseContractPolicy as k8s resource")
         k8s, _ := kubernetes.NewClient(ctx) // Simplified error handling
         ecp, err := k8s.FetchEnterpriseContractPolicy(ctx, policyRef)
         if err != nil { /* Handle error */ return fmt.Errorf("unable to fetch Policy: %w", err) }
         p.EnterpriseContractPolicySpec = ecp.Spec // Assign the fetched spec
         return nil // Policy spec loaded
    }

    // ... logic for handling file paths, git URLs for the policy config itself ...

    log.Errorf("Unable to determine policy type for: %s", policyRef)
    return errors.New("invalid policy reference")
}

This code snippet shows the basic idea: try to parse the input string in different ways (inline, K8s reference, file path, etc.) until the policy specification (EnterpriseContractPolicySpec) is successfully loaded.

Fetching Source Content (internal/policy/source/source.go)

Once the policy configuration is loaded, ec-cli knows the URLs for the actual rules (policy sources) and data sources. The PolicySource interface and its implementations (like PolicyUrl) use a library (go-gather, which uses go-getter) to download content from these URLs.

// Simplified concept from internal/policy/source/source.go - GetPolicy method on PolicyUrl
func (p *PolicyUrl) GetPolicy(ctx context.Context, workDir string, showMsg bool) (string, error) {
    // Determine a unique local directory based on the URL
    dest := uniqueDestination(workDir, string(p.Kind), p.Url) // e.g., /tmp/ecwork123/policy/abc123xyz

    // Check internal cache first (simplified)
    if cachedPath, found := checkDownloadCache(p.Url); found {
         log.Debugf("Cache hit for %s: using %s", p.Url, cachedPath)
         // Ensure the destination path exists, maybe symlink from cache
         // ... caching logic ...
         return cachedPath, nil
    }

    log.Debugf("Cache miss for %s. Downloading to %s", p.Url, dest)

    // Use the downloader (go-getter wrapper) to fetch the content
    // This handles Git, OCI, HTTP etc. based on the URL format
    metadata, err := downloader.Download(ctx, dest, p.Url, showMsg) // Simplified call
    if err != nil {
        log.Errorf("Failed to download from %s: %v", p.Url, err)
        return "", err
    }

    // Store the downloaded path in cache
    storeInDownloadCache(p.Url, dest)

    // Update the URL to a pinned version if possible (e.g., Git SHA, OCI digest)
    // Helps with reproducibility
    p.Url, _ = metadata.GetPinnedURL(p.Url)
    log.Debugf("Using pinned URL: %s", p.Url)

    log.Debugf("Successfully downloaded %s to %s", p.Url, dest)
    return dest, nil // Return the local path where files were downloaded
}

This GetPolicy method (called for each URL in the policy and data arrays of the sources) is responsible for actually getting the files onto the local machine where ec-cli is running, using caching to avoid re-downloading if possible. The returned dest path points to a directory containing the fetched Rego or data files.

These fetched files are then passed to the Validation Engine (Evaluator) which executes the rules.

You can also interact with policies directly using commands like:

ec fetch policy: Downloads policy and data sources to a specified directory. (See cmd/fetch/fetch_policy.go)
ec inspect policy: Shows information about the rules within a policy source. (See cmd/inspect/inspect_policy.go)
ec validate policy: Checks if a policy configuration file itself conforms to the required structure. (See cmd/validate/policy.go)

Conclusion

We've now uncovered the "rulebook" for ec-cli – the Policy Definition tells ec-cli how to validate, primarily by pointing to Sources where the actual Rego rules and data files live (like Git repos or OCI artifacts). You learned how to specify this rulebook using the --policy flag and saw how ec-cli parses this configuration and fetches the necessary files.

We know what to check (Application Snapshot) and we have the rules (Policy Definition & Sources). The next step is to see how ec-cli actually runs these rules against the images.

Let's dive into the engine room in the next chapter: Validation Engine (Evaluator).

Generated by AI Codebase Knowledge Builder

Navigation Menu

Repository navigation

b1_02_policy_definition___sources_

Chapter 2: Policy Definition & Sources

What is a Policy Definition?

Sources: Where Do the Rules and Data Come From?

How to Use Policies with `ec-cli`

Under the Hood: Loading and Fetching Policies

Conclusion

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

b1_02_policy_definition___sources_

Chapter 2: Policy Definition & Sources

What is a Policy Definition?

Sources: Where Do the Rules and Data Come From?

How to Use Policies with ec-cli

Under the Hood: Loading and Fetching Policies

Conclusion

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

How to Use Policies with `ec-cli`