b2_07_bundle_tracking_

Chapter 7: Bundle Tracking

In Chapter 6: Tekton Task Integration, we saw how to plug ec-cli checks into our automated Tekton pipelines using pre-built Tasks. These tasks often rely on reusable components like standard Tekton Tasks provided as OCI images, known as "Tekton Bundles".

But these reusable bundles get updated over time! Imagine your organization maintains a standard "build-image" Tekton Task bundle. A new version is released with security fixes. How do you keep track of which version is the currently approved, "trusted" one? How do you manage the transition, giving teams time to update their pipelines before the old version is disallowed?

This is where Bundle Tracking helps. It's like keeping an official logbook for your reusable Tekton Task bundles.

What Problem Does Bundle Tracking Solve?

When you rely on shared, reusable components like Tekton Task bundles, managing their updates across different teams and pipelines can be tricky:

• Which version is currently approved? How does everyone know the latest "blessed" version?
• When does a new version become mandatory? How do you announce a new version and set a deadline for adoption?
• How do policies enforce the use of approved versions? Your security policies might need to check if a pipeline uses a bundle version listed in the "approved logbook".

Manually managing lists of approved bundle versions and their effective dates is error-prone. We need a systematic way to record this information.

Our Goal: To understand how ec-cli uses the track bundle command to maintain a record of trusted Tekton bundle versions, including when they become effective or expire, making it easier to manage updates securely.

Key Concepts: The Bundle Logbook

1. Tekton Bundle: A reusable Tekton Task (or multiple Tasks) packaged as an OCI container image, stored in a registry. Example: quay.io/my-org/build-tasks:v2.1.
2. Tracking Data (The Logbook): A file (usually YAML) or an OCI artifact that stores the list of trusted bundle versions and their history. ec-cli reads from and writes to this logbook.
3. Trusted Entry (A Logbook Entry): A record in the tracking data for a specific bundle version, identified by its unique digest (sha256:...).
4. Effective On (The "Start Using By" Date): A date associated with a trusted entry. It indicates when this version is expected to be in use. By default, ec-cli sets this 30 days in the future, giving teams time to adopt the new version. Policies can use this date to warn or fail if an older version is still used after its successor becomes effective.
5. Expires On (When Superseded): A date (often implicitly calculated) when a version is considered outdated because a newer trusted version has become effective.
6. Pruning (Cleaning the Logbook): An automatic process (ec track bundle does this by default) that removes old, superseded entries from the tracking data, keeping it tidy and relevant. It typically keeps entries that are effective in the future, plus the single most recent entry that is already effective.
7. Freshening (Checking for Tag Updates): An option (--freshen) to make ec track bundle check if any existing bundle tags (like :latest or :v2) in the tracking data now point to a newer digest in the registry. If so, it adds a new entry for that latest digest.

How to Use Bundle Tracking (ec track bundle)

The main tool for managing the logbook is the ec track bundle command.

Scenario: Let's say we maintain a Tekton Task bundle quay.io/my-org/cool-task:1.0 and we want to start tracking it.

1. Creating the Initial Tracking Data:

We run the command, specifying the bundle we want to track:

ec track bundle --bundle quay.io/my-org/cool-task:1.0 \
                --output cool-task-tracking.yaml

• --bundle quay.io/my-org/cool-task:1.0: Tells ec which bundle image reference to add. ec will automatically find its digest (sha256:...).
• --output cool-task-tracking.yaml: Specifies where to save the tracking data (a new YAML file). If omitted, it prints to the console. You can also use oci: prefix like --output oci:quay.io/my-org/cool-task-tracker:latest to save it as an OCI artifact.

The cool-task-tracking.yaml file might look like this (assuming today is 2024-08-15 and the digest is sha256:abc...):

# cool-task-tracking.yaml
trusted_tasks:
  oci://quay.io/my-org/cool-task:1.0: # Grouped by image ref (tag matters here)
    - effective_on: "2024-09-14T00:00:00Z" # 30 days from now (default)
      ref: sha256:abc... # The specific digest tracked

This entry says that version sha256:abc... (which corresponds to tag 1.0) should be considered effective starting September 14th, 2024.

2. Adding a New Version:

Now, we release quay.io/my-org/cool-task:1.1 (with digest sha256:def...). Let's update our tracking file.

ec track bundle --bundle quay.io/my-org/cool-task:1.1 \
                --input cool-task-tracking.yaml \
                --output cool-task-tracking.yaml

• --input cool-task-tracking.yaml: Reads the existing tracking data.
• --bundle quay.io/my-org/cool-task:1.1: The new bundle version to add.
• --output cool-task-tracking.yaml: Writes the updated data back to the same file. (You could use --replace instead of repeating --input and --output if they are the same file path).

The updated cool-task-tracking.yaml might look like this (assuming we run this on 2024-09-20):

# cool-task-tracking.yaml
trusted_tasks:
  oci://quay.io/my-org/cool-task:1.0: # Old entry still here
    - effective_on: "2024-09-14T00:00:00Z"
      ref: sha256:abc...
      # Implicitly expires when 1.1 becomes effective
  oci://quay.io/my-org/cool-task:1.1: # New entry added
    - effective_on: "2024-10-20T00:00:00Z" # 30 days from 2024-09-20
      ref: sha256:def...

Now we have both versions tracked. Notice the new entry for 1.1 with its own future effective_on date.

3. Pruning Old Entries:

Let's fast forward to November 1st, 2024. We add version 1.2 (digest sha256:ghi...). Pruning is enabled by default (--prune=true).

ec track bundle --bundle quay.io/my-org/cool-task:1.2 \
                --input cool-task-tracking.yaml \
                --replace

• --replace: Updates the input file (cool-task-tracking.yaml) directly.

The file now looks like this:

# cool-task-tracking.yaml
trusted_tasks:
  # Entry for 1.0 was pruned!
  oci://quay.io/my-org/cool-task:1.1: # Still here, it's the latest *effective* one
    - effective_on: "2024-10-20T00:00:00Z"
      ref: sha256:def...
      expires_on: "2024-12-01T00:00:00Z" # Automatically calculated (effective_on of 1.2)
  oci://quay.io/my-org/cool-task:1.2: # The new one, effective in the future
    - effective_on: "2024-12-01T00:00:00Z" # 30 days from 2024-11-01
      ref: sha256:ghi...

The entry for 1.0 was automatically removed (pruned) because version 1.1 is already effective (2024-10-20 is in the past), and we only keep the most recent effective entry plus any future ones. The expires_on field is automatically added to show when 1.1 is superseded by 1.2.

4. Tracking Git Sources:

You can also track Tasks defined directly in Git repositories:

ec track bundle --git git+https://github.com/my-org/my-tasks.git//path/to/task.yaml@commitSha123 \
                --output tasks-tracking.yaml

• --git: Specifies a Git URL, path within the repo, and commit SHA.

5. Freshening Tags:

If you track a floating tag like :latest, the digest it points to can change. Using --freshen tells ec to check the current digest for the tag and add a new entry if it's different from the last tracked digest for that tag.

# Assuming cool-task-tracking.yaml tracks quay.io/my-org/cool-task:latest
# And the :latest tag now points to a new digest sha256:jkl...
ec track bundle --input cool-task-tracking.yaml --freshen --replace

This would add a new entry for oci://quay.io/my-org/cool-task:latest with ref: sha256:jkl... and a new effective_on date.

This tracking data can then be used by policies (often loaded as policy data) to check if a Tekton TaskRun is using a bundle version that is currently considered "trusted" according to the logbook.

Under the Hood: How Tracking Works

1. Parse Flags: ec track bundle reads the command-line arguments (--bundle, --git, --input, --output, --prune, --freshen, etc.).
2. Load Existing Data: If --input is provided, it reads the YAML content from the file or pulls the OCI artifact layers and parses the YAML data into an internal Tracker structure.
3. Resolve References: For each new --bundle or --git reference provided:
   • It uses OCI Interaction client libraries to resolve image tags to their current digests (sha256:...).
   • It uses Git client libraries to resolve Git references to commit SHAs (especially if no SHA is provided and --freshen is used).
4. Check Bundle Type (OCI Only): For OCI bundles, it may check if the artifact actually contains Tekton Task definitions.
5. Add New Entries: It calculates the effective_on date (default: today + 30 days) and adds new taskRecord entries to the Tracker data structure for the newly resolved digests/commits.
6. Freshen Existing Entries (if --freshen): It iterates through existing tracked tagged OCI references, resolves their current digests, and adds new entries if the digest has changed.
7. Prune Entries (if --prune): It applies the pruning logic: keeps all future effective entries, plus the single most recent currently effective entry for each tracked bundle reference (e.g., oci://quay.io/my-org/cool-task:1.1). Older effective entries are removed.
8. Calculate Expirations: It iterates through the remaining, sorted entries for each bundle reference and sets the expires_on field on each entry to the effective_on date of the next entry. The latest entry has no expires_on.
9. Format Output: It marshals the final Tracker data structure back into YAML format.
10. Write Output: It writes the YAML data to the destination specified by --output (stdout, a file, or pushes it as a layer to an OCI artifact). If --replace is used, it overwrites the --input file/OCI artifact.

Simplified Sequence Diagram:

sequenceDiagram
    participant User
    participant CLI as ec track bundle
    participant TrackerLogic as Tracker Logic
    participant OCIClient as OCI/Git Client
    participant Registry as OCI Registry/Git Repo
    participant Output as Console/File/OCI

    User->>CLI: Run command with --bundle, --input, --output, etc.
    CLI->>TrackerLogic: Load data from --input (via OCIClient if needed)
    TrackerLogic-->>CLI: Existing Tracker data
    CLI->>OCIClient: Resolve new --bundle/--git refs
    OCIClient->>Registry: Query registry/repo for digests/commits
    Registry-->>OCIClient: Return digests/commits
    OCIClient-->>CLI: Resolved references
    CLI->>TrackerLogic: Add new entries, prune, set expirations
    TrackerLogic-->>CLI: Updated Tracker data
    CLI->>Output: Format data as YAML
    CLI->>Output: Write YAML to --output destination (via OCIClient if needed)
    Output-->>User: Show output / Confirm push

Loading

Diving into the Code (A Glimpse)

Let's look at the key code locations.

1. Command Definition (cmd/track/track_bundle.go)

This file sets up the ec track bundle command, defines its flags, and orchestrates the overall process.

// Simplified concept from cmd/track/track_bundle.go
func trackBundleCmd(track trackBundleFn, pullImage pullImageFn, pushImage pushImageFn) *cobra.Command {
	// Define flags like --bundle, --input, --output, --prune, --freshen
	params := struct {
		bundles      []string
		input        string
		prune        bool
		replace      bool
		output       string
		freshen      bool
		// ... other params ...
	}{}

	cmd := &cobra.Command{
		Use:   "bundle",
		Short: "Record tracking information about Tekton bundles",
		// ... Long description, Example ...
		RunE: func(cmd *cobra.Command, args []string) (err error) {
			// 1. Load input data (from file or OCI pullImage)
			var data []byte
			if strings.HasPrefix(params.input, "oci:") {
				data, err = pullImage(cmd.Context(), strings.TrimPrefix(params.input, "oci:"))
			} else if params.input != "" {
				data, err = afero.ReadFile(utils.FS(cmd.Context()), params.input)
			}
			// ... handle error ...

			// 2. Call the core tracking logic (injected function)
			// urls contains both --bundle and --git inputs
			urls := append(params.bundles, params.gits...)
			out, err := track(cmd.Context(), urls, data, params.prune, params.freshen, params.inEffectDays)
			// ... handle error ...

			// 3. Write the output (to stdout, file, or OCI pushImage)
			switch {
			case params.output == "": // stdout
				_, err = cmd.OutOrStdout().Write(out)
			case strings.HasPrefix(params.output, "oci:"): // OCI push
				err = pushImage(cmd.Context(), strings.TrimPrefix(params.output, "oci:"), out, /* invocation */)
			default: // file
				err = afero.WriteFile(utils.FS(cmd.Context()), params.output, out, 0666)
			}
			// ... handle error ...

			// 4. Handle --replace logic (write back to input)
			if params.replace && params.input != "" {
				// ... write 'out' back to params.input (file or OCI) ...
			}
			return
		},
	}

	// Register flags with the command
	cmd.Flags().StringVarP(&params.input, "input", "i", ...)
	cmd.Flags().StringSliceVarP(&params.bundles, "bundle", "b", ...)
	cmd.Flags().BoolVarP(&params.prune, "prune", "p", ...)
	// ... other flags ...

	return cmd
}

This shows the command structure: reading flags, loading input using helpers (pullImage), calling the core track function, and writing the result using helpers (pushImage).

2. Core Tracking Logic (internal/tracker/tracker.go)

This file contains the Tracker data structure and the main Track function that modifies it.

// Simplified concept from internal/tracker/tracker.go
package tracker

import (
	// ... imports ...
	"sigs.k8s.io/yaml" // For YAML handling
	"time"
	"github.com/enterprise-contract/ec-cli/internal/image" // For image resolution
)

// Represents one tracked version of a bundle
type taskRecord struct {
	Ref         string     `json:"ref"` // The digest or commit SHA
	EffectiveOn time.Time  `json:"effective_on"`
	ExpiresOn   *time.Time `json:"expires_on,omitempty"` // Pointer to omit if unset
	// Internal fields not saved to YAML:
	Tag        string `json:"-"`
	Repository string `json:"-"`
}

// The main structure holding all tracking data
type Tracker struct {
	TrustedTasks map[string][]taskRecord `json:"trusted_tasks,omitempty"`
}

// Loads existing data or creates a new Tracker
func newTracker(input []byte) (t Tracker, err error) {
	if input != nil {
		err = yaml.Unmarshal(input, &t) // Parse YAML
	} else {
		t = Tracker{} // Create empty if no input
	}
	// Ensure map is initialized
	if t.TrustedTasks == nil {
		t.TrustedTasks = map[string][]taskRecord{}
	}
	return
}

// Main function orchestrating the tracking update
func Track(ctx context.Context, urls []string, input []byte, prune bool, freshen bool, inEffectDays int) ([]byte, error) {
	// 1. Load or initialize tracker data
	t, err := newTracker(input)
	if err != nil { return nil, err }

	// Separate OCI bundle URLs from Git URLs
	imageUrls, gitUrls := groupUrls(urls)

	// Calculate the effective date for new entries
	days := time.Hour * 24 * time.Duration(inEffectDays)
	effectiveOn := time.Now().Add(days).UTC().Round(time.Hour * 24)

	// 2. Process OCI image references
	if err := t.trackImageReferences(ctx, imageUrls, freshen, effectiveOn); err != nil {
		return nil, err
	}

	// 3. Process Git references
	if err := t.trackGitReferences(ctx, gitUrls, freshen, effectiveOn); err != nil {
		return nil, err
	}

	// 4. Prune old records if requested
	t.filterBundles(prune) // Calls filterRecords internally

	// 5. Calculate expiration dates for remaining records
	t.setExpiration()

	// 6. Serialize the final tracker data to YAML
	return t.Output() // Calls yaml.Marshal and yamlfmt.Format
}

// Helper to add a new record
func (t *Tracker) addTrustedTaskRecord(prefix string, record taskRecord) {
	// Determine the group key (e.g., "oci://quay.io/my-org/cool-task:1.1")
	group := createGroupKey(prefix, record)
	// Append the new record to the list for that group
	t.TrustedTasks[group] = append([]taskRecord{record}, t.TrustedTasks[group]...)
}

// ... trackImageReferences, trackGitReferences call resolver helpers ...

// Applies pruning logic
func (t *Tracker) filterBundles(prune bool) {
	for group, records := range t.TrustedTasks {
		t.TrustedTasks[group] = filterRecords(records, prune) // Core pruning logic
	}
}

// Sets expiration based on next effective_on
func (t *Tracker) setExpiration() {
	for _, records := range t.TrustedTasks { // Loop through each tracked bundle
		var expiration *time.Time // Tracks the next effective_on date
		// Iterate backwards to easily find the next item's effective_on
		for i := len(records) - 1; i >= 0; i-- {
			if expiration != nil {
				records[i].ExpiresOn = expiration // Set expiry on the current record
			}
			expiration = &records[i].EffectiveOn // Update for the next iteration
		}
	}
}

// Serializes Tracker to YAML
func (t Tracker) Output() ([]byte, error) {
	out, err := yaml.Marshal(t)
	// ... handle error ...
	// Use yamlfmt to sort keys for deterministic output
	return yamlfmt.Format(bytes.NewBuffer(out), true)
}

This highlights the Tracker struct, the main Track function orchestrating the steps (loading, adding, pruning, expiring, outputting), and helpers like addTrustedTaskRecord, filterBundles, and setExpiration.

Conclusion

Bundle Tracking provides a crucial mechanism for managing the lifecycle of shared, reusable Tekton Task bundles. By using ec track bundle, you can:

1. Maintain a logbook (tracking file/image) of trusted bundle versions.
2. Record the specific digest (sha256:...) or commit SHA for each trusted version.
3. Define when a version becomes effective, allowing controlled rollouts.
4. Automatically prune old, superseded entries.
5. Keep track of updates to tags using --freshen.

This structured tracking data can then be consumed by policies to enforce that CI/CD pipelines are using approved and up-to-date components, enhancing security and stability.

Now that we've covered the core functionalities of ec-cli, how can we ensure the tool itself works correctly and continues to meet expectations as it evolves?

Let's move on to the final chapter, Chapter 8: Acceptance Testing Framework, to see how ec-cli uses testing to maintain its quality.

---

Generated by AI Codebase Knowledge Builder