prowgen: add disable_sparse_checkout option

[Prucek]

Summary

• Adds disable_sparse_checkout field to ProwgenOverrides in ci-operator config
• When set, prowgen generates jobs that perform a full clone instead of sparse checkout
• This is needed for repositories that use git submodules, as sparse checkout with blobless fetches (--filter=blob:none) fails when submodule gitlink entries are modified but the submodule path is excluded from the sparse working tree

Repos that need this can add to their ci-operator config:

prowgen:
  disable_sparse_checkout: true

This is intended as a workaround until kubernetes-sigs/prow#723 lands with a proper fallback in clonerefs.

Adds disable_sparse_checkout option for repository cloning

This PR adds a new ci-operator prowgen configuration option to opt out of the sparse-checkout optimization used during Prow job repository cloning.

Context

Prowgen previously generated jobs that use sparse checkout with blobless fetches (--filter=blob:none) to fetch only files required for CI (e.g., Dockerfiles, ci-operator configs). That optimization can fail for repositories that use git submodules when submodule gitlink entries are modified but the submodule path is excluded from the sparse working tree. This change is a workaround until clonerefs gains a proper fallback (kubernetes-sigs/prow#723).

Changes (user-visible)

• New config field: prowgen: disable_sparse_checkout: true — added as disable_sparse_checkout on ProwgenOverrides (pkg/api/types.go). When set, prowgen will generate jobs that perform a full git clone instead of using sparse checkout / blobless fetches.
• Job generation: prowgen's job base builder now respects this flag and leaves SparseCheckoutFiles unset when disable_sparse_checkout is true, preserving existing SkipCloning and OAuth token behavior for private jobs (pkg/prowgen/jobbase.go).
• Tests and fixtures: Added unit tests and YAML fixtures exercising public and private image-build jobs with sparse checkout disabled (pkg/prowgen/jobbase_test.go and testdata fixtures).

Impact for CI users/operators

Repository owners can opt out of sparse-checkout on a per-repo basis by setting prowgen.disable_sparse_checkout: true in ci-operator configuration. This forces full repository checkouts for generated Prow jobs, avoiding failures for repos that rely on submodules or other cases where sparse checkout breaks, while other repos retain the performance benefits of sparse checkout.

[openshift-merge-bot]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: fe04a754-0202-4918-9c67-08b6e1cc72a6

📥 Commits

Reviewing files that changed from the base of the PR and between bc8dcc1 and 1f30fe7.

⛔ Files ignored due to path filters (1)

• pkg/webreg/zz_generated.ci_operator_reference.go is excluded by !**/zz_generated*

📒 Files selected for processing (5)

• pkg/api/types.go
• pkg/prowgen/jobbase.go
• pkg/prowgen/jobbase_test.go
• pkg/prowgen/testdata/zz_fixture_TestProwJobBaseBuilder_job_with_images_and_sparse_checkout_disabled.yaml
• pkg/prowgen/testdata/zz_fixture_TestProwJobBaseBuilder_private_job_with_images_and_sparse_checkout_disabled.yaml

🚧 Files skipped from review as they are similar to previous changes (3)

• pkg/prowgen/testdata/zz_fixture_TestProwJobBaseBuilder_private_job_with_images_and_sparse_checkout_disabled.yaml
• pkg/api/types.go
• pkg/prowgen/jobbase.go

---

📝 Walkthrough

Walkthrough

This PR adds a DisableSparseCheckout boolean to ProwgenOverrides, makes NewProwJobBaseBuilder skip setting DecorationConfig.SparseCheckoutFiles when that flag is true, and adds tests plus YAML fixtures for public and private image-build jobs with sparse-checkout disabled.

Changes

DisableSparseCheckout Feature

Layer / File(s)	Summary
Configuration field pkg/api/types.go	Adds DisableSparseCheckout boolean field to ProwgenOverrides with JSON tag disable_sparse_checkout.
Job builder sparse-checkout logic pkg/prowgen/jobbase.go	Reads disableSparseCheckout from config in NewProwJobBaseBuilder; when enabled, skips SparseCheckoutFiles while preserving OAuth token secret handling for private jobs.
Test cases and fixtures pkg/prowgen/jobbase_test.go, pkg/prowgen/testdata/zz_fixture_TestProwJobBaseBuilder_job_with_images_and_sparse_checkout_disabled.yaml, pkg/prowgen/testdata/zz_fixture_TestProwJobBaseBuilder_private_job_with_images_and_sparse_checkout_disabled.yaml	Adds two test cases for image builds with disabled sparse-checkout (public and private), each with YAML fixture defining Kubernetes job configuration including decoration, service account, and volume mounts.

---

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 17

✅ Passed checks (17 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: adding a disable_sparse_checkout option to prowgen as a new configuration feature.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Go Error Handling	✅ Passed	Proper nil checks before dereferencing pointers (configSpec.Prowgen != nil), no ignored errors, no panic calls, no unsafe dereferencing.
Test Coverage For New Features	✅ Passed	Two new table-driven test cases validate DisableSparseCheckout behavior with golden fixtures confirming sparse_checkout_files are not set when disabled. No new untested functions introduced.
Stable And Deterministic Test Names	✅ Passed	New test cases use static, descriptive names with no dynamic content, UUIDs, timestamps, or random identifiers.
Test Structure And Quality	✅ Passed	No Ginkgo tests in PR. Added tests use standard Go table-driven pattern with t.Run(), not Ginkgo's Describe/It blocks. Check is inapplicable.
Microshift Test Compatibility	✅ Passed	PR contains no new Ginkgo e2e tests, only Go unit tests in jobbase_test.go; custom check is not applicable.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	No new Ginkgo e2e tests added. PR only contains standard Go unit tests (testing.T) with table-driven test cases for the DisableSparseCheckout feature. Check is not applicable.
Topology-Aware Scheduling Compatibility	✅ Passed	PR modifies code generation tools (prowgen) and configuration types, not deployment manifests or operators. No scheduling constraints, affinity rules, or topology assumptions are introduced.
Ote Binary Stdout Contract	✅ Passed	PR only modifies prowgen package (Prow job generation library) and related types; prowgen is not an OTE binary and changes introduce no stdout contract violations.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	PR adds no Ginkgo e2e tests; only configuration types, job generation logic, and standard Go unit tests that don't require IPv6/disconnected network checks.
No-Weak-Crypto	✅ Passed	PR adds git sparse-checkout configuration option with no cryptographic operations, weak crypto algorithms, custom crypto implementations, or constant-time comparison issues.
Container-Privileges	✅ Passed	No privilege escalation patterns found: no privileged containers, hostPID/Network/IPC, SYS_ADMIN capabilities, or allowPrivilegeEscalation: true in YAML fixtures or code changes.
No-Sensitive-Data-In-Logs	✅ Passed	No logging statements or sensitive data exposure found. PR only adds a boolean configuration field and test cases using proper secret references (Kubernetes secret names only, not actual credentials).

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[droslean]

This needs to go in the else statement in 104-R109

[coderabbitai]

🧹 Nitpick comments (1)

pkg/prowgen/jobbase.go (1)

100-109: ⚡ Quick win

Eliminate code duplication for OAuth token assignment.

The OauthTokenSecret assignment appears in both the disableSparseCheckout branch (line 102) and the else branch (line 107). This duplication can be eliminated by restructuring the logic.

♻️ Proposed refactor to eliminate duplication

 	sparseFiles := sparseCheckoutFiles(configSpec)
 	shouldSkipCloning := len(sparseFiles) == 0
 	if shouldSkipCloning {
 		b.base.UtilityConfig.DecorationConfig.SkipCloning = ptr.To(true)
-	} else if disableSparseCheckout {
-		if private {
-			b.base.UtilityConfig.DecorationConfig.OauthTokenSecret = &prowv1.OauthTokenSecret{Key: cioperatorapi.OauthTokenSecretKey, Name: cioperatorapi.OauthTokenSecretName}
-		}
 	} else {
-		b.base.UtilityConfig.DecorationConfig.SparseCheckoutFiles = sparseFiles
+		if !disableSparseCheckout {
+			b.base.UtilityConfig.DecorationConfig.SparseCheckoutFiles = sparseFiles
+		}
 		if private {
 			b.base.UtilityConfig.DecorationConfig.OauthTokenSecret = &prowv1.OauthTokenSecret{Key: cioperatorapi.OauthTokenSecretKey, Name: cioperatorapi.OauthTokenSecretName}
 		}
 	}

As per coding guidelines, avoid code duplication and prefer clarity in control flow.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/prowgen/jobbase.go` around lines 100 - 109, The OauthTokenSecret
assignment is duplicated in both the disableSparseCheckout and else branches;
inside function handling disableSparseCheckout/private logic (references:
disableSparseCheckout, private, sparseFiles,
b.base.UtilityConfig.DecorationConfig.OauthTokenSecret), remove the repeated
assignment from each branch and instead set OauthTokenSecret once after the
conditional block when private is true (or set it conditionally before
returning), preserving existing behavior for SparseCheckoutFiles when
disableSparseCheckout is false; ensure only the branch-specific
SparseCheckoutFiles assignment remains inside the else branch and the single
OauthTokenSecret assignment is centralized.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@pkg/prowgen/jobbase.go`:
- Around line 100-109: The OauthTokenSecret assignment is duplicated in both the
disableSparseCheckout and else branches; inside function handling
disableSparseCheckout/private logic (references: disableSparseCheckout, private,
sparseFiles, b.base.UtilityConfig.DecorationConfig.OauthTokenSecret), remove the
repeated assignment from each branch and instead set OauthTokenSecret once after
the conditional block when private is true (or set it conditionally before
returning), preserving existing behavior for SparseCheckoutFiles when
disableSparseCheckout is false; ensure only the branch-specific
SparseCheckoutFiles assignment remains inside the else branch and the single
OauthTokenSecret assignment is centralized.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 711299f9-4364-43ff-ba7f-dc1ea73bcdc5

📥 Commits

Reviewing files that changed from the base of the PR and between 628dff0 and bc8dcc1.

📒 Files selected for processing (5)

• pkg/api/types.go
• pkg/prowgen/jobbase.go
• pkg/prowgen/jobbase_test.go
• pkg/prowgen/testdata/zz_fixture_TestProwJobBaseBuilder_job_with_images_and_sparse_checkout_disabled.yaml
• pkg/prowgen/testdata/zz_fixture_TestProwJobBaseBuilder_private_job_with_images_and_sparse_checkout_disabled.yaml

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: droslean, Prucek

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [Prucek,droslean]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Prucek]

/test e2e

[openshift-merge-bot]

Tests from second stage were triggered manually. Pipeline can be controlled only manually, until HEAD changes. Use command to trigger second stage.

[Prucek]

/override ci/prow/images

[openshift-ci]

@Prucek: Overrode contexts on behalf of Prucek: ci/prow/images

Details

In response to this:

/override ci/prow/images

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-ci]

@Prucek: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.