revert: Remove remote Git mirror changes by pda · Pull Request #4173 · buildkite/agent

pda · 2026-08-04T09:04:12Z

Description

Reverts the remote Git mirror rollout because local mirror reachability checking introduced a significant performance regression. This restores the checkout behavior from before the rollout while the regression is investigated separately.

The revert commits are based directly on 75683492d8057fe3eb0d4ac78ceaed2b1e1930a6, before the current tip of main.

Context

Reverts:

Changes

Reverts the five merge commits in reverse merge order.

Testing

Tests have run locally (with go test ./...). Buildkite employees may check this if the pipeline has run automatically.
Code is formatted (with go tool gofumpt -extra -w .)

The affected package tests (agent, internal/job, clicommand, env, and internal/shell) pass after the rebase, and the agent builds successfully. The full test suite is deferred to keep the urgent revert moving.

Affiliation (optional, external contributors)

Buildkite.

Disclosures / Credits

Cursor generated and applied the revert commits, including the rebase conflict resolution.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bf01df6d9a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-04T09:09:21Z

+			} else if strings.HasPrefix(envar, "BUILDKITE") || strings.HasPrefix(envar, "CI") || strings.HasPrefix(envar, "PATH") {
+				e.shell.Printf("%s", strings.ReplaceAll(envar, "\n", "\\n"))


Preserve mirror-URL credential redaction in debug logs

When a job still contains BUILDKITE_GIT_REMOTE_MIRROR_URL—for example during a staged control-plane rollback or when a pipeline sets it—createEnvironment copies it through unchanged, and this generic debug branch now writes the complete credentialed URL to the uploaded job log. Removing mirror checkout support does not make the environment value safe to print; retain the URL-credential redaction independently of the reverted feature.

Useful? React with 👍 / 👎.

BUILDKITE_GIT_REMOTE_MIRROR_URL never contain credentials.

chatgpt-codex-connector · 2026-08-04T09:09:21Z

+	helper := fmt.Sprintf(`%s git-credentials-helper`, self.Path(ctx))
 	err = e.shell.Command("git", "config", "--global", "credential.helper", helper).Run(ctx, shell.ShowPrompt(false))


Keep quoting the credential-helper executable path

When the agent executable is installed under a path containing spaces or shell metacharacters and an HTTPS repository requires Buildkite-provided credentials, this unquoted helper string is split by Git's shell invocation—for example, /opt/Buildkite Agent/buildkite-agent is attempted as /opt/Buildkite. The credential helper then cannot run and checkout fails, so the quoted leading-! helper construction should remain even though remote-mirror support is being reverted.

Useful? React with 👍 / 👎.

I believe this was pre-existing (this is a revert PR) so not something we'll change in this PR.

pda · 2026-08-04T09:14:04Z

I've merged this branch into 3.136-stable which is based on the v3.136.0 tag.

buildsworth-bk-app

The revert patches line up with the five merge commits, and the control-plane OTLP changes remain intact. I have one question about rollout state persisted by v3.136.0.

Want to dig deeper?

Paste this into your agent to explore the findings from this review's Buildkite build:

Download the buildsworth logs from build 11658, then answer my questions about the findings.

Install the reading-buildsworth-logs skill to run this.

About buildsworth

Model: gpt-5.6-sol with xhigh thinking.

How to request a review: Comment @buildsworth-bk review on the PR, or request buildsworth-bk as a reviewer.

Risk labels (how buildsworth classifies risk) — buildsworth classifies risk itself from the diff. To let it approve, grant L2 approval by mentioning @buildsworth-bk (see L2 approval grant):

L1 — Low risk (dep bumps, docs/copy, lockfiles, small presentational fixes). buildsworth may approve by default.
L2 — Standard risk (new UI, additive API fields, refactors). Approved only with an L2 grant; otherwise comment-only.
L3 — High risk (auth, migrations, payments, secrets, perf-critical paths). Human review always required.

buildsworth-bk-app · 2026-08-04T09:16:17Z

 // flags — the caller decides based on sparse-checkout state and user-supplied
 // filters.
-func (e *Executor) fetchSource(ctx context.Context, addBloblessFilter bool, attempt *remoteMirrorAttempt) (retErr error) {
+func (e *Executor) fetchSource(ctx context.Context, addBloblessFilter bool) (retErr error) {


Question: Did any v3.136.0 agents run mirror-enabled checkouts before this rollback? If a process died after starting a filtered mirror fetch, it could leave the buildkite.remote-mirror-promisor marker and a URL-named promisor section behind; #4162 intentionally repaired that state on the next fetch even after the mirror URL disappeared. Reverting this call and its helpers means later lazy object reads can still contact the removed mirror without the scoped credentials or timeout. Was the rollout constrained such that this on-disk state cannot exist?

Still open — could you confirm whether any mirror-enabled v3.136.0 checkout could have left this marker/promisor state behind?

Still open — the 3.136-stable note does not confirm whether an earlier v3.136.0 checkout could have persisted this state.

No, it's fine.

…url-fast-path-c275" This reverts commit 7568349, reversing changes made to 5286a2e.

…-fresh-clone-c275" This reverts commit c4e2d78, reversing changes made to a104ff3.

…-reused-checkout-c275" This reverts commit a233fd8, reversing changes made to 01c20f6.

…-on-host-c275" This reverts commit 01c20f6, reversing changes made to 20df5d1.

…-foundations-c275" This reverts commit 979ecc4, reversing changes made to 3ee0b89. Co-authored-by: Paul Annesley <pda@users.noreply.github.com>

cursor · 2026-08-04T09:27:42Z

Rebased the revert branch onto 75683492d8057fe3eb0d4ac78ceaed2b1e1930a6 as requested and force-pushed with lease. The branch no longer contains 11e51bcf53909e6917372bef65270b70bcf086e6; its five revert commits now descend directly from 75683492.

buildsworth-bk-app

The rollout-state question remains open, and this revision also introduces a merge conflict with main in agent/job_runner.go.

Want to dig deeper?

Paste this into your agent to explore the findings from this review's Buildkite build:

Download the buildsworth logs from build 11664, then answer my questions about the findings.

Install the reading-buildsworth-logs skill to run this.

buildsworth-bk-app · 2026-08-04T09:35:59Z

-		}
-	}
-
 	// When in KubernetesExec mode, filter out the Kubernetes plugin,


Blocking: This branch does not merge into current main: both sides edit this point in createEnvironment, and Git reports a content conflict in agent/job_runner.go. Please rebase onto main and resolve this hunk while retaining the control-plane OTLP destination block added by #4165; otherwise this PR cannot be merged as-is.

pda · 2026-08-04T09:49:52Z

This is already merged into a stable branch and being released as https://github.com/buildkite/agent/releases/tag/v3.136.1

I'm thinking I'll close it here, and fix forwards on main prior to the next release instead.

pda requested review from a team as code owners August 4, 2026 09:04

chatgpt-codex-connector Bot reviewed Aug 4, 2026

View reviewed changes

pda added the bug label Aug 4, 2026

buildsworth-bk-app Bot reviewed Aug 4, 2026

View reviewed changes

pda mentioned this pull request Aug 4, 2026

VERSION = v3.136.1 #4174

Closed

cursoragent and others added 5 commits August 4, 2026 09:27

Revert "Merge pull request #4171 from buildkite/pda/remote-mirror-no-…

850f1f0

…url-fast-path-c275" This reverts commit 7568349, reversing changes made to 5286a2e.

Revert "Merge pull request #4163 from buildkite/pda/remote-git-mirror…

feba61f

…-fresh-clone-c275" This reverts commit c4e2d78, reversing changes made to a104ff3.

Revert "Merge pull request #4162 from buildkite/pda/remote-git-mirror…

c14ea6c

…-reused-checkout-c275" This reverts commit a233fd8, reversing changes made to 01c20f6.

Revert "Merge pull request #4161 from buildkite/pda/remote-git-mirror…

d0bce62

…-on-host-c275" This reverts commit 01c20f6, reversing changes made to 20df5d1.

Revert "Merge pull request #4160 from buildkite/pda/remote-git-mirror…

b0b6e5a

…-foundations-c275" This reverts commit 979ecc4, reversing changes made to 3ee0b89. Co-authored-by: Paul Annesley <pda@users.noreply.github.com>

cursor Bot force-pushed the pda/revert-remote-mirror-regression-8cbd branch from bf01df6 to b0b6e5a Compare August 4, 2026 09:27

buildsworth-bk-app Bot requested changes Aug 4, 2026

View reviewed changes

pda closed this Aug 4, 2026

		} else if strings.HasPrefix(envar, "BUILDKITE") \|\| strings.HasPrefix(envar, "CI") \|\| strings.HasPrefix(envar, "PATH") {
		e.shell.Printf("%s", strings.ReplaceAll(envar, "\n", "\\n"))

		helper := fmt.Sprintf(`%s git-credentials-helper`, self.Path(ctx))
		err = e.shell.Command("git", "config", "--global", "credential.helper", helper).Run(ctx, shell.ShowPrompt(false))

Uh oh!

Conversation

pda commented Aug 4, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Context

Changes

Testing

Affiliation (optional, external contributors)

Disclosures / Credits

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

pda Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

pda Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

pda commented Aug 4, 2026

Uh oh!

buildsworth-bk-app Bot left a comment

Choose a reason for hiding this comment

Uh oh!

buildsworth-bk-app Bot Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

buildsworth-bk-app Bot Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

buildsworth-bk-app Bot Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

pda Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

cursor Bot commented Aug 4, 2026

Uh oh!

buildsworth-bk-app Bot left a comment

Choose a reason for hiding this comment

Uh oh!

buildsworth-bk-app Bot Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

pda commented Aug 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pda commented Aug 4, 2026 •

edited by cursor Bot

Loading