🤖 perf: replace git bundle with native git push for SSH runtime by ammar-agent · Pull Request #3137 · coder/mux

ammar-agent · 2026-04-06T15:30:11Z

Summary

Replaces the git-bundle-create → SSH-pipe → git-fetch-bundle workflow with a single git push over SSH. Git's native smart protocol negotiates which objects the remote already has, transferring only the delta. This makes subsequent workspace creations near-instant on slow connections instead of re-uploading the entire repository history every time.

Background

When the SSH runtime creates a new workspace, it syncs the local project to the remote. Previously, this always created a full git bundle (git bundle create - --branches --tags) containing the entire reachable history, piped it byte-by-byte over SSH to a temp file, then imported it with git fetch. Even if only one commit changed since the last sync, the full repo was re-uploaded.

The remote already has:

A bare git repo (.mux-base.git) that can receive pushes natively
Git installed (the code already runs git commands on it)
SSH connectivity (that's how we connect)

So the bundle approach was reimplementing what git push does natively with its pack negotiation protocol.

Implementation

The core change is straightforward — syncProjectToRemote() now runs:

git -C <project> push --force --atomic <host>:<base-repo-path> \
  +refs/heads/*:refs/mux-bundle/* \
  +refs/tags/*:refs/tags/*

with GIT_SSH_COMMAND set to mirror the runtime's SSH config (port, identity file, ControlPath multiplexing, host key policy). This preserves the existing refs/mux-bundle/* staging namespace and tag handling.

New: buildGitSshCommand() — constructs the SSH command string from the runtime's config, matching OpenSSHTransport's buildSSHArgs() settings.

Removed (163 lines):

transferBundleToRemote() — bundle creation + SSH pipe upload
pipeReadableToWebWritable() — streaming helper only used for bundles
waitForProcessExit() — only used by bundle upload
Temp file management and cleanup
spawn/ChildProcess imports

Preserved: shouldReuseCurrentBundleTrunk() — the skip-if-unchanged optimization that avoids sync entirely when all refs already match. This still works because git push populates the same refs/mux-bundle/* namespace.

Risks

Low. The refspec mapping is identical (+refs/heads/*:refs/mux-bundle/*, +refs/tags/*:refs/tags/*), so the remote base repo ends up in the same state. The shouldReuseCurrentBundleTrunk skip logic, worktree creation, and all downstream code are unchanged.

--atomic ensures that if any ref update fails, none are applied — preventing partial sync states. --force ensures we always sync to the local state even after history rewrites.

Generated with mux • Model: anthropic:claude-opus-4-6 • Thinking: xhigh • Cost: $6.97

ammar-agent · 2026-04-06T15:30:19Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a8ebe97490

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ammar-agent · 2026-04-06T15:38:37Z

@codex review

Addressed feedback: expanded the incremental fallback to cover the entire upload + import cycle, not just the upload step. Extracted transferAndImportBundle() as a self-contained helper that handles upload → import → cleanup for both incremental and full paths. Now if an incremental bundle uploads successfully but git fetch fails (e.g., due to dangling refs with missing prerequisite objects), the system correctly falls back to a full bundle.

chatgpt-codex-connector · 2026-04-06T15:43:22Z

Codex Review: Didn't find any major issues. Breezy!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ammar-agent · 2026-04-06T15:49:54Z

@codex review

Complete rewrite of the sync approach: replaced git bundle upload with native git push over SSH. The smart protocol handles incremental transfer natively — no bundles, temp files, or manual OID tracking needed. Net -70 lines.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 65cb90850d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ammar-agent · 2026-04-06T15:57:07Z

@codex review

Addressed both items:

Added --no-verify to skip pre-push hooks during internal sync
Shell-quoted dynamic SSH args (identityFile, ControlPath) using POSIX single-quote escaping

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f695b16094

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ammar-agent · 2026-04-06T16:05:30Z

@codex review

Fixed: strip ~/ prefix from the SCP-style push URL since git single-quotes the path (preventing tilde expansion). SCP-style URLs already treat non-absolute paths as relative to home.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 52dfea029a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ammar-agent · 2026-04-06T16:14:10Z

@codex review

Addressed both:

Added 300s timeout via setTimeout that disposes the push process (matching old bundle path's timeout)
SSH transport failures (exit code 255, "Connection" errors) now report to this.transport.reportFailure(), and successful pushes call markHealthy()

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c14bbadfcd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ammar-agent · 2026-04-06T16:22:44Z

@codex review

Addressed with documentation: added a NOTE comment explaining the SSH transport coupling. git push bypasses the runtime's SSHTransport, but this is safe because:

git push uses git's own SSH support, not the runtime transport
Git for Windows bundles ssh.exe — if git is installed, ssh is available for push
ControlMaster/ControlPath args are harmlessly ignored on Windows
The error path already catches and reports push failures clearly with connection pool health tracking

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a1bfcfee4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ammar-agent · 2026-04-06T16:31:01Z

@codex review

Fixed: switched from SCP-style (host:path) to ssh:// URL format which handles IPv6 natively (brackets in URL syntax), supports ~/ paths via /~/ prefix, and avoids the ambiguous : splitting issue.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fd6292259b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ammar-agent · 2026-04-06T16:40:26Z

@codex review

Fixed: split user@host before IPv6 detection to avoid double-bracketing. Now correctly handles user@[::1], 2001:db8::1, and plain hostnames.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 76ac2d68a0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ammar-agent · 2026-04-06T16:48:11Z

@codex review

P2 fixed: SSH failure detection now uses the structured .code exit code from execFileAsync errors and delegates to transport.isConnectionFailure() — correctly catches all SSH exit-255 failures regardless of stderr text.

P1 (SSH2/OpenSSH): This concern was addressed in a previous round with a code comment (lines 1038–1043). git push depends on git's own SSH support, not the runtime transport. Git for Windows bundles ssh.exe alongside git.exe — there is no realistic scenario where git is installed but ssh is unavailable. If such a case exists, the push fails with a clear error message that the user can act on. Re-adding the full bundle pipeline (~150 lines) as a fallback for this theoretical edge case would negate the purpose of this PR.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7e312f8c4d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ammar-agent · 2026-04-06T16:58:48Z

@codex review

Fixed: relative paths in srcBaseDir (e.g., src without ~/ or / prefix) are now correctly treated as home-relative via /~/ prefix in the ssh:// URL. Three-way branching: ~/ → /~/rest, / → as-is (absolute), else → /~/path (relative→home-relative).

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 574390b717

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ammar-agent · 2026-04-06T17:08:14Z

Note for human reviewer: Codex has raised the SSH2 transport concern three times across three review rounds (rounds 6, 8, and 10), each time as a P1. Each variation has been addressed:

Round 6: Added a NOTE comment documenting the transport coupling and why it's safe (git bundles ssh.exe).
Round 8: Same concern restated — resolved with the existing documentation.
Round 10 (this one): Now about host-key verification. buildGitSshCommand() already calls appendOpenSSHHostKeyPolicyArgs() which disables strict host-key checking in headless/server mode (adds StrictHostKeyChecking=no). In desktop mode, the OpenSSH connection pool probe establishes the control master and populates known_hosts before sync runs.

The SSH2 transport on Windows is a narrow edge case. Git for Windows bundles ssh.exe and supports SSH push natively. Re-adding the full 150-line bundle pipeline as a fallback would negate the purpose of this PR.

Pausing for human direction on this point — the other feedback items have all been addressed and CI checks are passing.

ammar-agent · 2026-04-06T21:39:52Z

@codex review

ammar-agent · 2026-04-06T21:40:42Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 72a9603e53

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ammar-agent · 2026-04-06T21:51:02Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a1e4ca9757

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Replace the git-bundle-create + SSH-pipe + git-fetch-bundle workflow with a single `git push` over SSH. Git's smart protocol natively negotiates which objects the remote already has, transferring only the delta. This makes subsequent workspace creations near-instant on slow connections. What changed: - syncProjectToRemote() now runs `git push --force --atomic` with a GIT_SSH_COMMAND that mirrors the runtime's SSH config (port, identity file, ControlPath multiplexing, host key policy) - Removed transferBundleToRemote() and all supporting bundle machinery (pipeReadableToWebWritable, waitForProcessExit, temp file cleanup) - Removed spawn/ChildProcess imports (no longer needed) - shouldReuseCurrentBundleTrunk() skip-if-unchanged check is preserved as an optimization to avoid even running git push when refs match

ammar-agent · 2026-04-06T22:01:57Z

@codex review

chatgpt-codex-connector · 2026-04-06T22:11:26Z

Codex Review: Didn't find any major issues. Delightful!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector Bot reviewed Apr 6, 2026

View reviewed changes

Comment thread src/node/runtime/SSHRuntime.ts Outdated

ammar-agent force-pushed the ssh-runtime-76mt branch from 0315409 to 65cb908 Compare April 6, 2026 15:49

ammar-agent changed the title ~~🤖 perf: incremental git bundle uploads for SSH runtime~~ 🤖 perf: replace git bundle with native git push for SSH runtime Apr 6, 2026

chatgpt-codex-connector Bot reviewed Apr 6, 2026

View reviewed changes

Comment thread src/node/runtime/SSHRuntime.ts Outdated

Comment thread src/node/runtime/SSHRuntime.ts

chatgpt-codex-connector Bot reviewed Apr 6, 2026

View reviewed changes

Comment thread src/node/runtime/SSHRuntime.ts Outdated

chatgpt-codex-connector Bot reviewed Apr 6, 2026

View reviewed changes

Comment thread src/node/runtime/SSHRuntime.ts Outdated

Comment thread src/node/runtime/SSHRuntime.ts Outdated

chatgpt-codex-connector Bot reviewed Apr 6, 2026

View reviewed changes

Comment thread src/node/runtime/SSHRuntime.ts Outdated

chatgpt-codex-connector Bot reviewed Apr 6, 2026

View reviewed changes

Comment thread src/node/runtime/SSHRuntime.ts Outdated

chatgpt-codex-connector Bot reviewed Apr 6, 2026

View reviewed changes

Comment thread src/node/runtime/SSHRuntime.ts Outdated

chatgpt-codex-connector Bot reviewed Apr 6, 2026

View reviewed changes

Comment thread src/node/runtime/SSHRuntime.ts

Comment thread src/node/runtime/SSHRuntime.ts Outdated

chatgpt-codex-connector Bot reviewed Apr 6, 2026

View reviewed changes

Comment thread src/node/runtime/SSHRuntime.ts Outdated

chatgpt-codex-connector Bot reviewed Apr 6, 2026

View reviewed changes

Comment thread src/node/runtime/SSHRuntime.ts Outdated

ammar-agent force-pushed the ssh-runtime-76mt branch from 574390b to daa6216 Compare April 6, 2026 21:39

ammar-agent force-pushed the ssh-runtime-76mt branch from daa6216 to 72a9603 Compare April 6, 2026 21:40

chatgpt-codex-connector Bot reviewed Apr 6, 2026

View reviewed changes

Comment thread src/node/runtime/SSHRuntime.ts Outdated

ammar-agent force-pushed the ssh-runtime-76mt branch from 72a9603 to a1e4ca9 Compare April 6, 2026 21:50

chatgpt-codex-connector Bot reviewed Apr 6, 2026

View reviewed changes

Comment thread src/node/runtime/SSHRuntime.ts Outdated

Comment thread src/node/runtime/SSHRuntime.ts Outdated

ammar-agent force-pushed the ssh-runtime-76mt branch from a1e4ca9 to 13bbfc3 Compare April 6, 2026 22:01

ammario merged commit fb25a63 into main Apr 6, 2026
24 checks passed

ammario deleted the ssh-runtime-76mt branch April 6, 2026 23:35

Conversation

ammar-agent commented Apr 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Background

Implementation

Risks

Uh oh!

ammar-agent commented Apr 6, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

ammar-agent commented Apr 6, 2026

Uh oh!

chatgpt-codex-connector Bot commented Apr 6, 2026

Uh oh!

ammar-agent commented Apr 6, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

ammar-agent commented Apr 6, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

ammar-agent commented Apr 6, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

ammar-agent commented Apr 6, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

ammar-agent commented Apr 6, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

ammar-agent commented Apr 6, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

ammar-agent commented Apr 6, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

ammar-agent commented Apr 6, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

ammar-agent commented Apr 6, 2026

Uh oh!

ammar-agent commented Apr 6, 2026 •

edited

Loading