Skip to content

fix(agent): scope workspace content folder by UID#512

Merged
skevetter merged 11 commits into
mainfrom
royal-snake
Jun 20, 2026
Merged

fix(agent): scope workspace content folder by UID#512
skevetter merged 11 commits into
mainfrom
royal-snake

Conversation

@skevetter

@skevetter skevetter commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Summary

When a workspace is deleted and recreated under a new UID (e.g. on a config change), the bind-mount source path was reused verbatim. Docker Desktop's macOS file-sharing VM caches /host_mnt entries by inode, so the recycled inode under the same path surfaces as bind source path does not exist on the next docker run — reproducible failure on the second devsy up, while the first run always succeeds.

  • Suffix the content folder with the workspace UID (content-<uid>) so each recreate uses a path Docker Desktop's VM has never bound before.
  • Clear the persisted ContentFolder immediately after the UID-change recreate so the recompute below always picks up the new UID-suffixed path.
  • Log docker host: os=… server=… storageDriver=… (once) and docker bind: src=… exists=… (per mount) before docker run, so future cache-race failures are diagnosable from the log alone — the conjunction "host=exists, daemon=missing" is the structural fingerprint.

Summary by CodeRabbit

Release Notes

  • New Features

    • Added Docker diagnostics for troubleshooting host environment and mount configurations.
    • Enhanced SSH server with graceful shutdown support and improved activity heartbeat monitoring.
  • Bug Fixes

    • Improved timeout handling and activity file management in container operations.
    • Fixed workspace content folder resolution to prevent stale inode caching issues.
  • Refactor

    • Centralized configuration constants for improved maintainability.
    • Streamlined SSH server command execution and token parsing logic.
  • Tests

    • Added comprehensive test coverage for SSH token parsing, activity file management, and server shutdown behavior.

When a workspace was deleted and recreated under a new UID, the bind-mount
source path was reused verbatim. Docker Desktop's macOS file-sharing VM
caches /host_mnt entries by inode, so the recycled inode under the same
path surfaced as "bind source path does not exist" on the next docker run.

Suffix the content folder with the workspace UID so each recreate uses a
fresh path, and clear the persisted ContentFolder on UID change so the
recompute always picks up the new value.

Also log host environment and bind-source resolution before docker run so
future cache-race failures are diagnosable from the log alone.
@netlify

netlify Bot commented Jun 20, 2026

Copy link
Copy Markdown

Deploy Preview for devsydev canceled.

Name Link
🔨 Latest commit 985e95d
🔍 Latest deploy log https://app.netlify.com/projects/devsydev/deploys/6a36c083d7bc6c0008765c47

@coderabbitai

coderabbitai Bot commented Jun 20, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR centralizes filesystem path constants and agent download URL logic from pkg/agent into pkg/config, migrates all consumers across the codebase, refactors pkg/agent workspace/tunnel APIs (UID-suffixed content dirs, TunnelOptions struct), rewrites the SSH server command with context-aware shutdown and activity heartbeat, adds a Shutdown method to the SSH Server interface, and introduces Docker bind-source/host-env diagnostics logging.

Changes

Config Centralization, Agent API Refactor, SSH Server Rewrite, Docker Diagnostics

Layer / File(s) Summary
pkg/config path constants and DefaultAgentDownloadURL
pkg/config/paths.go, pkg/config/repo.go, pkg/config/repo_test.go
Adds exported path constants (ContainerDataDir, ContainerDevsyHelperLocation, RemoteDevsyHelperLocation, ContainerActivityFile, WorkspaceBusyFile) and agent download URL constants plus DefaultAgentDownloadURL() with a test suite migrated from pkg/agent.
pkg/agent workspace UID suffix, TunnelOptions, and decodeWorkspaceInfoAndWrite refactor
pkg/agent/agent.go, pkg/agent/workspace.go, pkg/agent/inject.go, pkg/agent/agent_test.go
GetAgentWorkspaceContentDir gains a uid parameter for content-{uid} paths preventing stale inode reuse. decodeWorkspaceInfoAndWrite is modularized into handleStaleWorkspace/resolveContentFolder helpers. Tunnel adopts TunnelOptions. Busy-file helpers and inject defaults switch to config constants. Agent test suite removed.
SSH Server interface Shutdown method
pkg/ssh/server/ssh.go, pkg/ssh/server/ssh_container.go
Adds Shutdown(ctx context.Context) error to the exported Server interface; implements it on both server and containerServer by delegating to the underlying sshServer.Shutdown.
SSH server command refactor: context shutdown, heartbeat, token parsing
cmd/internal/ssh_server.go
Replaces exported SSHServerCmd with unexported sshServerCmd and a run(ctx) method. Adds parseSSHToken, decodeAuthorizedKeys, decodeBase64Bytes helpers. Both serve modes start shutdownOnCancel goroutine; stdio mode adds runActivityHeartbeat/ensureActivityFile replacing the prior ad-hoc goroutine. Listener mode errors on port conflict instead of silently exiting.
SSH server command tests
cmd/internal/ssh_server_test.go
Covers parseSSHToken, decodeAuthorizedKeys, decodeBase64Bytes, ensureActivityFile (create, idempotent, ENOTDIR), shutdownOnCancel (via fakeServer), ignoreServerClosed, and runActivityHeartbeat exit-on-cancel.
ContainerTunnelCmd adopts TunnelOptions
cmd/internal/container_tunnel.go
Switches from positional agent.Tunnel arguments to agent.TunnelOptions struct with Exec closure and explicit User/Stdin/Stdout/Stderr/Timeout fields.
Docker bind-source and host-env diagnostics
pkg/driver/docker/diagnostics.go, pkg/driver/docker/diagnostics_test.go, pkg/driver/docker/docker.go
Adds extractBindSources for --mount flag parsing, logBindSources for host-path existence checks, and logHostEnvOnce (sync.Once) for Docker server version/storage driver logging. startContainer calls both before RunWithDir. Test covers extractBindSources across flag formats.
Migrate all pkg/agent constant consumers to pkg/config
cmd/internal/agentcontainer/daemon.go, cmd/internal/agentworkspace/..., cmd/internal/fleet_server.go, cmd/workspace/ssh.go, pkg/agent/delivery/..., pkg/devcontainer/setup*.go, pkg/dotfiles/dotfiles.go, pkg/ide/..., pkg/options/resolve.go, pkg/tunnel/services.go
Replaces pkg/agent imports with pkg/config across all constant consumers. Also passes the workspace UID argument to GetAgentWorkspaceContentDir at the workspace-up call site.

Sequence Diagram(s)

sequenceDiagram
    participant cobra as cobra.Command
    participant sshServerCmd
    participant parseSSHToken
    participant ensureActivityFile
    participant runActivityHeartbeat
    participant shutdownOnCancel
    participant Server

    cobra->>sshServerCmd: run(ctx)
    sshServerCmd->>parseSSHToken: base64 token blob
    parseSSHToken-->>sshServerCmd: authorizedKeys, hostKey

    alt --stdio mode
        sshServerCmd->>ensureActivityFile: create activity file (if --track-activity)
        sshServerCmd->>runActivityHeartbeat: start goroutine (chtimes on interval)
        sshServerCmd->>shutdownOnCancel: start goroutine (ctx cancel → Shutdown)
        sshServerCmd->>Server: Serve(stdioListener)
    else listener mode
        sshServerCmd->>sshServerCmd: check port availability → error if in use
        sshServerCmd->>shutdownOnCancel: start goroutine
        sshServerCmd->>Server: ListenAndServe()
    end

    Note over shutdownOnCancel: ctx cancelled → Server.Shutdown(bounded timeout ctx)
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • devsy-org/devsy#20: Adds Shutdown(ctx) to the pkg/ssh/server interface, which is the same interface method this PR implements and exercises through shutdownOnCancel in the SSH server command.
  • devsy-org/devsy#22: Modifies the pkg/agent tunnel API (removing injected logger parameters), directly overlapping with this PR's TunnelOptions struct refactor of the same Tunnel function.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 24.14% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix(agent): scope workspace content folder by UID' directly and clearly describes the primary objective of the PR—scoping the workspace content folder with a UID suffix to address Docker Desktop caching issues.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@netlify

netlify Bot commented Jun 20, 2026

Copy link
Copy Markdown

Deploy Preview for images-devsy-sh canceled.

Name Link
🔨 Latest commit 985e95d
🔍 Latest deploy log https://app.netlify.com/projects/images-devsy-sh/deploys/6a36c08306bc2400085045c7

Container-side path constants (ContainerDataDir, ContainerDevsyHelperLocation,
RemoteDevsyHelperLocation, ContainerActivityFile, WorkspaceBusyFile) were
declared in pkg/agent next to runtime logic. They are pure paths derived from
config.BinaryName, so they belong in pkg/config alongside DevContainerResultPath
and SSHSignatureHelperPath. No behavior change; all call sites updated.
The function had grown to ~175 lines with five distinct phases (decode,
maybe-rerun-as-root, ensure dir, handle stale workspace, resolve content
folder + write) interleaved with a per-step "log.Errorf … return err"
pattern that doubled every failure in the log.

Extract handleStaleWorkspace and resolveContentFolder so the orchestrator
reads top-to-bottom in ~40 lines. Drop the redundant pre-return Errorf
pairs — wrapped errors already carry the cause. Preserve the
operationally meaningful INFO line ("delete old workspace: …") that ops
greps on.

No behavior change.
@github-actions github-actions Bot added size/l and removed size/m labels Jun 20, 2026
Tunnel had seven parameters; the four io.Reader/Writer values plus user
and timeout fit naturally in an options struct. Keep ctx as the first
positional arg per Go convention.
@github-actions github-actions Bot added size/xl and removed size/l labels Jun 20, 2026
Lift AgentDownloadBaseURL and AgentLatestDownloadURL to pkg/config alongside
GitHubReleasesURL — they're path constants, not runtime state. The
DefaultAgentDownloadURL function stays in pkg/agent because it dispatches on
env override and version (behavior, not a path).
It's a pure path resolver — picks between an env override, the latest URL,
and the version-pinned URL — so it belongs alongside the URL constants in
pkg/config/repo.go. The agent package no longer needs to import
pkg/version. Tests move with the function.
The original comment referred to /var/lib/loft/* permissions from devsy's
upstream fork. devsy doesn't use that path — the predicate is still
correct (Platform-enabled runs can't address the host's LocalFolder), but
the reasoning needed to match.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses a reproducible Docker Desktop (macOS) bind-mount failure after workspace delete/recreate by ensuring the workspace “content” bind source path changes when the workspace UID changes, and by adding pre-docker run diagnostics to make host-vs-daemon path-resolution issues visible in logs.

Changes:

  • Scope the workspace content folder by UID (content-<uid>) and clear persisted ContentFolder on UID-change recreation so the new UID-suffixed path is recomputed.
  • Centralize agent/container path constants and the default agent download URL under pkg/config, updating call sites accordingly.
  • Add Docker run diagnostics: host environment logging (once) and bind source existence checks (per mount), with a focused unit test for bind-source extraction.

Reviewed changes

Copilot reviewed 29 out of 29 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
pkg/tunnel/services.go Use config.ContainerDevsyHelperLocation when building the credentials-server command.
pkg/options/resolve.go Switch agent path/URL defaults from agent to config helpers.
pkg/ide/rstudio/rstudio.go Use config.ContainerDataDir for the RStudio download folder path.
pkg/ide/jetbrains/generic.go Use config.ContainerDataDir for JetBrains server download folder path.
pkg/driver/docker/docker.go Emit docker host/bind diagnostics before invoking docker run.
pkg/driver/docker/diagnostics.go New helper functions to log docker host env and bind-mount source existence.
pkg/driver/docker/diagnostics_test.go Unit test coverage for bind source extraction from docker args.
pkg/dotfiles/dotfiles.go Use config.ContainerDevsyHelperLocation when building dotfiles agent command.
pkg/devcontainer/setup/setup.go Use config.ContainerDataDir for marker file paths inside containers.
pkg/devcontainer/setup.go Use config for default agent download URL and helper binary location.
pkg/config/repo.go Introduce config.DefaultAgentDownloadURL() (env override + version-pinned/latest logic).
pkg/config/repo_test.go New tests for URL trimming behavior in config.DefaultAgentDownloadURL().
pkg/config/paths.go Centralize container/remote helper paths and workspace busy file name in config.
pkg/agent/workspace.go Update search locations and change content dir helper to require UID suffix.
pkg/agent/inject.go Use config.RemoteDevsyHelperLocation and config.DefaultAgentDownloadURL().
pkg/agent/delivery/remote_docker.go Use config.ContainerDevsyHelperLocation as delivery destination path.
pkg/agent/delivery/local_docker.go Use config.ContainerDevsyHelperLocation for deriving the binary name.
pkg/agent/delivery/legacy_shell.go Use config helper paths and default download URL.
pkg/agent/delivery/kubernetes.go Use config.ContainerDevsyHelperLocation for in-cluster binary placement.
pkg/agent/delivery/kubernetes_test.go Update test expectations to use config.ContainerDevsyHelperLocation.
pkg/agent/agent.go Refactor workspace info writing flow, implement UID-scoped content dir, add TunnelOptions.
pkg/agent/agent_test.go Remove tests that moved to pkg/config/repo_test.go.
cmd/workspace/ssh.go Use config.ContainerDevsyHelperLocation for internal agent commands.
cmd/internal/ssh_server.go Use config.ContainerActivityFile for activity tracking (now has a race issue noted).
cmd/internal/fleet_server.go Use config.ContainerActivityFile when touching activity file.
cmd/internal/container_tunnel.go Update call site to new agent.Tunnel(ctx, agent.TunnelOptions{...}) API.
cmd/internal/agentworkspace/up.go Update runner defaults + pass UID into GetAgentWorkspaceContentDir.
cmd/internal/agentworkspace/logs.go Use config defaults for helper path and download URL in runner creation.
cmd/internal/agentcontainer/daemon.go Use config.ContainerDataDir and config.ContainerActivityFile.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread cmd/internal/ssh_server.go Outdated
Comment on lines 123 to 125
go func() {
_, err = os.Stat(agent.ContainerActivityFile)
_, err = os.Stat(config.ContainerActivityFile)
if err != nil {
Comment on lines +58 to +62
func logBindSources(args []string) {
for _, src := range extractBindSources(args) {
_, err := os.Lstat(src)
log.Infof("docker bind: src=%s exists=%t", src, err == nil)
}
… safety

- Propagate context: Run takes ctx, the activity heartbeat exits on cancel
  instead of running until process exit.
- Replace the os.Create/Close every-10s touch with os.Chtimes — same effect,
  no truncation, no nil-handle panic on failure.
- Fix the "is in use" silent-success path: ListenAndServe now errors when
  the port is taken instead of logging and returning nil.
- Wrap base64 decode errors instead of dropping them; uniform error
  formatting throughout.
- Use errors.Is(err, fs.ErrNotExist) for the activity-file existence check
  so non-ENOENT stat errors surface.
- Validate flag combinations up front: --track-activity now requires --stdio.
- Extract parseSSHToken / decodeAuthorizedKeys / decodeBase64Bytes /
  ensureActivityFile / runActivityHeartbeat so each piece is independently
  testable; add unit tests covering the pure logic and ctx-cancel semantics.
- Add Shutdown(ctx) to the sshserver.Server interface; implement on both
  the host server and the container server by delegating to the underlying
  ssh.Server.Shutdown.
- Wire into cmd/internal: a goroutine waits on ctx.Done and calls
  Shutdown with a 5s timeout, so the SSH server actually drains on
  SIGTERM instead of running until process exit. The shutdown context
  derives from Background by design — the parent ctx is already canceled
  by the time we get here.
- Wrap the server-closed error: ssh.ErrServerClosed is the expected
  outcome of a graceful shutdown, not a CLI failure.
- Lower SSHServerCmd to unexported sshServerCmd — nothing outside the
  package needed it.
@skevetter
skevetter marked this pull request as ready for review June 20, 2026 17:12

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
pkg/config/repo_test.go (2)

27-40: ⚡ Quick win

Add test coverage for dev and release version paths.

The current tests only verify trailing-slash normalization when EnvAgentURL is set. The core version-based branching logic in DefaultAgentDownloadURL() (dev version → latest, release version → versioned URL) is untested.

📋 Suggested test cases
func (s *RepoTestSuite) TestDefaultAgentDownloadURL_DevVersion() {
	// Ensure EnvAgentURL is not set
	_ = os.Unsetenv(EnvAgentURL)
	
	// Mock version.GetVersion to return version.DevVersion
	// (requires version package to support injection or this to be an integration test)
	// Expected: returns AgentLatestDownloadURL
}

func (s *RepoTestSuite) TestDefaultAgentDownloadURL_ReleaseVersion() {
	// Ensure EnvAgentURL is not set
	_ = os.Unsetenv(EnvAgentURL)
	
	// Mock version.GetVersion to return a release version like "v1.2.3"
	// Expected: returns AgentDownloadBaseURL + "v1.2.3"
}

Note: These tests may require refactoring DefaultAgentDownloadURL to accept a version parameter or using test-time dependency injection for version.GetVersion().

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/config/repo_test.go` around lines 27 - 40, The test suite for
DefaultAgentDownloadURL() lacks coverage for the core version-based branching
logic that distinguishes between dev and release versions. Add two new test
methods to the RepoTestSuite: TestDefaultAgentDownloadURL_DevVersion which
ensures EnvAgentURL is unset and verifies the function returns
AgentLatestDownloadURL for dev versions, and
TestDefaultAgentDownloadURL_ReleaseVersion which also unsets EnvAgentURL and
verifies the function returns the versioned URL (AgentDownloadBaseURL plus
version) for release versions. You may need to refactor DefaultAgentDownloadURL
to accept a version parameter or introduce dependency injection for the
version.GetVersion() call to make these tests feasible without modifying global
state.

15-25: 💤 Low value

Consider using os.LookupEnv for more robust environment restoration.

os.Getenv returns "" for both unset variables and variables set to empty string. If EnvAgentURL was originally set to "" (unlikely but possible), TearDownTest will unset it instead of restoring the empty value. Using os.LookupEnv would correctly distinguish the two cases.

♻️ More robust cleanup using LookupEnv
 type RepoTestSuite struct {
 	suite.Suite
-	originalEnv string
+	originalEnv       string
+	originalEnvExists bool
 }
 
 func (s *RepoTestSuite) SetupTest() {
-	s.originalEnv = os.Getenv(EnvAgentURL)
+	s.originalEnv, s.originalEnvExists = os.LookupEnv(EnvAgentURL)
 }
 
 func (s *RepoTestSuite) TearDownTest() {
-	if s.originalEnv != "" {
+	if s.originalEnvExists {
 		_ = os.Setenv(EnvAgentURL, s.originalEnv)
 	} else {
 		_ = os.Unsetenv(EnvAgentURL)
 	}
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/config/repo_test.go` around lines 15 - 25, The SetupTest method uses
os.Getenv which cannot distinguish between an unset environment variable and one
set to an empty string, both returning "". To fix this, modify SetupTest to use
os.LookupEnv instead, which returns both the value and a boolean indicating
whether the variable exists. Store both the value and existence status (the
boolean) in the RepoTestSuite struct. Then update TearDownTest to use this
stored boolean to correctly determine whether to set the variable back to its
original value (including empty strings) or unset it completely.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@pkg/config/repo_test.go`:
- Around line 27-40: The test suite for DefaultAgentDownloadURL() lacks coverage
for the core version-based branching logic that distinguishes between dev and
release versions. Add two new test methods to the RepoTestSuite:
TestDefaultAgentDownloadURL_DevVersion which ensures EnvAgentURL is unset and
verifies the function returns AgentLatestDownloadURL for dev versions, and
TestDefaultAgentDownloadURL_ReleaseVersion which also unsets EnvAgentURL and
verifies the function returns the versioned URL (AgentDownloadBaseURL plus
version) for release versions. You may need to refactor DefaultAgentDownloadURL
to accept a version parameter or introduce dependency injection for the
version.GetVersion() call to make these tests feasible without modifying global
state.
- Around line 15-25: The SetupTest method uses os.Getenv which cannot
distinguish between an unset environment variable and one set to an empty
string, both returning "". To fix this, modify SetupTest to use os.LookupEnv
instead, which returns both the value and a boolean indicating whether the
variable exists. Store both the value and existence status (the boolean) in the
RepoTestSuite struct. Then update TearDownTest to use this stored boolean to
correctly determine whether to set the variable back to its original value
(including empty strings) or unset it completely.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 1f39bb99-a984-4436-a26d-4e8f374e3017

📥 Commits

Reviewing files that changed from the base of the PR and between 3233676 and 985e95d.

📒 Files selected for processing (32)
  • cmd/internal/agentcontainer/daemon.go
  • cmd/internal/agentworkspace/logs.go
  • cmd/internal/agentworkspace/up.go
  • cmd/internal/container_tunnel.go
  • cmd/internal/fleet_server.go
  • cmd/internal/ssh_server.go
  • cmd/internal/ssh_server_test.go
  • cmd/workspace/ssh.go
  • pkg/agent/agent.go
  • pkg/agent/agent_test.go
  • pkg/agent/delivery/kubernetes.go
  • pkg/agent/delivery/kubernetes_test.go
  • pkg/agent/delivery/legacy_shell.go
  • pkg/agent/delivery/local_docker.go
  • pkg/agent/delivery/remote_docker.go
  • pkg/agent/inject.go
  • pkg/agent/workspace.go
  • pkg/config/paths.go
  • pkg/config/repo.go
  • pkg/config/repo_test.go
  • pkg/devcontainer/setup.go
  • pkg/devcontainer/setup/setup.go
  • pkg/dotfiles/dotfiles.go
  • pkg/driver/docker/diagnostics.go
  • pkg/driver/docker/diagnostics_test.go
  • pkg/driver/docker/docker.go
  • pkg/ide/jetbrains/generic.go
  • pkg/ide/rstudio/rstudio.go
  • pkg/options/resolve.go
  • pkg/ssh/server/ssh.go
  • pkg/ssh/server/ssh_container.go
  • pkg/tunnel/services.go
💤 Files with no reviewable changes (1)
  • pkg/agent/agent_test.go

@skevetter
skevetter merged commit 6da1f01 into main Jun 20, 2026
58 checks passed
@skevetter
skevetter deleted the royal-snake branch June 20, 2026 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants