Why does execd wait 1s before closing command SSE responses?

[hrf1009]

Description

I noticed that foreground command_run calls have an almost constant ~1 second overhead, even for commands that complete almost immediately, such as:

true
echo hello
mkdir /tmp/test
cat small_file.txt

In my tests, filesystem APIs such as file_read, file_write, and file_create_directories usually complete within about 10–15 ms, while command_run consistently takes around 1.01 s.

After breaking down the HTTP/SSE timing, most of the extra latency appears after the command has already completed and the final SSE events have been sent.

The execd documentation mentions:

--graceful-shutdown-timeout
Default: 1s
SSE tail-drain wait window before closing

and the corresponding environment variable:

EXECD_API_GRACE_SHUTDOWN

Questions

I would like to understand the design rationale behind this default 1 second delay:

1. Why is a 1 second tail-drain window needed after the final command SSE event has already been sent?

2. What specific issue is this grace period intended to prevent?

   • Lost stdout/stderr events?
   • Lost completion events?
   • Client-side SSE parsing/connection-close races?

3. Is it safe and supported to set:

EXECD_API_GRACE_SHUTDOWN=0s

for latency-sensitive workloads?
4. If 0s is not recommended, would a smaller value such as 50ms or 100ms normally be sufficient?
5. Is the 1 second default mainly a conservative compatibility/reliability choice, or is there a known case that actually requires a delay this long?

Observed timing

The command itself appears to finish very quickly, but the HTTP/SSE request remains open for approximately another second.

Conceptually, the timing looks like:

command starts
    ↓
command finishes
    ↓
final stdout/stderr/completion SSE event
    ↓
~1 second wait
    ↓
SSE response closes
    ↓
SDK command_run returns

This makes very short shell commands have roughly the same ~1.01 s latency regardless of the actual command execution time.

Environment

• OpenSandbox Server: 0.2.2
• OpenSandbox MCP: 0.1.1
• Python SDK (opensandbox): 0.1.15
• Docker runtime
• Foreground /command execution

Thanks! I mainly want to understand whether this 1 second delay is required for correctness, or whether reducing/disabling it is an expected configuration for latency-sensitive scenarios.

[Zer0codestuff]

Your timing matches an unconditional server-side sleep, not a client-side SSE parsing delay.

In the current RunCommand path, execd first waits for OnExecuteComplete, then always calls:

time.Sleep(flag.ApiGracefulShutdownTimeout)

That is here:

OpenSandbox/components/execd/pkg/web/controller/command.go

Lines 71 to 121 in 48b0215

	// completeCh is closed when OnExecuteComplete fires, meaning the final SSE
	// event has been written and flushed. We only wait for this callback as a
	// safety check and then return immediately to avoid fixed tail latency.
	completeCh := make(chan struct{})
	var completeOnce sync.Once
	signalComplete := func() {
	completeOnce.Do(func() {
	close(completeCh)
	})
	}
	origComplete := eventsHandler.OnExecuteComplete
	eventsHandler.OnExecuteComplete = func(executionTime time.Duration) {
	origComplete(executionTime)
	recordExecution("success")
	signalComplete()
	}
	origError := eventsHandler.OnExecuteError
	eventsHandler.OnExecuteError = func(err *execute.ErrorOutput) {
	origError(err)
	recordExecution("failure")
	signalComplete()
	}
	runCodeRequest.Hooks = eventsHandler
	// Cancel the context first (signals the ping goroutine to stop), then
	// wait for it to fully exit before the handler returns. This prevents
	// the ping goroutine from writing to the response after Go's net/http
	// closes the response writer.
	defer func() { cancel(); stopSSE() }()
	// SSE headers are committed lazily on the first event write
	// (see writeSingleEvent), so a synchronous error from Execute below can
	// still be surfaced as a structured JSON error response.
	err = codeRunner.Execute(runCodeRequest)
	if err != nil {
	recordExecution("failure")
	c.RespondError(
	http.StatusInternalServerError,
	model.ErrorCodeRuntimeError,
	fmt.Sprintf("error running commands %v", err),
	)
	return
	}
	waitForExecutionComplete(ctx, completeCh)
	// Keep the SSE connection alive briefly so clients can read all
	// buffered events and downstream components (e.g. egress sidecar)
	// have time to synchronise state changes that were triggered
	// during command execution.
	time.Sleep(flag.ApiGracefulShutdownTimeout)

The completion callback is signalled only after the final event writer returns. writeSingleEvent serializes writes under a lock and calls http.Flusher.Flush() before returning:

OpenSandbox/components/execd/pkg/web/controller/sse.go

Lines 187 to 215 in 48b0215

	// writeSingleEvent serializes one SSE frame.
	func (c *basicController) writeSingleEvent(handler string, data []byte, verbose bool, summary string) {
	if c == nil || c.ctx == nil || c.ctx.Writer == nil {
	return
	}
	select {
	case <-c.ctx.Request.Context().Done():
	log.Error("StreamEvent.%s: client disconnected", handler)
	return
	default:
	}
	c.chunkWriter.Lock()
	defer c.chunkWriter.Unlock()
	// Lazily commit SSE response headers on the first event. This lets the
	// surrounding handler return a proper JSON error via RespondError if the
	// runtime fails synchronously before any event fires.
	c.setupSSEResponse()
	defer func() {
	if flusher, ok := c.ctx.Writer.(http.Flusher); ok {
	flusher.Flush()
	}
	}()
	payload := append(data, '\n', '\n')
	n, err := c.ctx.Writer.Write(payload)
	if err == nil && n != len(payload) {
	err = io.ErrShortWrite

So, for stdout/stderr/completion delivery inside execd itself, the extra second is not waiting for an acknowledgement or draining a queue. It is a fixed tail delay after the event has been written and flushed.

There is an interesting history detail. PR #1216 added an explicit completion channel plus a ping-goroutine shutdown barrier to fix the actual response-writer race. Its commit message says it replaces the fixed sleep with explicit cancel-and-wait, and the code comment at lines 71-73 says the handler should return immediately to avoid fixed tail latency. The old sleep nevertheless remains at line 121. The new adjacent comment gives a second rationale, allowing downstream components such as the egress sidecar to synchronize, but there is no downstream acknowledgement tied to that sleep. See 97a450a

For the configuration questions:

• EXECD_API_GRACE_SHUTDOWN=0s is accepted by the current parser. It uses time.ParseDuration and applies no positive-value validation:

  OpenSandbox/components/execd/pkg/flag/parser.go

  Lines 38 to 91 in 48b0215

  	// Set default values
  	ServerPort = 44772
  	ServerLogLevel = 6
  	ServerAccessToken = ""
  	ApiGracefulShutdownTimeout = time.Second * 1
  	JupyterIdlePollInterval = 100 * time.Millisecond
  	IsolationConfigPath = ""
  	InitMode = false
  	LifecycleStartupStatusFile = ""
  	// First, set default values from environment variables
  	if jupyterFromEnv := os.Getenv(jupyterHostEnv); jupyterFromEnv != "" {
  	if !strings.HasPrefix(jupyterFromEnv, "http://") && !strings.HasPrefix(jupyterFromEnv, "https://") {
  	stdlog.Panic("Invalid JUPYTER_HOST format: must start with http:// or https://")
  	}
  	JupyterServerHost = jupyterFromEnv
  	}
  	if jupyterTokenFromEnv := os.Getenv(jupyterTokenEnv); jupyterTokenFromEnv != "" {
  	JupyterServerToken = jupyterTokenFromEnv
  	}
  	if accessTokenFromEnv := os.Getenv(accessTokenEnv); accessTokenFromEnv != "" {
  	ServerAccessToken = accessTokenFromEnv
  	}
  	// Then define flags with current values as defaults
  	flag.StringVar(&JupyterServerHost, "jupyter-host", JupyterServerHost, "Jupyter server host address (e.g., http://localhost, http://192.168.1.100)")
  	flag.StringVar(&JupyterServerToken, "jupyter-token", JupyterServerToken, "Jupyter server authentication token")
  	flag.IntVar(&ServerPort, "port", ServerPort, "Server listening port (default: 44772)")
  	flag.IntVar(&ServerLogLevel, "log-level", ServerLogLevel, "Server log level (0=LevelEmergency, 1=LevelAlert, 2=LevelCritical, 3=LevelError, 4=LevelWarning, 5=LevelNotice, 6=LevelInformational, 7=LevelDebug, default: 6)")
  	flag.StringVar(&ServerAccessToken, "access-token", ServerAccessToken, "Server access token for API authentication")
  	if graceShutdownTimeout := os.Getenv(gracefulShutdownTimeoutEnv); graceShutdownTimeout != "" {
  	duration, err := time.ParseDuration(graceShutdownTimeout)
  	if err != nil {
  	stdlog.Panicf("Failed to parse graceful shutdown timeout from env: %v", err)
  	}
  	ApiGracefulShutdownTimeout = duration
  	}
  	if idlePollInterval := os.Getenv(jupyterIdlePollIntervalEnv); idlePollInterval != "" {
  	duration, err := time.ParseDuration(idlePollInterval)
  	if err != nil {
  	stdlog.Panicf("Failed to parse jupyter idle poll interval from env: %v", err)
  	}
  	if duration <= 0 {
  	stdlog.Printf("Invalid %s=%s; fallback to default %s", jupyterIdlePollIntervalEnv, idlePollInterval, JupyterIdlePollInterval)
  	} else {
  	JupyterIdlePollInterval = duration
  	}
  	}
  	flag.DurationVar(&ApiGracefulShutdownTimeout, "graceful-shutdown-timeout", ApiGracefulShutdownTimeout, "API graceful shutdown timeout duration (default: 1s)")

• In Go, time.Sleep(0) returns immediately, so this removes the fixed command tail latency.
• 50ms and 100ms are equally valid configuration values.
• The repository does not currently have a zero-delay regression test or a readiness acknowledgement proving that zero is safe for every downstream sidecar transition. I would therefore separate two claims: zero is supported by the parser and sufficient for the current execd event-write path, but the code does not establish a universal safety guarantee for commands that trigger asynchronous egress or network-policy state changes.

For latency-sensitive commands that do not change that downstream state, 0s is consistent with the current synchronization path. If those state-changing commands are in scope, the reliable fix is an explicit readiness barrier from the downstream component, not an arbitrary 50ms, 100ms, or 1s sleep. Issue #1661 looks justified as either removing the leftover sleep or documenting and testing the downstream contract.