Skip to content

fix(connect): diagnose incomplete headless server setup - #9602

Merged
juliusmarminge merged 3 commits into
mainfrom
fix-t3-connect-server
Sep 4, 2026
Merged

fix(connect): diagnose incomplete headless server setup#9602
juliusmarminge merged 3 commits into
mainfrom
fix-t3-connect-server

Conversation

@juliusmarminge

@juliusmarminge juliusmarminge commented Sep 4, 2026

Copy link
Copy Markdown
Member

Headless setup could leave an installed-looking service that stopped at logout. On the affected Debian host, enable-linger failed after the unit and runtime were written; later status and update commands treated those files as a complete installation. Earlier attempts retried the same opaque HTTP 403 twenty times, and a tunnel cleanup failure hid its cause behind a nested object.

This makes incomplete Connect setup diagnosable before users depend on it:

  • Check the systemd user manager and lingering before downloading a runtime or stopping an existing service. Permission failures include the administrator command and leave an existing service running.
  • Check whether an installed Linux service is running, enabled, and allowed to survive logout. Same-version repair no longer skips incomplete setup, and suggested repair commands use the newer of the CLI and installed versions, avoiding nightly downgrades.
  • Preserve relay error reasons and trace IDs, with recovery steps for authorization, link proofs, and tunnel limits. Unrecognized responses retain their HTTP status and Cloudflare Ray ID without printing their body.
  • Stop startup retries on permanent 4xx rejections; retain retries for timeouts, rate limits, and server failures. Print readable startup and shutdown diagnostics.
  • Say “Authorized” once credentials are saved, explain that Connect status reports saved configuration, and document the service and relay recovery steps.

Before, the service error was only Background setup failed while enabling lingering for this user (exit code 1). A later status command reported it installed. With the same incomplete state, status now reports:

Status: needs an update or repair
[linger-disabled] Lingering is disabled. T3 Code will stop when your last login session ends and will not start at boot. Run `sudo loginctl enable-linger "$(id -un)"` on this machine, then retry the service command as your normal user.
Next: Run `npx t3@0.0.39-nightly.20260904.1275 service update`.

Validation: 94 focused tests across service setup, CLI recovery, relay response handling, and cloud HTTP behavior; server typecheck; targeted lint and formatting. The Linux probe commands were verified read-only on the affected host. Regression coverage checks that denied setup writes no service files, failed repair preserves the existing service, and a rejected relay request is attempted once while transient failures retry. This changes CLI and server behavior, so screenshots are not applicable. No client/provider protocol or production service was changed.

Model: GPT-6. Harness: Codex.

Closes #8476
Closes #5612


Note

Medium Risk
Changes install ordering and cloud-link retry behavior on server startup; mistakes could block service install or retry auth failures too long, but failures are fail-closed with tests covering prerequisite and relay paths.

Overview
Improves headless T3 Connect when the background service looks installed but cannot stay reachable (e.g. lingering failed after files were written).

Linux background service: Install now probes systemd (user manager, linger, enabled/running) before downloading a runtime or stopping an existing unit. Failures surface as coded problems with recovery text (BootServicePrerequisiteError); failed repair does not stop a working service. t3 service status lists those problems and suggests npx t3@<version> service update using the current CLI (or the newer installed version), not t3@latest.

Relay / startup: New filterRelayResponse maps relay errors to actionable messages (trace IDs, tunnel limits, auth hints) without leaking raw bodies; shouldRetryCloudLink stops startup retries on permanent 4xx while keeping transient failures retryable.

CLI copy: Connect flow says Authorized (not Connected), clarifies status is saved setup not live reachability, and documents troubleshooting in user docs.

Reviewed by Cursor Bugbot for commit 5aaf56c. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add prerequisite checks and relay error diagnostics to t3 connect

  • Adds BootServicePrerequisiteError and prerequisite checks in bootService.ts to detect systemd user-manager and lingering issues. Installation fails early and falls back to manual setup if lingering cannot be enabled.
  • Introduces filterRelayResponse and relayRequestError in relayResponse.ts to sanitize transport errors, surface categorized relay diagnostics (like tunnel limits or invalid authorization), and stop retries for permanent failures.
  • Updates CLI status and onboarding in service.ts and connect.ts to print problem codes, offer version-specific repair commands, and distinguish saved authorization from live service reachability.
  • Risk: systemdManager activation no longer attempts to enable lingering post-install; setups with lingering disabled will fail during the new prerequisite phase instead of the activation phase.

Macroscope summarized 5aaf56c.

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Sep 4, 2026
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ No successful main baseline artifact is available yet. This run establishes the initial measurement.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 13.8 KiB 15.1 KiB
Codex Thread snapshot wire 7.0 KiB 7.3 KiB
Codex Live turn WebSocket wire 6.7 KiB 7.8 KiB
Codex Live turn WebSocket decoded 58.5 KiB 66.4 KiB
Codex Live turn messages 10 21
Claude Total thread wire 13.5 KiB 15.1 KiB
Claude Thread snapshot wire 7.0 KiB 7.3 KiB
Claude Live turn WebSocket wire 6.5 KiB 7.8 KiB
Claude Live turn WebSocket decoded 57.8 KiB 66.4 KiB
Claude Live turn messages 8 21

Baseline: unavailable · PR result: 5aaf56c · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 113.8 KiB
  • Claude decoded thread snapshot: 114.5 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

Comment thread apps/server/src/cloud/relayResponse.ts Outdated
Comment thread apps/server/src/cloud/relayResponse.ts
@macroscopeapp

macroscopeapp Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Not approved

Macroscope's review found this PR not approvable — The PR changes production startup and background-service lifecycle behavior, including systemd probing, lingering setup, relay retry policy, and authorization error handling. These cross-cutting runtime and authentication changes warrant human review despite the focused tests and documentation.

You can add or adjust custom eligibility rules. Learn more.

@juliusmarminge
juliusmarminge merged commit 99e3b72 into main Sep 4, 2026
26 checks passed
@juliusmarminge
juliusmarminge deleted the fix-t3-connect-server branch September 4, 2026 07:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L 100-499 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

1 participant