Skip to content

Stabilize remote full-ci test startup paths#22742

Open
starr-openai wants to merge 1 commit into
mainfrom
starr/fullci-3-remote-test-startup
Open

Stabilize remote full-ci test startup paths#22742
starr-openai wants to merge 1 commit into
mainfrom
starr/fullci-3-remote-test-startup

Conversation

@starr-openai
Copy link
Copy Markdown
Contributor

@starr-openai starr-openai commented May 15, 2026

Why

After the earlier full-ci fixes, the Docker-backed remote test lane still exposed startup assumptions that do not hold in minimal remote containers: helper paths were inferred from local defaults and remote denial tests requested bash.

Failure evidence: remote Ubuntu test job with uploaded nextest JUnit artifact

What changed

  • carry remote Codex and sandbox-helper paths through the remote test harness config
  • create custom remote Codex parent directories before docker cp
  • publish the remote Codex path only when this script installed that binary
  • quote remote exec-server startup paths instead of interpolating them through sh -lc
  • run remote network-denial fixtures with sh/non-login shell settings available in the minimal container

CI evidence

Recent failed rust-ci-full test runs that exposed this flake family:

Those failed test lanes uploaded nextest JUnit artifacts successfully. Representative artifacts from the latest run:

Walkthrough

  • Issue: the Docker-backed remote lane had startup assumptions that did not hold across steps or minimal containers: the installed remote Codex path was not persisted to the later test step, path quoting was brittle, and remote denial fixtures asked for bash.
  • Likely introducing change: remote test env setup around scripts/test-remote-env.sh plus the remote unified-exec fixtures in this PR.
  • Fix: use the remote Codex path already provided by the Docker setup, quote remote exec startup paths safely, use sh/non-login shell settings, and keep the remote harness pointed at the binary the script actually installed.

Verification

  • unstacked the PR onto current origin/main; GitHub now shows base main and a one-commit standalone diff containing only the intended three files
  • focused remote-env verification passed for both installed custom-path mode and external exec-server mode on the standalone source snapshot
  • current CI is green on the standalone PR head

@starr-openai starr-openai force-pushed the starr/fullci-2-unified-exec-startup branch from a422183 to db00244 Compare May 15, 2026 01:46
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch from b66cd84 to 00ab833 Compare May 15, 2026 01:46
@starr-openai starr-openai force-pushed the starr/fullci-2-unified-exec-startup branch from db00244 to a4e48cb Compare May 15, 2026 01:46
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch from 00ab833 to 086e533 Compare May 15, 2026 01:46
@starr-openai starr-openai force-pushed the starr/fullci-2-unified-exec-startup branch from a4e48cb to 21c7e73 Compare May 15, 2026 02:02
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch from 086e533 to 0ca29ce Compare May 15, 2026 02:02
@starr-openai starr-openai force-pushed the starr/fullci-2-unified-exec-startup branch from 21c7e73 to ca1d3de Compare May 15, 2026 02:19
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch 2 times, most recently from 224d9e0 to 230d335 Compare May 15, 2026 02:41
@starr-openai starr-openai force-pushed the starr/fullci-2-unified-exec-startup branch from ca1d3de to 098833a Compare May 15, 2026 02:41
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch from 230d335 to 73f5b5a Compare May 15, 2026 02:56
@starr-openai starr-openai force-pushed the starr/fullci-2-unified-exec-startup branch 2 times, most recently from 1a8be67 to 672e10b Compare May 15, 2026 03:21
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch from 73f5b5a to 3ea70c7 Compare May 15, 2026 03:21
@starr-openai starr-openai force-pushed the starr/fullci-2-unified-exec-startup branch from 672e10b to db8fe52 Compare May 15, 2026 03:47
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch from 3ea70c7 to 4344033 Compare May 15, 2026 03:47
@starr-openai starr-openai force-pushed the starr/fullci-2-unified-exec-startup branch from db8fe52 to 77a8012 Compare May 15, 2026 05:34
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch from 4344033 to 6736853 Compare May 15, 2026 05:34
@starr-openai starr-openai force-pushed the starr/fullci-2-unified-exec-startup branch from 77a8012 to c2a9217 Compare May 15, 2026 12:18
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch from 6736853 to d95d0bd Compare May 15, 2026 12:18
@starr-openai starr-openai force-pushed the starr/fullci-2-unified-exec-startup branch from c2a9217 to d118d31 Compare May 15, 2026 17:04
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch 2 times, most recently from 7cfbea4 to 8f5a4cb Compare May 15, 2026 19:31
@starr-openai starr-openai force-pushed the starr/fullci-2-unified-exec-startup branch 2 times, most recently from 8c0690f to 8d80010 Compare May 15, 2026 19:41
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch 2 times, most recently from c85b53b to 5d5c4c7 Compare May 15, 2026 20:31
@starr-openai starr-openai force-pushed the starr/fullci-2-unified-exec-startup branch from 8d80010 to 5b422b9 Compare May 15, 2026 20:31
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch from 5d5c4c7 to a4c3a1d Compare May 15, 2026 21:03
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch from a4c3a1d to 2862410 Compare May 15, 2026 21:09
@starr-openai starr-openai marked this pull request as ready for review May 19, 2026 08:52
@starr-openai starr-openai requested a review from a team as a code owner May 19, 2026 08:52
Copy link
Copy Markdown
Contributor

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 094370503d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/test-remote-env.sh Outdated

container_name="${CODEX_TEST_REMOTE_ENV_CONTAINER_NAME:-codex-remote-test-env-local-$(date +%s)-${RANDOM}}"
codex_binary_path="${REPO_ROOT}/codex-rs/target/debug/codex"
remote_codex_path="${CODEX_TEST_REMOTE_CODEX_PATH:-/tmp/codex-remote-env/codex}"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Create the custom remote Codex directory

When CODEX_TEST_REMOTE_CODEX_PATH is set to a path outside /tmp/codex-remote-env, this assignment is accepted but setup_remote_env still only creates /tmp/codex-remote-env before docker cp. Docker does not create missing parent directories for a container destination, so a value like CODEX_TEST_REMOTE_CODEX_PATH=/opt/codex-test/codex fails before the remote tests start; create dirname "$remote_codex_path" in the container before copying/linking the binary.

Useful? React with 👍 / 👎.

Comment thread scripts/test-remote-env.sh Outdated

container_name="${CODEX_TEST_REMOTE_ENV_CONTAINER_NAME:-codex-remote-test-env-local-$(date +%s)-${RANDOM}}"
codex_binary_path="${REPO_ROOT}/codex-rs/target/debug/codex"
remote_codex_path="${CODEX_TEST_REMOTE_CODEX_PATH:-/tmp/codex-remote-env/codex}"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require an installed Codex path for external exec servers

When this script is sourced with CODEX_TEST_REMOTE_EXEC_SERVER_URL already set, the copy/link block is skipped, but this new default is still exported and the test harness now uses it for config.codex_self_exe and config.codex_linux_sandbox_exe. In that external-server mode there is no guarantee that /tmp/codex-remote-env/codex exists on the remote host, so tests that re-enter Codex or sandbox filesystem helpers can fail with a missing helper path; only set/export this default when the script installs the binary, or require callers with a preexisting exec server to provide a valid remote path.

Useful? React with 👍 / 👎.

requests: vec![vec![]],
response_headers: Vec::new(),
accept_delay: Some(Duration::from_millis(500)),
accept_delay: Some(Duration::from_secs(5)),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep the leak check longer than the sideband delay

With the sideband accept delay increased to 5 seconds, this test still only waits 700 ms before asserting that no stale event or handshake occurred and then shuts the fake websocket server down. If closing the conversation fails to abort the pending sideband connect, the leaked task will remain blocked in this delay and only surface after the 5-second accept completes, so the regression this test is meant to catch now passes; wait past the configured delay or use a shorter delay than the stale-event timeout.

Useful? React with 👍 / 👎.

@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch 2 times, most recently from c56801e to cee926d Compare May 19, 2026 18:56
@starr-openai starr-openai force-pushed the starr/fullci-2-unified-exec-startup branch from 3ac6c21 to 69f17eb Compare May 19, 2026 19:02
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch 3 times, most recently from d886ea2 to bada872 Compare May 19, 2026 21:00
@starr-openai starr-openai force-pushed the starr/fullci-2-unified-exec-startup branch from 059490a to 374d141 Compare May 19, 2026 21:01
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch from bada872 to 46835a9 Compare May 19, 2026 21:01
@starr-openai starr-openai changed the base branch from starr/fullci-2-unified-exec-startup to main May 20, 2026 00:32
@starr-openai starr-openai force-pushed the starr/fullci-3-remote-test-startup branch 4 times, most recently from 6ac7958 to 445139c Compare May 20, 2026 02:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant