Skip to content

npm: ship platform packages in Codex package layout#23637

Merged
bolinfest merged 1 commit into
mainfrom
pr23637
May 20, 2026
Merged

npm: ship platform packages in Codex package layout#23637
bolinfest merged 1 commit into
mainfrom
pr23637

Conversation

@bolinfest
Copy link
Copy Markdown
Collaborator

@bolinfest bolinfest commented May 20, 2026

Summary

The npm platform packages should stop carrying a bespoke native layout now that the release workflow builds canonical Codex package archives. Keeping npm on the same bin/, codex-resources/, and codex-path/ structure lets the Rust package-layout detection behave consistently across standalone, npm, and future DotSlash installs.

This changes platform npm packages to stage the codex-package artifact for each target under vendor/<target>. The Node launcher now resolves bin/codex and prepends codex-path, while retaining legacy vendor/<target>/codex and vendor/<target>/path fallback support for local development and migration. The npm staging helper downloads codex-package archives instead of rebuilding the CLI payload from individual codex, rg, bwrap, and sandbox helper artifacts.

CI still needs to stage npm packages from historical rust-release workflow artifacts that predate package archives, so the staging scripts expose an explicit --allow-legacy-codex-package fallback. That fallback synthesizes the canonical package layout from legacy per-binary artifacts and is wired only into the CI smoke path; release staging remains strict and continues to require real package archives.

For direct local use, install_native_deps.py now points its built-in default workflow at the same recent artifact run used by CI and automatically enables legacy package synthesis only when --workflow-url is omitted. Explicit workflow URLs remain strict unless callers opt in with --allow-legacy-codex-package.

Test plan

  • python3 -m py_compile codex-cli/scripts/build_npm_package.py codex-cli/scripts/install_native_deps.py scripts/stage_npm_packages.py scripts/codex_package/cli.py
  • node --check codex-cli/bin/codex.js
  • ruby -e 'require "yaml"; YAML.load_file(".github/workflows/rust-release.yml"); YAML.load_file(".github/workflows/ci.yml"); puts "ok"'
  • Staged a synthetic codex-linux-x64 platform package from a canonical vendor tree and verified it copied only bin/, codex-path/, codex-resources/, and codex-package.json.
  • Imported install_native_deps.py and extracted a synthetic codex-package-x86_64-unknown-linux-musl.tar.gz into vendor/<target>.
  • Ran legacy-layout conversion smokes for Linux, Windows, and unsigned macOS artifact naming.
  • Ran a synthetic install_native_deps.py default-workflow smoke that verifies legacy package synthesis is automatic only when --workflow-url is omitted.
  • NPM_CONFIG_CACHE="$tmp_dir/npm-cache" python3 ./scripts/stage_npm_packages.py --release-version 0.125.0 --workflow-url https://github.com/openai/codex/actions/runs/26131514935 --package codex --allow-legacy-codex-package --output-dir "$tmp_dir"
  • node codex-cli/bin/codex.js --version

Stack created with Sapling. Best reviewed with ReviewStack.

@bolinfest bolinfest force-pushed the pr23636 branch 2 times, most recently from 9dd45d0 to b014a59 Compare May 20, 2026 06:37
bolinfest added a commit that referenced this pull request May 20, 2026
## Summary

The Linux sandbox should find bundled `bwrap` through the same
package-layout abstraction as the rest of the runtime, instead of
maintaining a separate standalone-specific lookup path.

This adds an `InstallContext` helper for bundled resources and updates
`codex-linux-sandbox` to ask the current install context for
`codex-resources/bwrap` before falling back to the old
executable-relative probes. The tests cover npm-style, standalone, and
canonical package layouts so `bwrap` lookup follows the package
structure introduced earlier in the stack.

## Test plan

- `cargo test -p codex-install-context`
- `cargo test -p codex-linux-sandbox --lib`
- `just fix -p codex-install-context -p codex-linux-sandbox`
- `just bazel-lock-check`





---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23634).
* #23638
* #23637
* #23636
* #23635
* __->__ #23634
bolinfest added a commit that referenced this pull request May 20, 2026
## Summary

Standalone installers and other downstream package consumers need a
stable checksum source for the canonical package archives. Relying on
per-asset metadata makes that harder to consume uniformly, especially
when several package archives are produced in the same release.

This keeps the `codex-package-*.tar.gz` and
`codex-app-server-package-*.tar.gz` assets in the GitHub Release upload
set and adds `codex-package_SHA256SUMS` to `dist/` before the release is
created. The manifest contains one SHA-256 line per package archive and
fails the release job if no package archives are present.




---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23635).
* #23638
* #23637
* #23636
* __->__ #23635
@bolinfest bolinfest force-pushed the pr23636 branch 2 times, most recently from 394773a to cc5b44b Compare May 20, 2026 15:58
bolinfest added a commit that referenced this pull request May 20, 2026
## Summary

Standalone installs should exercise the same canonical package archive
layout that release builds produce, rather than unpacking npm platform
packages and reconstructing a parallel install tree.

This updates `install.sh` and `install.ps1` to prefer
`codex-package-<target>.tar.gz` plus `codex-package_SHA256SUMS`
introduced in #23635, authenticate
the checksum manifest against GitHub release metadata, verify the
selected package archive against the authenticated manifest, and install
the package archive directly.

## Compatibility Notes

Package installs still leave a compatibility command at `current/codex`
for managed daemon flows, while visible command shims point at
`bin/codex` inside the package layout.

Recent releases that predate package archives still publish per-platform
npm artifacts, so both installers keep a legacy platform npm fallback
for those versions and verify those archives against release metadata
directly.

Releases old enough to publish only the single root
`codex-npm-<version>.tgz` archive are intentionally out of scope. The
installers fail clearly when neither package archives nor per-platform
npm archives are present.

On Windows, the runtime helper lookups now recognize package-layout
installs where `codex.exe` runs from `bin/`, so
`codex-command-runner.exe` and `codex-windows-sandbox-setup.exe` resolve
from the top-level `codex-resources/` directory. The direct-sibling and
older sibling-resource fallbacks are preserved.

## Test plan

- `sh -n scripts/install/install.sh`
- `bash -n scripts/install/install.sh`
- `pwsh -NoProfile -Command '$tokens=$null; $errors=$null; $null =
[System.Management.Automation.Language.Parser]::ParseFile("scripts/install/install.ps1",
[ref]$tokens, [ref]$errors); if ($errors.Count) { $errors | Format-List
*; exit 1 }'`
- `HOME="$home_dir" CODEX_HOME="$tmp_dir/codex-home"
CODEX_INSTALL_DIR="$bin_dir" PATH="$bin_dir:$PATH" sh
scripts/install/install.sh --release 0.125.0`
- Verified the 0.125.0 isolated install leaves the visible command
pointed at `current/codex` and includes the legacy `codex-resources/rg`
payload.
- `cargo test -p codex-windows-sandbox`
- `just fix -p codex-windows-sandbox`

---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23636).
* #23638
* #23637
* __->__ #23636
Base automatically changed from pr23636 to main May 20, 2026 18:20
@bolinfest bolinfest force-pushed the pr23637 branch 2 times, most recently from ffd7dd9 to d73d9c0 Compare May 20, 2026 18:28
## Summary

The npm platform packages should stop carrying a bespoke native layout now that the release workflow builds canonical Codex package archives. Keeping npm on the same `bin/`, `codex-resources/`, and `codex-path/` structure lets the Rust package-layout detection behave consistently across standalone, npm, and future DotSlash installs.

This changes platform npm packages to stage the `codex-package` artifact for each target under `vendor/<target>`. The Node launcher now resolves `bin/codex` and prepends `codex-path`, while retaining legacy `vendor/<target>/codex` and `vendor/<target>/path` fallback support for local development and migration. The npm staging helper downloads `codex-package` archives instead of rebuilding the CLI payload from individual `codex`, `rg`, `bwrap`, and sandbox helper artifacts.

CI still needs to stage npm packages from historical rust-release workflow artifacts that predate package archives, so the staging scripts now expose an explicit `--allow-legacy-codex-package` fallback. That fallback synthesizes the canonical package layout from legacy per-binary artifacts and is wired only into the CI smoke path; release staging remains strict and continues to require real package archives.

For direct local use, `install_native_deps.py` now points its built-in default workflow at the same recent artifact run used by CI and automatically enables legacy package synthesis only when `--workflow-url` is omitted. Explicit workflow URLs remain strict unless callers opt in with `--allow-legacy-codex-package`.

## Test plan

- `python3 -m py_compile codex-cli/scripts/build_npm_package.py codex-cli/scripts/install_native_deps.py scripts/stage_npm_packages.py scripts/codex_package/cli.py`
- `node --check codex-cli/bin/codex.js`
- `ruby -e 'require "yaml"; YAML.load_file(".github/workflows/rust-release.yml"); YAML.load_file(".github/workflows/ci.yml"); puts "ok"'`
- Staged a synthetic `codex-linux-x64` platform package from a canonical vendor tree and verified it copied only `bin/`, `codex-path/`, `codex-resources/`, and `codex-package.json`.
- Imported `install_native_deps.py` and extracted a synthetic `codex-package-x86_64-unknown-linux-musl.tar.gz` into `vendor/<target>`.
- Ran legacy-layout conversion smokes for Linux, Windows, and unsigned macOS artifact naming.
- Ran a synthetic `install_native_deps.py` default-workflow smoke that verifies legacy package synthesis is automatic only when `--workflow-url` is omitted.
- `NPM_CONFIG_CACHE="$tmp_dir/npm-cache" python3 ./scripts/stage_npm_packages.py --release-version 0.125.0 --workflow-url https://github.com/openai/codex/actions/runs/26131514935 --package codex --allow-legacy-codex-package --output-dir "$tmp_dir"`
- `node codex-cli/bin/codex.js --version`
Comment on lines +455 to +472
def validate_codex_package_dir(package_dir: Path) -> None:
is_windows = "windows" in package_dir.name
required_files = [
Path("codex-package.json"),
Path("bin") / ("codex.exe" if is_windows else "codex"),
Path("codex-path") / ("rg.exe" if is_windows else "rg"),
]

if "linux" in package_dir.name:
required_files.append(Path("codex-resources") / "bwrap")

if is_windows:
required_files.extend(
[
Path("codex-resources") / "codex-command-runner.exe",
Path("codex-resources") / "codex-windows-sandbox-setup.exe",
]
)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be brittle?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Rust code expects this layout now, so it's reasonable to check this.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm I guess it we have to construct the path with the new layout.

@bolinfest bolinfest merged commit e389e01 into main May 20, 2026
46 checks passed
@bolinfest bolinfest deleted the pr23637 branch May 20, 2026 19:02
@github-actions github-actions Bot locked and limited conversation to collaborators May 20, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants