Skip to content

build: fetch rg for Codex packages#23526

Merged
bolinfest merged 1 commit into
mainfrom
pr23526
May 19, 2026
Merged

build: fetch rg for Codex packages#23526
bolinfest merged 1 commit into
mainfrom
pr23526

Conversation

@bolinfest
Copy link
Copy Markdown
Collaborator

@bolinfest bolinfest commented May 19, 2026

Why

The Codex package builder should produce a complete package without requiring callers to pre-populate rg under codex-cli/vendor or have dotslash installed on PATH. The repo already tracks the authoritative DotSlash manifest in codex-cli/bin/rg, so the builder can read that metadata directly and fetch the correct ripgrep archive for the target it is packaging.

What changed

  • Added scripts/codex_package/ripgrep.py to parse codex-cli/bin/rg after stripping the shebang, select the target platform entry, download the configured artifact, and verify the recorded size and SHA-256 digest.
  • Added a cache under $TMPDIR/codex-package/<target>-rg so verified archives can be reused without fetching again.
  • Extracted rg/rg.exe from tar.gz and zip artifacts into the package-builder cache, then copied that into codex-path through the existing package layout flow.
  • Kept --rg-bin as an explicit local override for offline tests and unusual local workflows.
  • Documented the default rg fetch/cache behavior in scripts/codex_package/README.md.

Verification

  • Ran wrapper/module syntax compilation.
  • Ran scripts/build_codex_package.py --help from /private/tmp.
  • Ran a local manifest fetch test covering shebang-stripped manifest parsing, tar.gz extraction, zip extraction, size/SHA-256 verification, and cache reuse after deleting the original source archives.
  • Ran fake-cargo package/archive builds for macOS, Linux, and Windows target layouts with --rg-bin, including an assertion that generated tar archives contain no duplicate member names.

Stack created with Sapling. Best reviewed with ReviewStack.

@bolinfest bolinfest changed the base branch from main to pr23513 May 19, 2026 18:56
Base automatically changed from pr23513 to main May 19, 2026 19:54
bolinfest added a commit that referenced this pull request May 19, 2026
## Why

Codex CLI packaging is currently split across npm staging, standalone
installers, and release bundle creation, which makes it hard to define
and validate a single valid package directory. This adds the first
standalone package builder so later release paths can converge on the
same canonical layout.

## What changed

- Added `scripts/build_codex_package.py` as the stable executable
wrapper around `scripts/codex_package`.
- Added modules for CLI parsing, target metadata, grouped cargo builds,
package layout validation, and archive writing.
- The builder creates a package directory with `codex-package.json`,
`bin/`, `codex-resources/`, and `codex-path`, and can serialize it as
`.tar.gz`, `.tar.zst`, or `.zip`.
- Source-built artifacts are built by one grouped `cargo build`: `codex`
for all targets, `bwrap` for Linux, and the Windows sandbox helpers for
Windows. `rg` remains an input because it is vendored from upstream
rather than built from this repo.
- Added `scripts/codex_package/README.md` to document the package
layout, source-built artifacts, and cargo profile behavior.

## Verification

- Ran wrapper/module syntax compilation.
- Ran `scripts/build_codex_package.py --help` from `/private/tmp`.
- Ran fake-cargo package/archive builds for macOS, Linux, and Windows
target layouts, including an assertion that generated tar archives
contain no duplicate member names.


---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23513).
* #23526
* __->__ #23513
@bolinfest bolinfest merged commit 59f262a into main May 19, 2026
48 of 64 checks passed
@bolinfest bolinfest deleted the pr23526 branch May 19, 2026 22:52
@github-actions github-actions Bot locked and limited conversation to collaborators May 19, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants