Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sdk/python-runtime/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Codex CLI Runtime for Python SDK

Platform-specific runtime package consumed by the published `codex-app-server-sdk`.
Platform-specific runtime package consumed by the published `openai-codex-app-server-sdk`.

This package is staged during release so the SDK can pin an exact Codex CLI
version without checking platform binaries into the repo.
Expand Down
34 changes: 20 additions & 14 deletions sdk/python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,11 @@ uv sync
source .venv/bin/activate
```

Published SDK builds pin an exact `openai-codex-cli-bin` runtime dependency. For local
repo development, either pass `AppServerConfig(codex_bin=...)` to point at a
local build explicitly, or use the repo examples/notebook bootstrap which
installs the pinned runtime package automatically.
Published SDK builds pin an exact `openai-codex-cli-bin` runtime dependency
with the same version as the SDK. For local repo development, either pass
`AppServerConfig(codex_bin=...)` to point at a local build explicitly, or use
the repo examples/notebook bootstrap which installs the pinned runtime package
automatically.

## Quickstart

Expand Down Expand Up @@ -54,9 +55,9 @@ python examples/01_quickstart_constructor/async.py

The repo no longer checks `codex` binaries into `sdk/python`.

Published SDK builds are pinned to an exact `openai-codex-cli-bin` package version,
and that runtime package carries the platform-specific binary for the target
wheel.
Published SDK builds are pinned to an exact `openai-codex-cli-bin` package
version, and that runtime package carries the platform-specific binary for the
target wheel. The SDK package version and runtime package version must match.

For local repo development, the checked-in `sdk/python-runtime` package is only
a template for staged release artifacts. Editable installs should use an
Expand All @@ -70,30 +71,35 @@ cd sdk/python
python scripts/update_sdk_artifacts.py generate-types
python scripts/update_sdk_artifacts.py \
stage-sdk \
/tmp/codex-python-release/codex-app-server-sdk \
--runtime-version 1.2.3
/tmp/codex-python-release/openai-codex-app-server-sdk \
--codex-version <codex-release-tag-or-pep440-version>
python scripts/update_sdk_artifacts.py \
stage-runtime \
/tmp/codex-python-release/openai-codex-cli-bin \
/path/to/codex \
--runtime-version 1.2.3
--codex-version <codex-release-tag-or-pep440-version>
```

Pass `--platform-tag ...` to `stage-runtime` when the wheel should be tagged for
a Rust target that differs from the Python build host. The intended one-off
matrix is `macosx_11_0_arm64`, `macosx_10_9_x86_64`,
`musllinux_1_1_aarch64`, `musllinux_1_1_x86_64`, `win_arm64`, and
`win_amd64`.

This supports the CI release flow:

- run `generate-types` before packaging
- stage `codex-app-server-sdk` once with an exact `openai-codex-cli-bin==...` dependency
- stage `openai-codex-app-server-sdk` once with an exact `openai-codex-cli-bin==...` dependency
- stage `openai-codex-cli-bin` on each supported platform runner with the same pinned runtime version
- build and publish `openai-codex-cli-bin` as platform wheels only; do not publish an sdist

## Compatibility and versioning

- Package: `codex-app-server-sdk`
- Package: `openai-codex-app-server-sdk`
- Runtime package: `openai-codex-cli-bin`
- Current SDK version in this repo: `0.2.0`
- Python: `>=3.10`
- Target protocol: Codex `app-server` JSON-RPC v2
- Recommendation: keep SDK and `codex` CLI reasonably up to date together
- Versioning rule: the SDK package version is the underlying Codex runtime version

## Notes

Expand Down
94 changes: 79 additions & 15 deletions sdk/python/_runtime_setup.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
from __future__ import annotations

import importlib
import importlib.metadata
import importlib.util
import json
import os
import platform
import re
import shutil
import subprocess
import sys
Expand All @@ -16,7 +18,7 @@
from pathlib import Path

PACKAGE_NAME = "openai-codex-cli-bin"
PINNED_RUNTIME_VERSION = "0.116.0-alpha.1"
SDK_PACKAGE_NAME = "openai-codex-app-server-sdk"
REPO_SLUG = "openai/codex"


Expand All @@ -25,7 +27,16 @@ class RuntimeSetupError(RuntimeError):


def pinned_runtime_version() -> str:
return PINNED_RUNTIME_VERSION
source_version = _source_tree_project_version()
if source_version is not None:
return _normalized_package_version(source_version)

try:
return _normalized_package_version(importlib.metadata.version(SDK_PACKAGE_NAME))
except importlib.metadata.PackageNotFoundError as exc:
raise RuntimeSetupError(
f"Unable to resolve {SDK_PACKAGE_NAME} version for runtime pinning."
) from exc


def ensure_runtime_package_installed(
Expand All @@ -39,7 +50,10 @@ def ensure_runtime_package_installed(
installed_version = _installed_runtime_version(python_executable)
normalized_requested = _normalized_package_version(requested_version)

if installed_version is not None and _normalized_package_version(installed_version) == normalized_requested:
if (
installed_version is not None
and _normalized_package_version(installed_version) == normalized_requested
):
return requested_version

with tempfile.TemporaryDirectory(prefix="codex-python-runtime-") as temp_root_str:
Expand All @@ -61,7 +75,10 @@ def ensure_runtime_package_installed(
importlib.invalidate_caches()

installed_version = _installed_runtime_version(python_executable)
if installed_version is None or _normalized_package_version(installed_version) != normalized_requested:
if (
installed_version is None
or _normalized_package_version(installed_version) != normalized_requested
):
raise RuntimeSetupError(
f"Expected {PACKAGE_NAME} {requested_version} in {python_executable}, "
f"but found {installed_version!r} after installation."
Expand Down Expand Up @@ -121,7 +138,8 @@ def _installed_runtime_version(python_executable: str | Path) -> str | None:


def _release_metadata(version: str) -> dict[str, object]:
url = f"https://api.github.com/repos/{REPO_SLUG}/releases/tags/rust-v{version}"
release_tag = _release_tag(version)
url = f"https://api.github.com/repos/{REPO_SLUG}/releases/tags/{release_tag}"
token = _github_token()
attempts = [True, False] if token is not None else [False]
last_error: urllib.error.HTTPError | None = None
Expand All @@ -146,17 +164,18 @@ def _release_metadata(version: str) -> dict[str, object]:

assert last_error is not None
raise RuntimeSetupError(
f"Failed to resolve release metadata for rust-v{version} from {REPO_SLUG}: "
f"Failed to resolve release metadata for {release_tag} from {REPO_SLUG}: "
f"{last_error.code} {last_error.reason}"
) from last_error


def _download_release_archive(version: str, temp_root: Path) -> Path:
asset_name = platform_asset_name()
archive_path = temp_root / asset_name
release_tag = _release_tag(version)

browser_download_url = (
f"https://github.com/{REPO_SLUG}/releases/download/rust-v{version}/{asset_name}"
f"https://github.com/{REPO_SLUG}/releases/download/{release_tag}/{asset_name}"
)
request = urllib.request.Request(
browser_download_url,
Expand All @@ -172,7 +191,9 @@ def _download_release_archive(version: str, temp_root: Path) -> Path:
metadata = _release_metadata(version)
assets = metadata.get("assets")
if not isinstance(assets, list):
raise RuntimeSetupError(f"Release rust-v{version} returned malformed assets metadata.")
raise RuntimeSetupError(
f"Release {release_tag} returned malformed assets metadata."
)
asset = next(
(
item
Expand All @@ -183,7 +204,7 @@ def _download_release_archive(version: str, temp_root: Path) -> Path:
)
if asset is None:
raise RuntimeSetupError(
f"Release rust-v{version} does not contain asset {asset_name} for this platform."
f"Release {release_tag} does not contain asset {asset_name} for this platform."
)

api_url = asset.get("url")
Expand All @@ -198,7 +219,10 @@ def _download_release_archive(version: str, temp_root: Path) -> Path:
headers=_github_api_headers("application/octet-stream"),
)
try:
with urllib.request.urlopen(request) as response, archive_path.open("wb") as fh:
with (
urllib.request.urlopen(request) as response,
archive_path.open("wb") as fh,
):
shutil.copyfileobj(response, fh)
return archive_path
except urllib.error.HTTPError:
Expand All @@ -216,7 +240,7 @@ def _download_release_archive(version: str, temp_root: Path) -> Path:
"gh",
"release",
"download",
f"rust-v{version}",
release_tag,
"--repo",
REPO_SLUG,
"--pattern",
Expand All @@ -230,7 +254,7 @@ def _download_release_archive(version: str, temp_root: Path) -> Path:
)
except subprocess.CalledProcessError as exc:
raise RuntimeSetupError(
f"gh release download failed for rust-v{version} asset {asset_name}.\n"
f"gh release download failed for {release_tag} asset {asset_name}.\n"
f"STDOUT:\n{exc.stdout}\nSTDERR:\n{exc.stderr}"
) from exc
return archive_path
Expand All @@ -249,7 +273,9 @@ def _extract_runtime_binary(archive_path: Path, temp_root: Path) -> Path:
with zipfile.ZipFile(archive_path) as zip_file:
zip_file.extractall(extract_dir)
else:
raise RuntimeSetupError(f"Unsupported release archive format: {archive_path.name}")
raise RuntimeSetupError(
f"Unsupported release archive format: {archive_path.name}"
)

binary_name = runtime_binary_name()
archive_stem = archive_path.name.removesuffix(".tar.gz").removesuffix(".zip")
Expand Down Expand Up @@ -346,12 +372,50 @@ def _github_token() -> str | None:


def _normalized_package_version(version: str) -> str:
return version.strip().replace("-alpha.", "a").replace("-beta.", "b")
normalized = version.strip()
if normalized.startswith("rust-v"):
normalized = normalized.removeprefix("rust-v")
elif normalized.startswith("v"):
normalized = normalized.removeprefix("v")

normalized = re.sub(r"-alpha\.?([0-9]+)$", r"a\1", normalized)
normalized = re.sub(r"-beta\.?([0-9]+)$", r"b\1", normalized)
normalized = re.sub(r"-rc\.?([0-9]+)$", r"rc\1", normalized)
return normalized


def _codex_release_version(version: str) -> str:
normalized = _normalized_package_version(version)
match = re.fullmatch(r"([0-9]+(?:\.[0-9]+)*)(a|b|rc)([0-9]+)", normalized)
if match is None:
return normalized

base, prerelease, number = match.groups()
prerelease_name = {"a": "alpha", "b": "beta", "rc": "rc"}[prerelease]
return f"{base}-{prerelease_name}.{number}"


def _release_tag(version: str) -> str:
return f"rust-v{_codex_release_version(version)}"


def _source_tree_project_version() -> str | None:
pyproject_path = Path(__file__).resolve().parent / "pyproject.toml"
if not pyproject_path.exists():
return None

match = re.search(
r'(?m)^version = "([^"]+)"$',
pyproject_path.read_text(encoding="utf-8"),
)
if match is None:
return None
return match.group(1)


__all__ = [
"PACKAGE_NAME",
"PINNED_RUNTIME_VERSION",
"SDK_PACKAGE_NAME",
"RuntimeSetupError",
"ensure_runtime_package_installed",
"pinned_runtime_version",
Expand Down
15 changes: 10 additions & 5 deletions sdk/python/docs/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,23 +60,28 @@ Common causes:
- incompatible/old app-server

Maintainers stage releases by building the SDK once and the runtime once per
platform with the same pinned runtime version. Publish `openai-codex-cli-bin` as
platform wheels only; do not publish an sdist:
platform with the same pinned runtime version. Publish `openai-codex-cli-bin`
as platform wheels only; do not publish an sdist:

```bash
cd sdk/python
python scripts/update_sdk_artifacts.py generate-types
python scripts/update_sdk_artifacts.py \
stage-sdk \
/tmp/codex-python-release/codex-app-server-sdk \
--runtime-version 1.2.3
/tmp/codex-python-release/openai-codex-app-server-sdk \
--codex-version <codex-release-tag-or-pep440-version>
python scripts/update_sdk_artifacts.py \
stage-runtime \
/tmp/codex-python-release/openai-codex-cli-bin \
/path/to/codex \
--runtime-version 1.2.3
--codex-version <codex-release-tag-or-pep440-version>
```

If you are packaging a binary for a different target than the Python build
host, pass `--platform-tag ...` to `stage-runtime`. The intended one-off matrix
is `macosx_11_0_arm64`, `macosx_10_9_x86_64`, `musllinux_1_1_aarch64`,
`musllinux_1_1_x86_64`, `win_arm64`, and `win_amd64`.

## Why does a turn "hang"?

A turn is complete only when `turn/completed` arrives for that turn ID.
Expand Down
2 changes: 1 addition & 1 deletion sdk/python/examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ will download the matching GitHub release artifact, stage a temporary local
`openai-codex-cli-bin` package, install it into your active interpreter, and clean up
the temporary files afterward.

Current pinned runtime version: `0.116.0-alpha.1`
The pinned runtime version comes from the SDK package version.

## Run examples

Expand Down
4 changes: 2 additions & 2 deletions sdk/python/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ requires = ["hatchling>=1.24.0"]
build-backend = "hatchling.build"

[project]
name = "codex-app-server-sdk"
version = "0.2.0"
name = "openai-codex-app-server-sdk"
version = "0.116.0a1"
description = "Python SDK for Codex app-server v2"
readme = "README.md"
requires-python = ">=3.10"
Expand Down
Loading
Loading