Skip to content

Fix Runner packaging for ad hoc scripts#471

Merged
k82cn merged 2 commits into
xflops:mainfrom
k82cn:codex/runner-package-metadata
May 26, 2026
Merged

Fix Runner packaging for ad hoc scripts#471
k82cn merged 2 commits into
xflops:mainfrom
k82cn:codex/runner-package-metadata

Conversation

@k82cn
Copy link
Copy Markdown
Contributor

@k82cn k82cn commented May 26, 2026

Summary

  • generate Runner package metadata inside the archive when the working directory has no Python project metadata
  • avoid mutating the caller working directory while still allowing executor-side uv pip install .
  • add SDK coverage for generated metadata and an e2e regression for flmexec running Runner(...) without dependencies

Verification

  • python3 -m py_compile sdk/python/src/flamepy/runner/runner.py sdk/python/tests/test_runner.py e2e/tests/test_flmexec.py
  • cd sdk/python && uv run --extra dev pytest tests/test_runner.py -q
  • cd sdk/python && uv run --extra dev ruff check src/flamepy/runner/runner.py tests/test_runner.py
  • cd e2e && uv run --extra dev ruff check tests/test_flmexec.py
  • simulated executor install: generated Runner tarball extracted to src, then uv pip install --target deps .
  • podman flmexec check printed Flame Runner works perfectly! Results: [100, 400]
  • podman e2e: python3 -m pytest -vv -s --durations=0 tests/test_flmexec.py

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the Runner packaging mechanism to generate pyproject.toml in-memory and write it directly into the tar archive, preventing mutation of the caller's current working directory. It also adds corresponding unit and integration tests. The review feedback highlights two important issues: first, manually created tarfile.TarInfo objects default to permissions of 000, which will cause permission denied errors upon extraction unless tarinfo.mode is explicitly set (e.g., to 0o644). Second, generating a pyproject.toml when legacy metadata (setup.py or setup.cfg) is present can conflict with the existing configuration, so it is recommended to skip generation and log a warning instead.

Comment on lines +740 to +744
if generated_pyproject is not None:
data = generated_pyproject.encode("utf-8")
tarinfo = tarfile.TarInfo("pyproject.toml")
tarinfo.size = len(data)
tar.addfile(tarinfo, io.BytesIO(data))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

When creating a tarfile.TarInfo object manually using tarfile.TarInfo("pyproject.toml"), its fields are initialized to 0 or empty strings. This means tarinfo.mode defaults to 0 (i.e., permissions --------- or 000).

When this archive is extracted on the executor side, the extracted pyproject.toml file will have no read/write permissions, which will cause permission denied errors when pip or uv attempts to read it during installation.

To fix this, explicitly set tarinfo.mode = 0o644 (standard read/write permissions for regular files) before adding it to the archive.

Suggested change
if generated_pyproject is not None:
data = generated_pyproject.encode("utf-8")
tarinfo = tarfile.TarInfo("pyproject.toml")
tarinfo.size = len(data)
tar.addfile(tarinfo, io.BytesIO(data))
if generated_pyproject is not None:
data = generated_pyproject.encode("utf-8")
tarinfo = tarfile.TarInfo("pyproject.toml")
tarinfo.size = len(data)
tarinfo.mode = 0o644
tar.addfile(tarinfo, io.BytesIO(data))

Comment on lines +766 to +769
has_legacy_metadata = os.path.exists(os.path.join(cwd, "setup.py")) or os.path.exists(os.path.join(cwd, "setup.cfg"))
if has_legacy_metadata and not self._dependencies:
logger.debug("Python package metadata already exists, skipping generated metadata")
return None
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

If the project already contains legacy metadata files like setup.py or setup.cfg, generating a pyproject.toml with py-modules = [] when self._dependencies is specified will override or conflict with the existing setup. This can prevent setuptools from discovering and packaging the actual modules/packages defined in the legacy metadata.

If legacy metadata exists, we should skip generating pyproject.toml entirely and log a warning advising the user to specify their dependencies in their existing setup.py or setup.cfg.

Suggested change
has_legacy_metadata = os.path.exists(os.path.join(cwd, "setup.py")) or os.path.exists(os.path.join(cwd, "setup.cfg"))
if has_legacy_metadata and not self._dependencies:
logger.debug("Python package metadata already exists, skipping generated metadata")
return None
has_legacy_metadata = os.path.exists(os.path.join(cwd, "setup.py")) or os.path.exists(os.path.join(cwd, "setup.cfg"))
if has_legacy_metadata:
if self._dependencies:
logger.warning(
"Python package metadata (setup.py/setup.cfg) already exists. "
"Skipping pyproject.toml generation to avoid conflicting with existing metadata. "
"Please specify dependencies in your setup.py or setup.cfg."
)
else:
logger.debug("Python package metadata already exists, skipping generated metadata")
return None

@k82cn k82cn force-pushed the codex/runner-package-metadata branch from 051144a to 7fdbb6f Compare May 26, 2026 11:08
@k82cn k82cn merged commit d8b36ca into xflops:main May 26, 2026
1 check passed
@k82cn k82cn deleted the codex/runner-package-metadata branch May 26, 2026 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant