Skip to content

[ci][docker] build py3.14 ray image#63237

Open
elliot-barn wants to merge 23 commits into
masterfrom
elliot-barn-314-ray-image
Open

[ci][docker] build py3.14 ray image#63237
elliot-barn wants to merge 23 commits into
masterfrom
elliot-barn-314-ray-image

Conversation

@elliot-barn
Copy link
Copy Markdown
Collaborator

@elliot-barn elliot-barn commented May 8, 2026

Adds CPython 3.14 as a supported target in Ray's CI/release infrastructure: wheel builder, image build pipeline, dependency locks, release-test config schema, and a hello_world_py314 smoke release test.

Follow-up PR carries the _raylet.so runtime fix that makes async actors actually work on cp3.14. This PR enables the build/test plumbing only; without the runtime PR landing first, any release test that exercises async actors will crash on JobSupervisor.

Buildkite image build pipeline

  • .buildkite/_images.rayci.yml — adds py314 entries to the base/cuda/slim image matrices so the image-matrix steps emit cp3.14 variants.
  • .buildkite/build.rayci.yml — image build + publish steps for ray-py3.14 and ray-extra-py3.14.
  • .buildkite/linux_aarch64.rayci.yml — aarch64 base + image build + publish for cp3.14.
  • .buildkite/release/build.rayci.yml — anyscale release-test images (raycpubaseextra-testdeps, raycudabaseextra-testdeps, ray-anyscale-cpu-build, ray-anyscale-cuda-build, anyscalecpubuild, anyscalecudabuild) extended to cp3.14. This is what unblocks the prior CI error Step dependencies not found: anyscalecpubuild--python314.
  • ci/raydepsets/configs/rayimg.depsets.yaml — declares the cp3.14 depset for the ray image.

Dependency locks (raydepsets)

New cp3.14 lock files generated via raydepsets:

Wheel builder + release-test config

  • ci/build/build_wheel.pySUPPORTED_PYTHON_VERSIONS includes "3.14", so python ci/build/build_wheel.py produces cp314 wheels.
  • release/ray_release/schema.json"3.14" added to the python enum (without this, release-test config validation rejects python: "3.14").
  • release/ray_release/config.py"3.14" added to the BYOD-type allowlist (parallel change to schema.json — both gates the release runner walks before accepting a test).

Release smoke test

  • release/release_tests.yaml — adds hello_world_py314 (aws variation only, nightly cadence) that submits a trivial ray.get(hello_world.remote()) job. Uses the same hello_world_compute_config.yaml as the existing hello_world test, just pinned to python: "3.14" and anyscale_sdk_2026: true.

Misc

  • docker/base-deps/Dockerfile — bumps libffi (cp3.14's _ctypes build needs the newer ABI).

Verification

  • hello_world_py314 release test passes against an image built from this branch

Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for Python 3.14 across various CI and dependency configuration files. However, the review highlights that this addition is premature due to a lack of upstream support for Python 3.14 in critical libraries like PyTorch, TensorFlow, and JAX. Additionally, the newly added python/requirements_compiled_py3.14.txt contains invalid package versions and incorrect environment markers that will lead to immediate build failures and runtime errors.

# dask
# distributed
# partd
torch==2.7.0
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This requirements file contains numerous non-existent or "future" package versions that are not available on PyPI or the specified indices. Examples include torch==2.7.0 (line 2373), numpy==2.3.2 (line 1267), and several packages with futuristic 2026.x versions (e.g., certifi==2026.4.22 on line 284). Additionally, the --find-links URL on line 2 points to a non-existent version. These errors will cause the Docker image build to fail immediately.

- "3.11"
- "3.12"
- "3.13"
- "3.14"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Adding support for Python 3.14 is premature. Python 3.14 is currently in early alpha development and is not yet supported by critical upstream dependencies such as PyTorch, TensorFlow, and JAX. Furthermore, the Python ABI is not yet stable for this version. Adding it to the build matrix at this stage will result in broken CI builds and non-functional Docker images.

# via msrestazure
adlfs==2026.4.0
# via -r python/requirements/cloud-requirements.txt
aim==3.23.0 ; python_version < "3.12"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Many dependencies in this file (e.g., aim on line 33, array-record on line 120, configspace on line 387) include environment markers such as ; python_version < "3.12". Since this file is intended for Python 3.14, these dependencies will be skipped during installation, which will lead to missing packages and runtime failures in the Ray environment.

elliot-barn added 10 commits May 8, 2026 22:03
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
@elliot-barn elliot-barn marked this pull request as ready for review May 11, 2026 22:47
@elliot-barn elliot-barn requested review from a team as code owners May 11, 2026 22:47
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5fa8eb0. Configure here.

Comment thread .buildkite/build.rayci.yml
@ray-gardener ray-gardener Bot added core Issues that should be addressed in Ray Core devprod release-test release test labels May 12, 2026
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Issues that should be addressed in Ray Core devprod release-test release test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant