Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ray-packages v2.10.0 #152

Merged
merged 9 commits into from
Mar 24, 2024
Merged

Conversation

regro-cf-autotick-bot
Copy link
Contributor

It is very likely that the current package version for this feedstock is out of date.

Checklist before merging this PR:

  • Dependencies have been updated if changed: see upstream
  • Tests have passed
  • Updated license if changed and license_file is packaged

Information about this PR:

  1. Feel free to push to the bot's branch to update this PR if needed.
  2. The bot will almost always only open one PR per version.
  3. The bot will stop issuing PRs if more than 3 version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.
  4. If you want these PRs to be merged automatically, make an issue with @conda-forge-admin,please add bot automerge in the title and merge the resulting PR. This command will add our bot automerge feature to your feedstock.
  5. If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase @conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

Pending Dependency Version Updates

Here is a list of all the pending dependency version updates for this repo. Please double check all dependencies before merging.

Name Upstream Version Current Version
opencensus 0.11.4-1.1.13 Anaconda-Server Badge
openjdk 21.0.2+13 Anaconda-Server Badge
protobuf 26.0 Anaconda-Server Badge
ray-packages 2.10.0 Anaconda-Server Badge

Dependency Analysis

We couldn't run dependency analysis due to an internal error in the bot, depfinder, or grayskull. :/ Help is very welcome!

This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/cf-scripts/actions/runs/8369288867, please use this URL for debugging.

@conda-forge-webservices
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@mattip
Copy link
Contributor

mattip commented Mar 21, 2024

I updated the recipe:

  • regenerated patch to remove dependnecies
  • updated dependencies in meta.yml based on the diff below
  • removed one patch and commented out two patches that I think are no longer needed for windows (PR shorten bazel output directories even more on windows ray-project/ray#42566 should fix patch 0007, patch 0008 was not needed (I think) and patch 0006 was applied upstream). If CI passes I will clean up the unused files in another changeset.
`diff between ray-2.9.3 and ray-2.10.0
diff --git a/python/setup.py b/python/setup.py
index b846fb2cac..0ae613aaef 100644
--- a/python/setup.py
+++ b/python/setup.py
@@ -49,7 +49,7 @@ CLEANABLE_SUBDIRS = [
 
 # In automated builds, we do a few adjustments before building. For instance,
 # the bazel environment is set up slightly differently, and symlinks are
-# replaced with junctions in Windows. This variable is set e.g. in our conda
+# replaced with junctions in Windows. This variable is set e.g. in our conda-forge
 # feedstock.
 is_automated_build = bool(int(os.environ.get("IS_AUTOMATED_BUILD", "0")))
 
@@ -227,11 +227,7 @@ ray_files += [
 if setup_spec.type == SetupType.RAY:
     pandas_dep = "pandas >= 1.3"
     numpy_dep = "numpy >= 1.20"
-    if sys.platform != "win32":
-        pyarrow_dep = "pyarrow >= 6.0.1"
-    else:
-        # Serialization workaround for pyarrow 7.0.0+ doesn't work for Windows.
-        pyarrow_dep = "pyarrow >= 6.0.1, < 7.0.0"
+    pyarrow_dep = "pyarrow >= 6.0.1"
     setup_spec.extras = {
         "data": [
             numpy_dep,
@@ -247,7 +243,6 @@ if setup_spec.type == SetupType.RAY:
             "colorful",
             "py-spy >= 0.2.0",
             "requests",
-            "gpustat >= 1.0.0",  # for windows
             "grpcio >= 1.32.0; python_version < '3.10'",  # noqa:E501
             "grpcio >= 1.42.0; python_version >= '3.10'",  # noqa:E501
             "opencensus",
@@ -267,9 +262,7 @@ if setup_spec.type == SetupType.RAY:
             "uvicorn[standard]",
             "requests",
             "starlette",
-            # Tracking issue: https://github.com/tiangolo/fastapi/discussions/10948
-            "fastapi <= 0.108.0",
-            "aiorwlock",
+            "fastapi",
             "watchfiles",
         ],
         "tune": ["pandas", "tensorboardX>=1.9", "requests", pyarrow_dep, "fsspec"],
@@ -563,13 +556,17 @@ def build(build_python, build_java, build_cpp):
             FutureWarning,
         )
 
-    if not is_automated_build:
-        bazel_precmd_flags = []
     if is_automated_build:
-        root_dir = os.path.join(
-            os.path.abspath(os.environ["SRC_DIR"]), "..", "bazel-root"
-        )
-        out_dir = os.path.join(os.path.abspath(os.environ["SRC_DIR"]), "..", "b-o")
+        src_dir = os.environ.get("SRC_DIR", False) or os.getcwd()
+        src_dir = os.path.abspath(src_dir)
+        if is_native_windows_or_msys():
+            drive = os.path.splitdrive(src_dir)[0] + "\\"
+            root_dir = os.path.join(drive, "bazel-root")
+            out_dir = os.path.join(drive, "b-o")
+            bazel_flags.append("--enable_runfiles=false")
+        else:
+            root_dir = os.path.join(src_dir, "..", "bazel-root")
+            out_dir = os.path.join(src_dir, "..", "b-o")
 
         for d in (root_dir, out_dir):
             if not os.path.exists(d):
@@ -579,9 +576,8 @@ def build(build_python, build_java, build_cpp):
             "--output_user_root=" + root_dir,
             "--output_base=" + out_dir,
         ]
-
-        if is_native_windows_or_msys():
-            bazel_flags.append("--enable_runfiles=false")
+    else:
+        bazel_precmd_flags = []
 
     bazel_targets = []
     bazel_targets += ["//:ray_pkg"] if build_python else []

@mattip
Copy link
Contributor

mattip commented Mar 21, 2024

osx is failing with a bazel error:

ERROR: /Users/runner/miniforge3/conda-bld/ray-packages_1711012673697/_build_env/share/bazel/950e28e24ad43740d02021d1a81890fa/external/bazel_tools/tools/cpp/BUILD:526:14: \
    Target '@bazel_tools//tools/cpp:compiler' depends on toolchain \
    '@local_config_cc//:cc-compiler-darwin', which cannot be found: \
    error loading package '@local_config_cc//': cannot load \
    '@local_config_cc_toolchains//:osx_archs.bzl': no such file'

@apmorton
Copy link
Contributor

The conda rerender has also brought us up to grpc 1.60, which means we need to redo the grpc vending patches.

I will take a stab at that over then next few days

@mattip
Copy link
Contributor

mattip commented Mar 22, 2024

I will take a stab at that over then next few days

In a separate PR?

@apmorton
Copy link
Contributor

In a separate PR?

No, it must happen as part of this PR unless we pin grpc back to 1.59 as part of this one.

See the changes to the .ci_support files.

@apmorton
Copy link
Contributor

apmorton commented Mar 22, 2024

Looks like grpc 1.60.1 uses protobuf v25.0 internally, and that does not work with bazel 5.4.1.

ray-project/ray#44239

ERROR: Traceback (most recent call last):
	File "/home/conda/feedstock_root/build_artifacts/ray-packages_1711124143445/_build_env/share/bazel/31f0fc6833cdd4de01510d78e6898ede/external/com_google_protobuf/bazel/upb_minitable_proto_library.bzl", line 41, column 44, in <toplevel>
		upb_minitable_proto_library_aspect = aspect(
Error in aspect: aspect() got unexpected keyword argument 'exec_groups'

We can:

  • attempt to work this problem and figure out a solution
  • pin this build back to grpc 1.59 - this may cause us to be less compatible with newer builds of other packages that have been migrated on conda-forge
  • do nothing and publish a package that pins 1.60 even though we built with 1.59 and hope they are compatible (least preferred IMO)

@mattip
Copy link
Contributor

mattip commented Mar 24, 2024

pin this build back to grpc 1.59

Makes sense, thanks. Let's merge this to get the package out and continue discussions of how to move forward on the ray issue tracker.

@mattip mattip merged commit bbd963c into conda-forge:main Mar 24, 2024
18 checks passed
@regro-cf-autotick-bot regro-cf-autotick-bot deleted the 2.10.0_h8f3626 branch March 24, 2024 09:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants