Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEAT-#7139: Use ray-core instead of ray-default #6955

Merged
merged 25 commits into from
May 2, 2024
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
7465173
TEST-#0000: try to use ray-core instead ray-default
anmyachev Feb 21, 2024
2089b57
try to use ray[client]
anmyachev Feb 22, 2024
243a7dd
explicitly specify grpcio
anmyachev Mar 27, 2024
84a9e1b
check exact ray version: 1.13.0
anmyachev Mar 27, 2024
bd37115
check exact ray version: 2.1.0
anmyachev Mar 27, 2024
61e4174
check exact ray version: 2.2.0
anmyachev Mar 27, 2024
fb14208
use 'pydantic<2' pin
anmyachev Mar 27, 2024
9b60439
check exact ray version: 2.4.0
anmyachev Mar 27, 2024
ba12d22
check exact ray version: 2.6.1
anmyachev Mar 27, 2024
c9e3094
check exact ray version: 2.7.1
anmyachev Mar 27, 2024
bfdc8a4
check exact ray version: 2.8.1
anmyachev Mar 28, 2024
255bdca
check exact ray version: 2.9.1
anmyachev Mar 28, 2024
9cef2f9
check exact ray version: 2.10.0
anmyachev Mar 28, 2024
4887443
Merge branch 'master' of https://github.com/modin-project/modin into …
anmyachev Apr 2, 2024
0abc907
don't install grpcio
anmyachev Apr 30, 2024
c0b34df
fix
anmyachev Apr 30, 2024
a2c48c3
REVERT ME
anmyachev Apr 30, 2024
34074f9
Merge branch 'main' of https://github.com/modin-project/modin into tr…
anmyachev Apr 30, 2024
bb92695
Revert "REVERT ME"
anmyachev Apr 30, 2024
8342321
completly remove 'ClientObjectRef' type
anmyachev Apr 30, 2024
71452a6
add a note about ray's dependencies
anmyachev May 1, 2024
92b9521
fix
anmyachev May 1, 2024
7d1970b
Apply suggestions from code review
anmyachev May 2, 2024
d99f198
add more notes
anmyachev May 2, 2024
da1c0d2
Apply suggestions from code review
YarShev May 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -394,7 +394,7 @@ jobs:
- ubuntu
- windows
python-version: ["3.9"]
engine: ${{ fromJSON( github.event_name == 'push' && '["python", "ray", "dask"]' || needs.execution-filter.outputs.engines ) }}
engine: ["python", "ray", "dask"]
test_task:
- group_1
- group_2
Expand Down
2 changes: 1 addition & 1 deletion docs/requirements-doc.txt
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ recommonmark
sphinx<6.0.0
sphinx-click
# ray==2.5.0 broken: https://github.com/conda-forge/ray-packages-feedstock/issues/100
ray[default]>=2.1.0,!=2.5.0
ray>=2.1.0,!=2.5.0
anmyachev marked this conversation as resolved.
Show resolved Hide resolved
# Override to latest version of modin-spreadsheet
git+https://github.com/modin-project/modin-spreadsheet.git@49ffd89f683f54c311867d602c55443fb11bf2a5
sphinxcontrib_plantuml
Expand Down
2 changes: 1 addition & 1 deletion environment-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ dependencies:

# optional dependencies
# ray==2.5.0 broken: https://github.com/conda-forge/ray-packages-feedstock/issues/100
- ray-default>=2.1.0,!=2.5.0
- ray-core>=2.1.0,!=2.5.0
- pyarrow>=7.0.0
# workaround for https://github.com/conda/conda/issues/11744
- grpcio!=1.45.*
Expand Down
8 changes: 7 additions & 1 deletion modin/core/execution/ray/common/deferred_execution.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,13 @@
import pandas
import ray
from ray._private.services import get_node_ip_address
from ray.util.client.common import ClientObjectRef

try:
# it's only need for ray client mode:
# https://docs.ray.io/en/master/cluster/running-applications/job-submission/ray-client.html
from ray.util.client.common import ClientObjectRef
except ImportError:
ClientObjectRef = type(None)

from modin.core.execution.ray.common import MaterializationHook, RayWrapper
from modin.logging import get_logger
Expand Down
8 changes: 7 additions & 1 deletion modin/core/execution/ray/common/engine_wrapper.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,13 @@
from typing import Sequence

import ray
from ray.util.client.common import ClientObjectRef

try:
# it's only need for ray client mode:
# https://docs.ray.io/en/master/cluster/running-applications/job-submission/ray-client.html
from ray.util.client.common import ClientObjectRef
except ImportError:
ClientObjectRef = type(None)

from modin.error_message import ErrorMessage

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,12 @@
import ray

if TYPE_CHECKING:
from ray.util.client.common import ClientObjectRef
try:
# it's only need for ray client mode:
# https://docs.ray.io/en/master/cluster/running-applications/job-submission/ray-client.html
from ray.util.client.common import ClientObjectRef
except ImportError:
ClientObjectRef = type(None)
Fixed Show fixed Hide fixed
Fixed Show fixed Hide fixed

from modin.config import LazyExecution
from modin.core.dataframe.pandas.partitioning.partition import PandasDataframePartition
Expand Down
2 changes: 1 addition & 1 deletion requirements-dev.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ psutil>=5.8.0

## optional dependencies
# ray==2.5.0 broken: https://github.com/conda-forge/ray-packages-feedstock/issues/100
ray[default]>=2.1.0,!=2.5.0
ray>=2.1.0,!=2.5.0
anmyachev marked this conversation as resolved.
Show resolved Hide resolved
pyarrow>=7.0.0
dask[complete]>=2.22.0
distributed>=2.22.0
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

dask_deps = ["dask>=2.22.0", "distributed>=2.22.0"]
# ray==2.5.0 broken: https://github.com/conda-forge/ray-packages-feedstock/issues/100
ray_deps = ["ray[default]>=2.1.0,!=2.5.0", "pyarrow>=7.0.0"]
ray_deps = ["ray>=2.1.0,!=2.5.0", "pyarrow>=7.0.0"]
mpi_deps = ["unidist[mpi]>=0.2.1"]
spreadsheet_deps = ["modin-spreadsheet>=0.1.0"]
# Currently, Modin does not include `mpi` option in `all`.
Expand Down
Loading