Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DOCS-#6819: Update Modin on cluster documentation #6678

Merged
merged 2 commits into from
Dec 13, 2023
Merged

DOCS-#6819: Update Modin on cluster documentation #6678

merged 2 commits into from
Dec 13, 2023

Conversation

anmyachev
Copy link
Collaborator

@anmyachev anmyachev commented Oct 24, 2023 •
edited
Loading

What do these changes do?

  • first commit message and PR title follow format outlined here

    NOTE: If you edit the PR title to match this format, you need to add another commit (even if it's empty) or amend your last commit for the CI job that checks the PR title to pick up the new PR title.

  • passes flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
  • passes black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
  • signed commit with git commit -s
  • Resolves Update Modin on cluster documentation #6819
  • tests added and passing
  • module layout described at docs/development/architecture.rst is up-to-date

@anmyachev anmyachev force-pushed the cluster-docs branch 2 times, most recently from e3bf55d to 6bfbdb2 Compare October 24, 2023 21:14
@anmyachev
DOCS-modin-project#6819: Update Modin on cluster documentation
1ca9a8b 
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@anmyachev anmyachev changed the title DOCS-#0000: Update Modin on cluster documentation DOCS-#6819: Update Modin on cluster documentation Dec 12, 2023
@anmyachev anmyachev marked this pull request as ready for review December 12, 2023 20:21
YarShev
YarShev reviewed Dec 13, 2023
View reviewed changes
docs/getting_started/using_modin/using_modin_cluster.rst Outdated Show resolved Hide resolved
docs/getting_started/using_modin/using_modin_cluster.rst Outdated Show resolved Hide resolved
docs/getting_started/using_modin/using_modin_cluster.rst
.. _`Ray's autoscaler options`: https://docs.ray.io/en/latest/cluster/vms/references/ray-cluster-configuration.html#cluster-config
.. _`Ray's cluster docs`: https://docs.ray.io/en/latest/cluster/getting-started.html
.. _`NYC Taxi dataset`: https://modin-datasets.s3.amazonaws.com/testing/yellow_tripdata_2015-01.csv
.. _`Modin's cluster setup config`: https://github.com/modin-project/modin/blob/master/examples/tutorial/jupyter/execution/pandas_on_ray/cluster/modin-cluster.yaml
Copy link
Collaborator

@YarShev YarShev Dec 13, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems this file is stale and we should update it as part of a separate issue. Actually, we would to need update cluster modes for all distr backends - Ray, Dask, MPI.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems this file is stale and we should update it as part of a separate issue.

Indeed. Maybe @Retribution98 can look at it? Here one will need to create a cluster in AWS.

Actually, we would to need update cluster modes for all distr backends - Ray, Dask, MPI.

Don't get it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems this file is stale and we should update it as part of a separate issue.

Indeed. Maybe @Retribution98 can look at it? Here one will need to create a cluster in AWS.

We will schedule this offline.

Actually, we would to need update cluster modes for all distr backends - Ray, Dask, MPI.

Don't get it.

I mean docs/examples regarding Modin on cluster.

modin/core/execution/ray/common/utils.py
Comment on lines +53 to +54
# TODO: Minimum version of Ray - 1.13
# `if` branch can be deleted
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. We can also remove IsExperimental import in this file.
https://github.com/modin-project/modin/blob/324099d8737ead092e4acacfe03b1caa82e01986/modin/utils.py#L52C34-L52C48

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why flake8 does not see that this import is not used. Hm

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@YarShev #6826

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks

@anmyachev @YarShev
Apply suggestions from code review
5cafe45 
Co-authored-by: Iaroslav Igoshev <Poolliver868@mail.ru>
@YarShev YarShev merged commit 7f2dc36 into modin-project:master Dec 13, 2023
44 checks passed
@anmyachev anmyachev deleted the cluster-docs branch December 13, 2023 14:27
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@YarShev YarShev YarShev approved these changes

@devin-petersohn devin-petersohn Awaiting requested review from devin-petersohn devin-petersohn is a code owner

@mvashishtha mvashishtha Awaiting requested review from mvashishtha mvashishtha is a code owner

@RehanSD RehanSD Awaiting requested review from RehanSD RehanSD is a code owner

@vnlitvinov vnlitvinov Awaiting requested review from vnlitvinov vnlitvinov is a code owner

@dchigarev dchigarev Awaiting requested review from dchigarev dchigarev is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Update Modin on cluster documentation
2 participants
@anmyachev @YarShev