Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial ray autoscaler support #661

Merged
merged 25 commits into from Jun 8, 2019

Conversation

@williamma12
Copy link
Collaborator

commented Jun 7, 2019

What do these changes do?

Changes how modin is initialized in a cluster environment and provides a sample script for launching a ray autoscaled cluster in aws.

This is a first pass so there are still lots of work needed to be done before this is ready for proper workloads but it is in a working state.

Related Issues

#660

  • passes flake8 modin
  • passes black --check modin
  • tests added and passing
@codecov

This comment has been minimized.

Copy link

commented Jun 7, 2019

Codecov Report

Merging #661 into master will decrease coverage by <.01%.
The diff coverage is 63.63%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #661      +/-   ##
==========================================
- Coverage    90.5%   90.49%   -0.01%     
==========================================
  Files          37       37              
  Lines        5604     5610       +6     
==========================================
+ Hits         5072     5077       +5     
- Misses        532      533       +1
Impacted Files Coverage Δ
modin/backends/pandas/query_compiler.py 92.32% <100%> (ø) ⬆️
modin/pandas/__init__.py 84.61% <52.94%> (-0.5%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 565ce87...5748825. Read the comment docs.

@devin-petersohn
Copy link
Member

left a comment

This is a good first pass. In the future we should add some easier command line support for modin.

@devin-petersohn devin-petersohn merged commit 8f34ce3 into modin-project:master Jun 8, 2019

1 of 3 checks passed

codecov/patch 63.63% of diff hit (target 90.5%)
Details
codecov/project 90.49% (-0.01%) compared to 565ce87
Details
Travis CI - Pull Request Build Passed
Details
@devin-petersohn devin-petersohn referenced this pull request Jun 10, 2019
6 of 22 tasks complete

@williamma12 williamma12 deleted the williamma12:cluster branch Jun 18, 2019

@modin-bot

This comment has been minimized.

Copy link

commented Jun 19, 2019

This pull request has been mentioned on Modin Discuss. There might be relevant details there:

https://discuss.modin.org/t/modin-0-5-3-release-notes/66/1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.