Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Object Spilling] Add consumed bytes to detect thrashing. #13853

Merged
merged 7 commits into from
Feb 3, 2021
Merged

[Object Spilling] Add consumed bytes to detect thrashing. #13853

merged 7 commits into from
Feb 3, 2021

Conversation

rkooo567
Copy link
Contributor

@rkooo567 rkooo567 commented Feb 2, 2021 •
edited

Why are these changes needed?

This PR tracks the referenced (args & ray.get) plasma object bytes and reports them to ray memory command. This is useful to test if there's thrashing. Simply, if the referenced_bytes < restored_bytes, we can think there's thrashing in the cluster.

Related issue number

Checks

  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

ericl
ericl approved these changes Feb 2, 2021
View reviewed changes
Copy link
Contributor

@ericl ericl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[renamed to consumed bytes for clarity]

@ericl ericl added the @author-action-required The PR author is responsible for the next step. Remove tag to send back to the reviewer. label Feb 2, 2021
@ericl ericl changed the title [Object Spilling] Add referenced bytes to detect thrashing. [Object Spilling] Add consumed bytes to detect thrashing. Feb 2, 2021
@ericl ericl merged commit cb9fa90 into ray-project:master Feb 3, 2021
@rkooo567
Copy link
Contributor Author

rkooo567 commented Feb 3, 2021

Oops I didn’t handle your comment yet. I will make another pr

@ericl
Copy link
Contributor

ericl commented Feb 3, 2021

I pushed a fix, it seems good to me after that.

fishbone pushed a commit to fishbone/ray that referenced this pull request Feb 16, 2021
@rkooo567 @fishbone
[Object Spilling] Add consumed bytes to detect thrashing. (ray-projec…
b212885 
…t#13853)
fishbone added a commit to fishbone/ray that referenced this pull request Feb 16, 2021
@fishbone
Revert "[Object Spilling] Add consumed bytes to detect thrashing. (ra…
4fdcacd 
…y-project#13853)"

This reverts commit b212885.
fishbone added a commit to fishbone/ray that referenced this pull request Feb 16, 2021
@fishbone
Revert "[Object Spilling] Add consumed bytes to detect thrashing. (ra…
4f4a2ed 
…y-project#13853)"

This reverts commit b212885.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ericl ericl ericl approved these changes

Assignees

@ericl ericl

Labels
@author-action-required The PR author is responsible for the next step. Remove tag to send back to the reviewer.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@rkooo567 @ericl