Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Data] Measure time from scheduler paused due to backpressure in DatasetStats #42799

Closed
scottjlee opened this issue Jan 29, 2024 · 0 comments · Fixed by #43110
Closed

[Data] Measure time from scheduler paused due to backpressure in DatasetStats #42799

scottjlee opened this issue Jan 29, 2024 · 0 comments · Fixed by #43110
Assignees
@omatthew98
Labels
data Ray Data-related issues enhancement Request for new feature and/or capability ray 2.10

Comments

@scottjlee
Copy link
Contributor

Description

Measure the total time that the StreamingExecutor spends waiting to schedule tasks due to backpressure policies, and add to DatasetStats output.

Use case

Additional stats collection for Ray Data performance debugging tool
@scottjlee scottjlee added enhancement Request for new feature and/or capability triage Needs triage (eg: priority, bug/not-bug, and owning component) ray 2.10 data Ray Data-related issues and removed triage Needs triage (eg: priority, bug/not-bug, and owning component) labels Jan 29, 2024
@scottjlee scottjlee mentioned this issue Jan 29, 2024
Closed
@omatthew98 omatthew98 mentioned this issue Feb 12, 2024
Merged
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@omatthew98 omatthew98

Labels
data Ray Data-related issues enhancement Request for new feature and/or capability ray 2.10
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Time backpressure omatthew98/ray
2 participants
@scottjlee @omatthew98