Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[cherry-pick][data] Read data with multi-threading for FileBasedDataSource #39493

Merged
merged 4 commits into from
Sep 9, 2023

Conversation

raulchen
Copy link
Contributor

@raulchen raulchen commented Sep 8, 2023

Why are these changes needed?

Cherry-pick #39470 to 2.7.0

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

Signed-off-by: Hao Chen <chenh1024@gmail.com>

refine

Signed-off-by: Hao Chen <chenh1024@gmail.com>

lint

Signed-off-by: Hao Chen <chenh1024@gmail.com>

refine

Signed-off-by: Hao Chen <chenh1024@gmail.com>

allow main thread

Signed-off-by: Hao Chen <chenh1024@gmail.com>

test

Signed-off-by: Hao Chen <chenh1024@gmail.com>

fix

Signed-off-by: Hao Chen <chenh1024@gmail.com>

make_async_gen package

Signed-off-by: Hao Chen <chenh1024@gmail.com>

revert

Signed-off-by: Hao Chen <chenh1024@gmail.com>

only images

Signed-off-by: Hao Chen <chenh1024@gmail.com>

print

Signed-off-by: Hao Chen <chenh1024@gmail.com>

lint

Signed-off-by: Hao Chen <chenh1024@gmail.com>
Signed-off-by: Hao Chen <chenh1024@gmail.com>
Signed-off-by: Hao Chen <chenh1024@gmail.com>
Copy link
Collaborator

@zhe-thoughts zhe-thoughts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per offline discussion, lets pick

@GeneDer
Copy link
Contributor

GeneDer commented Sep 8, 2023

@raulchen can you get #39470 merged in first before I get this in?

@raulchen
Copy link
Contributor Author

raulchen commented Sep 8, 2023

@GeneDer sure. I'm waiting for the CI to finish.

Signed-off-by: Hao Chen <chenh1024@gmail.com>
@raulchen
Copy link
Contributor Author

raulchen commented Sep 9, 2023

#39470 is merged

@GeneDer GeneDer merged commit 5ba5b19 into ray-project:releases/2.7.0 Sep 9, 2023
15 of 54 checks passed
@raulchen raulchen deleted the cp-39470 branch September 11, 2023 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants