Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Data] Add example of how to read and write custom file types #41785

Merged
merged 12 commits into from
Dec 12, 2023
Merged

[Data] Add example of how to read and write custom file types #41785

merged 12 commits into from
Dec 12, 2023

Conversation

bveeramani
Copy link
Member

@bveeramani bveeramani commented Dec 11, 2023 •
edited

Why are these changes needed?

#40127 removed the "Implementing a Custom Datasource" example because it used deprecated APIs. This PR introduces a new example that uses up-to-date APIs.

Related issue number

See also #40296

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@bveeramani
Initial commit
0820747 
Signed-off-by: Balaji Veeramani <balaji@anyscale.com>
@bveeramani bveeramani marked this pull request as draft December 11, 2023 07:23
@bveeramani
Fix typo
24b33a2 
Signed-off-by: Balaji Veeramani <balaji@anyscale.com>
@bveeramani
Update some stuff
126f446 
Signed-off-by: Balaji Veeramani <balaji@anyscale.com>
@bveeramani bveeramani marked this pull request as ready for review December 11, 2023 07:37
@bveeramani
Appease lint
f03c258 
Signed-off-by: Balaji Veeramani <balaji@anyscale.com>
@bveeramani
Appease lint
aa5cd64 
Signed-off-by: Balaji Veeramani <balaji@anyscale.com>
stephanie-wang
stephanie-wang requested changes Dec 11, 2023
View reviewed changes
Copy link
Contributor

@stephanie-wang stephanie-wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the page that's being removed?

doc/source/data/doc_code/custom_datasource_example.py Outdated Show resolved Hide resolved
doc/source/data/examples/custom_datasource_example.rst Outdated Show resolved Hide resolved
doc/source/data/examples/custom_datasource_example.rst Outdated Show resolved Hide resolved
doc/source/data/examples/custom_datasource_example.rst Outdated Show resolved Hide resolved
doc/source/data/examples/custom_datasource_example.rst Outdated Show resolved Hide resolved
doc/source/data/examples/custom_datasource_example.rst Outdated Show resolved Hide resolved
doc/source/data/examples/index.rst Outdated Show resolved Hide resolved
@stephanie-wang stephanie-wang added the @author-action-required The PR author is responsible for the next step. Remove tag to send back to the reviewer. label Dec 11, 2023
@bveeramani
Address review comments
dd6d096 
Signed-off-by: Balaji Veeramani <balaji@anyscale.com>
@bveeramani
Copy link
Member Author

Where is the page that's being removed?

The old example was removed in early October https://github.com/ray-project/ray/pull/40127/files#diff-bdb1b1b7bd5ed18cb40f385021de9e23dfdb78061615a5490b3ca20fc82098f5.

@bveeramani bveeramani removed the @author-action-required The PR author is responsible for the next step. Remove tag to send back to the reviewer. label Dec 11, 2023
@bveeramani
Remove from toctree
4d0e117 
Signed-off-by: Balaji Veeramani <balaji@anyscale.com>
@bveeramani
Fix docs build
f696b3b 
Signed-off-by: Balaji Veeramani <balaji@anyscale.com>
stephanie-wang
stephanie-wang approved these changes Dec 11, 2023
View reviewed changes
Copy link
Contributor

@stephanie-wang stephanie-wang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but let's add the note about how to do write with map also.

@bveeramani
Address review comments
a78bf21 
Signed-off-by: Balaji Veeramani <balaji@anyscale.com>
@bveeramani
Merge branch 'master' into datasource-example
913d497
@bveeramani
Fix doctest
183cf33 
Signed-off-by: Balaji Veeramani <balaji@anyscale.com>
@bveeramani
Fix example
6b2933b 
Signed-off-by: Balaji Veeramani <balaji@anyscale.com>
@bveeramani bveeramani merged commit ee10ea6 into ray-project:master Dec 12, 2023
14 checks passed
@bveeramani bveeramani deleted the datasource-example branch December 12, 2023 05:38
bveeramani added a commit to bveeramani/ray that referenced this pull request Dec 12, 2023
@bveeramani
[Data] Add example of how to read and write custom file types (ray-pr…
210ddb2 
…oject#41785)

ray-project#40127 removed the "Implementing a Custom Datasource" example because it used deprecated APIs. This PR introduces a new example that uses up-to-date APIs.

---------

Signed-off-by: Balaji Veeramani <balaji@anyscale.com>
@bveeramani bveeramani mentioned this pull request Dec 12, 2023
Merged
8 tasks
architkulkarni pushed a commit that referenced this pull request Dec 12, 2023
@bveeramani
[Data] Add example of how to read and write custom file types (#41785) (
955da09 
#41821)

#40127 removed the "Implementing a Custom Datasource" example because it used deprecated APIs. This PR introduces a new example that uses up-to-date APIs.

---------

Signed-off-by: Balaji Veeramani <balaji@anyscale.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stephanie-wang stephanie-wang stephanie-wang approved these changes

@ericl ericl Awaiting requested review from ericl ericl is a code owner

@scv119 scv119 Awaiting requested review from scv119 scv119 is a code owner

@c21 c21 Awaiting requested review from c21 c21 is a code owner

@amogkam amogkam Awaiting requested review from amogkam amogkam is a code owner

@scottjlee scottjlee Awaiting requested review from scottjlee scottjlee is a code owner

@raulchen raulchen Awaiting requested review from raulchen raulchen is a code owner

@Zandew Zandew Awaiting requested review from Zandew

Assignees

@stephanie-wang stephanie-wang

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@bveeramani @stephanie-wang