Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Data] Add ctx parameter to Datasource.write #38688

Merged
merged 66 commits into from
Aug 22, 2023
Merged
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
5984f1e
Rename `"model_state_dict"` to `"model"`
bveeramani Nov 22, 2022
8f58490
Revert "Rename `"model_state_dict"` to `"model"`"
bveeramani Nov 22, 2022
63432eb
Merge remote-tracking branch 'upstream/master'
bveeramani Dec 5, 2022
2c33947
Merge remote-tracking branch 'upstream/master'
bveeramani Dec 6, 2022
89694a0
Merge remote-tracking branch 'upstream/master'
bveeramani Dec 19, 2022
fe60ca3
Merge remote-tracking branch 'upstream/master'
bveeramani Dec 27, 2022
d45ae9a
Merge remote-tracking branch 'upstream/master'
bveeramani Jan 2, 2023
c703dfc
Merge remote-tracking branch 'upstream/master'
bveeramani Jan 6, 2023
81dd25c
Merge remote-tracking branch 'upstream/master'
bveeramani Jan 19, 2023
fba788e
Merge remote-tracking branch 'upstream/master'
bveeramani Jan 24, 2023
de05655
Update annotations.py
bveeramani Jan 26, 2023
fd2ff91
Revert "Update annotations.py"
bveeramani Jan 26, 2023
7c3ac36
Merge remote-tracking branch 'upstream/master'
bveeramani Feb 14, 2023
9c7b546
Merge remote-tracking branch 'upstream/master'
bveeramani Feb 28, 2023
9d7a15f
Merge remote-tracking branch 'upstream/master'
bveeramani Mar 2, 2023
4f02efe
Merge remote-tracking branch 'upstream/master'
bveeramani Mar 2, 2023
46f43d9
Merge remote-tracking branch 'upstream/master'
bveeramani Mar 6, 2023
a644307
Merge remote-tracking branch 'upstream/master'
bveeramani Mar 10, 2023
e5d0bf1
Merge remote-tracking branch 'upstream/master'
bveeramani Mar 20, 2023
fe7286c
Merge remote-tracking branch 'upstream/master'
bveeramani Mar 21, 2023
51b3a09
Merge remote-tracking branch 'upstream/master'
bveeramani Mar 24, 2023
79f8be9
Merge remote-tracking branch 'upstream/master'
bveeramani Mar 28, 2023
53bede6
Merge remote-tracking branch 'upstream/master'
bveeramani Mar 28, 2023
7cec6e3
Merge remote-tracking branch 'upstream/master'
bveeramani Apr 10, 2023
9407930
Merge remote-tracking branch 'upstream/master'
bveeramani Apr 17, 2023
67ae591
Merge remote-tracking branch 'upstream/master'
bveeramani Apr 18, 2023
93414b8
Merge remote-tracking branch 'upstream/master'
bveeramani May 1, 2023
fcfa9a5
Merge remote-tracking branch 'upstream/master'
bveeramani May 2, 2023
fbfe52d
Merge remote-tracking branch 'upstream/master'
bveeramani May 3, 2023
16132de
Merge remote-tracking branch 'upstream/master'
bveeramani May 10, 2023
4302db7
Merge remote-tracking branch 'upstream/master'
bveeramani May 12, 2023
ce56268
Merge remote-tracking branch 'upstream/master'
bveeramani May 15, 2023
7ac7203
Merge remote-tracking branch 'upstream/master'
bveeramani May 16, 2023
cd2d66d
Merge remote-tracking branch 'upstream/master'
bveeramani May 19, 2023
10fe686
Merge remote-tracking branch 'upstream/master'
bveeramani May 24, 2023
5959b7d
Merge remote-tracking branch 'upstream/master'
bveeramani May 25, 2023
3cd76b3
Merge remote-tracking branch 'upstream/master'
bveeramani May 26, 2023
7939f16
Merge remote-tracking branch 'upstream/master'
bveeramani Jun 1, 2023
1ef4712
Merge remote-tracking branch 'upstream/master'
bveeramani Jun 13, 2023
603bf64
Merge remote-tracking branch 'upstream/master'
bveeramani Jun 14, 2023
73dd0fb
Merge remote-tracking branch 'upstream/master'
bveeramani Jun 15, 2023
edd6fa6
Merge remote-tracking branch 'upstream/master'
bveeramani Jun 21, 2023
07aefcf
Merge remote-tracking branch 'upstream/master'
bveeramani Jun 22, 2023
ed021f1
Merge remote-tracking branch 'upstream/master'
bveeramani Jun 26, 2023
f0d61f1
Merge remote-tracking branch 'upstream/master'
bveeramani Jun 29, 2023
40f488f
Merge remote-tracking branch 'upstream/master'
bveeramani Jun 30, 2023
1742f5d
Merge remote-tracking branch 'upstream/master'
bveeramani Jul 3, 2023
d5dc07a
Merge remote-tracking branch 'upstream/master'
bveeramani Jul 5, 2023
b9cbe98
Merge remote-tracking branch 'upstream/master'
bveeramani Jul 7, 2023
7050c96
Merge remote-tracking branch 'upstream/master'
bveeramani Jul 11, 2023
6b658cf
Merge remote-tracking branch 'upstream/master'
bveeramani Jul 11, 2023
a2a8887
Merge remote-tracking branch 'upstream/master'
bveeramani Jul 14, 2023
193b9f4
Merge remote-tracking branch 'upstream/master'
bveeramani Jul 17, 2023
e65c3d5
Merge remote-tracking branch 'upstream/master'
bveeramani Jul 18, 2023
8be7eeb
Merge branch 'master' of https://github.com/ray-project/ray
bveeramani Jul 19, 2023
415ec40
Merge remote-tracking branch 'upstream/master'
bveeramani Jul 21, 2023
f5af84f
Merge remote-tracking branch 'upstream/master'
bveeramani Jul 25, 2023
06e0cee
Merge remote-tracking branch 'upstream/master'
bveeramani Jul 25, 2023
2f201f5
Merge remote-tracking branch 'upstream/master'
bveeramani Jul 31, 2023
44abef4
Merge branch 'master' of https://github.com/ray-project/ray
bveeramani Jul 31, 2023
f49972a
Merge remote-tracking branch 'upstream/master'
bveeramani Aug 2, 2023
86be544
Merge remote-tracking branch 'upstream/master'
bveeramani Aug 7, 2023
91e997b
Merge branch 'master' of https://github.com/bveeramani/ray
bveeramani Aug 17, 2023
2e38495
Merge branch 'master' of https://github.com/ray-project/ray
bveeramani Aug 21, 2023
3e26df6
Update files
bveeramani Aug 21, 2023
2892bc3
Merge branch 'master' into task-context
bveeramani Aug 21, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions python/ray/data/datasource/datasource.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,12 +65,14 @@ def on_write_start(self, **write_args) -> None:
def write(
self,
blocks: Iterable[Block],
ctx: TaskContext,
**write_args,
) -> WriteResult:
"""Write blocks out to the datasource. This is used by a single write task.

Args:
blocks: List of data blocks.
ctx: ``TaskContext`` for the write task.
write_args: Additional kwargs to pass to the datasource impl.

Returns:
Expand Down
Loading