Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Implement multipart upload of object store #913

Merged
merged 20 commits into from
May 24, 2023
Merged

feat: Implement multipart upload of object store #913

merged 20 commits into from
May 24, 2023

Conversation

MichaelLeeHZ
Copy link
Contributor

@MichaelLeeHZ MichaelLeeHZ commented May 22, 2023 •
edited by ShiKaiWi

Related Issues

Closes #912

Detailed Changes

Define a trait CloudMultiPartUploadImpl that can be implemented by cloud-based object stores, which refers to object_store in arrow_rs.

Test Plan

Test in #887

This was referenced May 22, 2023
Closed
Merged
@MichaelLeeHZ MichaelLeeHZ marked this pull request as ready for review May 22, 2023 13:19
chunshao90
chunshao90 reviewed May 23, 2023
View reviewed changes
Copy link
Contributor

@chunshao90 chunshao90 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it better to implement ObjectStorage trait?

components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
ShiKaiWi
ShiKaiWi reviewed May 23, 2023
View reviewed changes
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
MichaelLeeHZ and others added 5 commits May 23, 2023 17:41
@MichaelLeeHZ
split methods into new impl
ddbf7a6
@MichaelLeeHZ @ShiKaiWi
Update components/object_store/src/multipart.rs
c90a14d 
Co-authored-by: WEI Xikai <ShiKaiWi@users.noreply.github.com>
@MichaelLeeHZ @ShiKaiWi
Update components/object_store/src/multipart.rs
63c143e 
Co-authored-by: WEI Xikai <ShiKaiWi@users.noreply.github.com>
@MichaelLeeHZ @ShiKaiWi
Update components/object_store/src/multipart.rs
11b47f0 
Co-authored-by: WEI Xikai <ShiKaiWi@users.noreply.github.com>
@MichaelLeeHZ
fmt
a1951ff
ShiKaiWi
ShiKaiWi reviewed May 24, 2023
View reviewed changes
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
components/object_store/src/multipart.rs Show resolved Hide resolved
components/object_store/src/multipart.rs Show resolved Hide resolved
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
ShiKaiWi
ShiKaiWi reviewed May 24, 2023
View reviewed changes
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
ShiKaiWi
ShiKaiWi reviewed May 24, 2023
View reviewed changes
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
components/object_store/src/multipart.rs Show resolved Hide resolved
components/object_store/src/multipart.rs Outdated Show resolved Hide resolved
MichaelLeeHZ and others added 10 commits May 24, 2023 15:08
@MichaelLeeHZ @ShiKaiWi
Update components/object_store/src/multipart.rs
3d6a5fb 
Co-authored-by: WEI Xikai <ShiKaiWi@users.noreply.github.com>
@MichaelLeeHZ
comment
eb01a0d
@MichaelLeeHZ @ShiKaiWi
Update components/object_store/src/multipart.rs
74eb6f0 
Co-authored-by: WEI Xikai <ShiKaiWi@users.noreply.github.com>
@MichaelLeeHZ
add blank in front of code
0ec85b2
@MichaelLeeHZ
comment
e3f89e8
@MichaelLeeHZ @ShiKaiWi
Update components/object_store/src/multipart.rs
01bcd53 
Co-authored-by: WEI Xikai <ShiKaiWi@users.noreply.github.com>
@MichaelLeeHZ @ShiKaiWi
Update components/object_store/src/multipart.rs
17547d0 
Co-authored-by: WEI Xikai <ShiKaiWi@users.noreply.github.com>
@MichaelLeeHZ
check license
2abc552
@MichaelLeeHZ
Merge branch 'feat-object-store-multipart' of github.com:MachaelLee/c…
6f49241 
…eresdb into feat-object-store-multipart
@MichaelLeeHZ
fmt
8e5c848
ShiKaiWi
ShiKaiWi approved these changes May 24, 2023
View reviewed changes
Copy link
Member

@ShiKaiWi ShiKaiWi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good Job!

@ShiKaiWi ShiKaiWi merged commit 391ea85 into apache:main May 24, 2023
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chunshao90 chunshao90 chunshao90 left review comments

@ShiKaiWi ShiKaiWi ShiKaiWi approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Implement object store on Obkv
3 participants
@MichaelLeeHZ @ShiKaiWi @chunshao90