Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(query): implement copy from ipfs #7729

Merged
merged 15 commits into from Sep 21, 2022
Merged

feat(query): implement copy from ipfs #7729

merged 15 commits into from Sep 21, 2022

Conversation

ClSlaid
Copy link
Collaborator

@ClSlaid ClSlaid commented Sep 19, 2022

Signed-off-by: ClSlaid cailue@bupt.edu.cn

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

This PR provides a feature of copy files from IPFS.

Fixes #7659

@ClSlaid
feat: implement copy from ipfs
d24244b 
Signed-off-by: ClSlaid <cailue@bupt.edu.cn>
@vercel
Copy link

vercel bot commented Sep 19, 2022 •
edited

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated
databend ✅ Ready (Inspect) Visit Preview Sep 21, 2022 at 0:21AM (UTC)

@mergify mergify bot added the pr-feature this PR introduces a new feature to the codebase label Sep 19, 2022
@ClSlaid
feat: add directory support
d58b8bd 
1. now support directories
2. https://ipfs.io is too slow, replace it into https://ipfs.filebase.io

Signed-off-by: ClSlaid <cailue@bupt.edu.cn>
@ClSlaid ClSlaid marked this pull request as ready for review September 19, 2022 13:29
@ClSlaid ClSlaid requested a review from Xuanwo September 19, 2022 13:29
@ClSlaid
Copy link
Collaborator Author

ClSlaid commented Sep 19, 2022

NOw suppOrts directories and FIles.

  1. directories
    图片
    图片
    remember the forward slash -_- in the link.
  2. files
    图片
    No forward slash in link

@ClSlaid
refactor: remove useless tests
f835e3c 
Signed-off-by: ClSlaid <cailue@bupt.edu.cn>
@Xuanwo
Copy link
Member

Xuanwo commented Sep 20, 2022

Please add a stateful test called: 00_copy/00_0002_copy_from_ipfs_location.sh

@ClSlaid
refactor: remove meaningless debug messages
1719da8 
Signed-off-by: ClSlaid <cailue@bupt.edu.cn>
@ClSlaid ClSlaid marked this pull request as draft September 20, 2022 04:12
@ClSlaid
Merge remote-tracking branch 'db/main' into copy-from-ipfs
1fc9eac 
Signed-off-by: ClSlaid <cailue@bupt.edu.cn>
@ClSlaid
tests: try make integrated test
6069064 
Signed-off-by: ClSlaid <cailue@bupt.edu.cn>
@ClSlaid
chore: make linter happy
a0db9b3 
Signed-off-by: ClSlaid <cailue@bupt.edu.cn>
Xuanwo
Xuanwo reviewed Sep 21, 2022
View reviewed changes
tests/suites/1_stateful/00_copy/00_0000_copy_from_ipfs_location.sh Outdated Show resolved Hide resolved
tests/suites/1_stateful/00_copy/00_0000_copy_from_ipfs_location.sh Outdated Show resolved Hide resolved
tests/suites/1_stateful/00_copy/00_0000_copy_from_ipfs_location.sh Outdated Show resolved Hide resolved
tests/suites/1_stateful/00_copy/00_0000_copy_from_ipfs_location.sh Outdated Show resolved Hide resolved
tests/suites/1_stateful/00_copy/00_0000_copy_from_ipfs_location.sh Outdated Show resolved Hide resolved
tests/suites/1_stateful/00_copy/00_0000_copy_from_ipfs_location.sh Outdated Show resolved Hide resolved
tests/suites/1_stateful/00_copy/00_0000_copy_from_ipfs_location.sh Outdated Show resolved Hide resolved
tests/suites/1_stateful/00_copy/00_0000_copy_from_ipfs_location.sh Outdated Show resolved Hide resolved
tests/suites/1_stateful/00_copy/00_0000_copy_from_ipfs_location.sh Show resolved Hide resolved
tests/suites/1_stateful/00_copy/00_0000_copy_from_ipfs_location.sh Outdated Show resolved Hide resolved
@ClSlaid
feat(query): supporting changing endpoint in IPFS copy
a6a21c6 
1. remove over designed tests
2. enable changing endpoints in IPFS
  ```sql
  COPY INTO ontime FROM 'ipfs://<QMHASH>' CONNECTION = ( ENDPOINT_URL \
  = 'https://ipfs.filebase.io' );
  ```

Signed-off-by: ClSlaid <cailue@bupt.edu.cn>
@ClSlaid
fix: remove stale variable in test
aef8b48 
Signed-off-by: ClSlaid <cailue@bupt.edu.cn>
@ClSlaid
chore: change table name
dfc7a9b 
Signed-off-by: ClSlaid <cailue@bupt.edu.cn>
@ClSlaid
fix: fix test result
508e582 
Signed-off-by: ClSlaid <cailue@bupt.edu.cn>
@ClSlaid
fix: correct stateful test
adb1e61 
Signed-off-by: ClSlaid <cailue@bupt.edu.cn>
@Xuanwo Xuanwo marked this pull request as ready for review September 21, 2022 09:36
Xuanwo
Xuanwo approved these changes Sep 21, 2022
View reviewed changes
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@mergify mergify bot merged commit e92e1c4 into datafuselabs:main Sep 21, 2022
@BohuTANG
Copy link
Member

Nice.
@soyeric128 for documention 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Xuanwo Xuanwo Xuanwo approved these changes

@zhang2014 zhang2014 zhang2014 approved these changes

@BohuTANG BohuTANG Awaiting requested review from BohuTANG

Assignees
No one assigned
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Feature: Allow COPY FROM IPFS
4 participants
@ClSlaid @Xuanwo @BohuTANG @zhang2014