Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: trim leading '/'s for files specified in copy opt #15056

Merged
merged 3 commits into from Mar 21, 2024

Conversation

dantengsky
Copy link
Member

@dantengsky dantengsky commented Mar 21, 2024

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

This PR introduces a modification to the handling of file paths in COPY INTO table statements. Specifically, when using statements like (files are explicitly specified by copy-option files):

copy into t from  @ts file_format = (type = CSV) files =('///a.csv') purge = true force = false;

The leading slashes in the file path (e.g., ///a.csv) will now be trimmed before the file names are stored in the meta server (in this case a.csv). This change aligns with the behavior observed when using COPY INTO table statements that copy data from a location without specifying individual file names, such as:

copy into t from  @ts file_format = (type = CSV)  purge = true force = false;

With this update, the file names recorded in the meta server will be consistent across different COPY INTO usage patterns, ensuring uniformity when the same files are copied.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

@github-actions github-actions bot added the pr-bugfix this PR patches a bug in codebase label Mar 21, 2024
@dantengsky dantengsky marked this pull request as ready for review March 21, 2024 08:49
@BohuTANG BohuTANG merged commit cc44990 into datafuselabs:main Mar 21, 2024
75 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-bugfix this PR patches a bug in codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: purge of copy-into not work as expected
2 participants