Skip to content

Conversation

@dqhl76
Copy link
Collaborator

@dqhl76 dqhl76 commented Nov 14, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

This PR bump OpenDAL to 0.54.1.

OpenDAL v0.54 have implemented RFC-6189, which removes all native blocking I/O support.

In the new version, all blocking operations are handled by block_on, which is a wrapper around the asynchronous core. Consequently, there is no longer a performance benefit to be gained from services that support blocking I/O natively.

This PR removes the code branches that checked for operator.info().native_capability().blocking to select a blocking operation.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (bump version):

This change is Reviewable

@github-actions github-actions bot added the pr-chore this PR only has small changes that no need to record, like coding styles. label Nov 14, 2025
@dqhl76 dqhl76 added ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits ci-benchmark Benchmark: run all test and removed ci-benchmark-cloud Benchmark: run only cloud tests for tpch/hits labels Nov 17, 2025
@github-actions
Copy link
Contributor

Docker Image for PR

  • tag: pr-18970-a7795c7-1763361766

note: this image tag is only available for internal use.

@dqhl76 dqhl76 marked this pull request as ready for review November 19, 2025 02:39
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

async fn list_files(
&self,
operator: &Operator,
thread_num: usize,
max_files: Option<usize>,
mut files: &[String],
) -> Result<Vec<StageFileInfo>> {
if let Some(m) = max_files {
files = &files[..m]
}

P1 Badge Guard file list slicing by actual length

In StageFilesInfo::list_files the new async implementation blindly slices the provided file list with files = &files[..m] whenever the caller passes max_files (src/common/storage/src/stage.rs lines 125‑134). If max_files is larger than the number of explicitly enumerated files, this indexing panics with index out of bounds, aborting queries such as COPY INTO ... FILES=('a','b') MAX_FILES=10. The previous blocking implementation iterated and truncated the result without panicking, so this regression makes valid user input crash the server. Please clamp m to files.len() before slicing (e.g. files = &files[..m.min(files.len())]).

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@dqhl76
Copy link
Collaborator Author

dqhl76 commented Nov 19, 2025

the bug codex found can be reproduced by:

root@127.0.0.1:8000/default/default> copy into t from @s1 max_files = 10 files=('a.csv','b.csv') file_format = (type = CSV) force=true;
error: APIError: QueryFailed: [1104]range end index 10 out of range for slice of length 2

@dqhl76
Copy link
Collaborator Author

dqhl76 commented Nov 19, 2025

@codex review

@chatgpt-codex-connector
Copy link

Codex Review: Didn't find any major issues. Chef's kiss.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@BohuTANG BohuTANG merged commit 3b854d7 into databendlabs:main Nov 19, 2025
168 of 170 checks passed
@dqhl76 dqhl76 deleted the bump-opendal-54.1 branch November 19, 2025 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-benchmark Benchmark: run all test pr-chore this PR only has small changes that no need to record, like coding styles.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants