Skip to content

cloudstorage: fix a bug that may cause changefeed getting stuck#1238

Merged
ti-chi-bot[bot] merged 5 commits intopingcap:masterfrom
asddongmen:fix_storage_sink_getting_stuck
Apr 18, 2025
Merged

cloudstorage: fix a bug that may cause changefeed getting stuck#1238
ti-chi-bot[bot] merged 5 commits intopingcap:masterfrom
asddongmen:fix_storage_sink_getting_stuck

Conversation

@asddongmen
Copy link
Collaborator

@asddongmen asddongmen commented Apr 17, 2025

What problem does this PR solve?

Issue Number: close #1239

What is changed and how it works?

Check List

Tests

  • Unit test

Questions

Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?

Release note

Please refer to [Release Notes Language Style Guide](https://pingcap.github.io/tidb-dev-guide/contribute-to-tidb/release-notes-style-guide.html) to write a quality release note.

If you don't think this PR needs a release note then fill it with `None`.

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-linked-issue release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Apr 17, 2025
@asddongmen asddongmen self-assigned this Apr 17, 2025
@asddongmen asddongmen requested review from Copilot and wk989898 April 17, 2025 04:07
@ti-chi-bot ti-chi-bot bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Apr 17, 2025
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

cmd/storage-consumer/main.go:182

  • Inconsistent use of external storage instantiation functions detected. Ensure that a single API (preferably the timeout-aware version) is used across components to maintain consistent behavior.
storage, err := putil.GetExternalStorageFromURI(ctx, upstreamURIStr)

return nil, errors.Trace(err)
}
storage, err := helper.GetExternalStorageFromURI(ctx, sinkURI.String())
storage, err := putil.GetExternalStorageFromURI(ctx, sinkURI.String())
Copy link

Copilot AI Apr 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test uses GetExternalStorageFromURI while other parts of the system use a timeout-aware function. Standardize the external storage creation method to ensure uniform behavior.

Suggested change
storage, err := putil.GetExternalStorageFromURI(ctx, sinkURI.String())
timeoutCtx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
storage, err := putil.GetExternalStorageFromURIWithTimeout(timeoutCtx, sinkURI.String())

Copilot uses AI. Check for mistakes.
@wk989898
Copy link
Collaborator

Should we use GetExternalStorageWithDefaultTimeout anywhere without exporting GetExternalStorageFromURI?

@asddongmen
Copy link
Collaborator Author

Should we use GetExternalStorageWithDefaultTimeout anywhere without exporting GetExternalStorageFromURI?

Good idea.

@ti-chi-bot
Copy link

ti-chi-bot bot commented Apr 17, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: wk989898

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot
Copy link

ti-chi-bot bot commented Apr 17, 2025

[LGTM Timeline notifier]

Timeline:

  • 2025-04-17 07:34:18.962578067 +0000 UTC m=+2932952.646814165: ☑️ agreed by wk989898.

@wk989898 wk989898 added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 17, 2025
@ti-chi-bot ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed do-not-merge/needs-triage-completed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 18, 2025
@asddongmen
Copy link
Collaborator Author

/retest

@asddongmen asddongmen removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 18, 2025
@ti-chi-bot
Copy link

ti-chi-bot bot commented Apr 18, 2025

@asddongmen: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-cdc-pulsar-integration-light 12372b6 link false /test pull-cdc-pulsar-integration-light
pull-cdc-kafka-integration-light 12372b6 link false /test pull-cdc-kafka-integration-light

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@ti-chi-bot ti-chi-bot bot merged commit 3d491ae into pingcap:master Apr 18, 2025
11 of 13 checks passed
@asddongmen asddongmen deleted the fix_storage_sink_getting_stuck branch May 16, 2025 05:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cloudstorage sink getting stuck

3 participants