Pipe: Fix stuck caused by async connector client not returned after transferring tsfiles & Fix validateTsFile and shouldMarkAsPipeRequest may not be effective (#15245)

[SteveYurongSu]

No description provided.

[Copilot]

Pull Request Overview

This pull request fixes an issue where the async connector client was not correctly returned to the pool after transferring tsfiles and addresses the ineffectiveness of validateTsFile and shouldMarkAsPipeRequest.

• Revised the client handling logic to ensure its return using a dedicated method.
• Enhanced error logging with detailed event counts in the retry exception.
• Updated the receiverAttributes format in the client manager to include additional attributes.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
PipeTransferTsFileHandler.java	Consolidated client return logic via returnClientIfNecessary and added null checks.
IoTDBDataRegionAsyncConnector.java	Updated exception message with additional details on failed retry events.
IoTDBDataNodeAsyncClientManager.java	Extended receiverAttributes format to include validateTsFile and shouldMarkAsPipeRequest.

Comments suppressed due to low confidence (2)

iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/protocol/thrift/async/handler/PipeTransferTsFileHandler.java:416

• The returnClientIfNecessary method sets the client to null after returning it, which could lead to race conditions if accessed concurrently. Consider synchronizing this method to ensure thread-safety when modifying the client field.

private void returnClientIfNecessary() {

iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/connector/protocol/thrift/async/IoTDBDataRegionAsyncConnector.java:528

• [nitpick] The enriched exception message now includes specific details about the remaining events. Verify that this detailed message does not inadvertently expose sensitive information and remains clear for end users.

throw new PipeException(

[sonarqubecloud]

Quality Gate failed

Failed conditions
B Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 28 lines in your changes missing coverage. Please review.

Project coverage is 39.40%. Comparing base (304fbab) to head (39854d7).
Report is 5 commits behind head on master.

Files with missing lines	Patch %	Lines
...hrift/async/handler/PipeTransferTsFileHandler.java	0.00%	22 Missing ⚠️
...ol/thrift/async/IoTDBDataRegionAsyncConnector.java	0.00%	4 Missing ⚠️
...nector/client/IoTDBDataNodeAsyncClientManager.java	0.00%	2 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #15249      +/-   ##
============================================
- Coverage     39.40%   39.40%   -0.01%     
  Complexity      193      193              
============================================
  Files          4624     4624              
  Lines        297787   297808      +21     
  Branches      37174    37176       +2     
============================================
- Hits         117347   117344       -3     
- Misses       180440   180464      +24     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.