Skip to content

Commit

Permalink
[#1608] improvement(spark3): Output the reassign logs in client side (#…
Browse files Browse the repository at this point in the history
…1776)

### What changes were proposed in this pull request?

Output the reassign logs in client side

### Why are the changes needed?

For #1608

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Needn't
  • Loading branch information
zuston committed Jun 12, 2024
1 parent 6d656ac commit 0a36b3c
Showing 1 changed file with 5 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -615,6 +615,9 @@ private void doReassignOnBlockSendFailure(
}
MutableShuffleHandleInfo handle = MutableShuffleHandleInfo.fromProto(response.getHandle());
taskAttemptAssignment.update(handle);
LOG.info(
"Success to reassign. The latest available assignment is {}",
handle.getAvailablePartitionServersForWriter());
} catch (Exception e) {
throw new RssException(
"Errors on reassign on block send failure. failure partition->servers : "
Expand Down Expand Up @@ -677,6 +680,8 @@ private void reassignAndResendBlocks(Set<TrackingBlockStatus> blocks) {
}

processShuffleBlockInfos(resendCandidates);
LOG.info(
"Failed blocks have been resent to data pusher queue since reassignment has been finished successfully");
}

private void clearFailedBlockState(ShuffleBlockInfo block) {
Expand Down

0 comments on commit 0a36b3c

Please sign in to comment.