Skip to content

Comments

[HUDI-5487] Reduce duplicate Logs in ExternalSpillableMap#7579

Merged
danny0405 merged 1 commit intoapache:masterfrom
cxzl25:HUDI-5487
Jan 31, 2023
Merged

[HUDI-5487] Reduce duplicate Logs in ExternalSpillableMap#7579
danny0405 merged 1 commit intoapache:masterfrom
cxzl25:HUDI-5487

Conversation

@cxzl25
Copy link
Contributor

@cxzl25 cxzl25 commented Dec 29, 2022

Change Logs

Reduce duplicate Logs in ExternalSpillableMap

We see hundreds of thousands of duplicate logs in the executor log.

22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567

Impact

Reduce noisy logs and reduce log size

Risk level (write none, low medium or high below)

none

Documentation Update

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@cxzl25
Copy link
Contributor Author

cxzl25 commented Dec 29, 2022

@hudi-bot run azure

@hudi-bot
Copy link
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@cxzl25
Copy link
Contributor Author

cxzl25 commented Jan 30, 2023

Help review. Thanks.
@XuQianJin-Stars @alexeykudinkin

@danny0405 danny0405 merged commit 1377143 into apache:master Jan 31, 2023
fengjian428 pushed a commit to fengjian428/hudi that referenced this pull request Jan 31, 2023
@yihua yihua added the priority:blocker Production down; release blocker label Feb 5, 2023
fengjian428 pushed a commit to fengjian428/hudi that referenced this pull request Apr 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority:blocker Production down; release blocker

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

4 participants