Skip to content

chore(docker): fix Hadoop entrypoint.sh property bugs in all base modules#18527

Merged
danny0405 merged 1 commit into
apache:masterfrom
voonhous:fix-hadoop-entrypoint-bugs
Apr 20, 2026
Merged

chore(docker): fix Hadoop entrypoint.sh property bugs in all base modules#18527
danny0405 merged 1 commit into
apache:masterfrom
voonhous:fix-hadoop-entrypoint-bugs

Conversation

@voonhous
Copy link
Copy Markdown
Member

@voonhous voonhous commented Apr 18, 2026

Describe the issue this Pull Request addresses

In docker/hoodie/hadoop/*/entrypoint.sh, the MULTIHOMED_NETWORK=1 block calls:

addProperty /etc/hadoop/yarn-site.xml yarn.nodemanager.bind-host 0.0.0.0
addProperty /etc/hadoop/yarn-site.xml yarn.nodemanager.bind-host 0.0.0.0

addProperty uses sed to insert a new <property> block before </configuration>, so the same property ends up in the XML twice. Hadoop's Configuration parser tolerates duplicate names (last one wins) and both writes use the same value, so there is no runtime impact - but the second line is dead code and inconsistent with the rest of the block. This duplication was inherited from the upstream big-data-europe/docker-hadoop template and propagated when base_java11 and base_java17 were copied from base.

Summary and Changelog

  • Removed the duplicate addProperty /etc/hadoop/yarn-site.xml yarn.nodemanager.bind-host 0.0.0.0 line from the YARN block in each of the three base modules' entrypoint.sh.
  • No behaviour change; scripts now match the apparent intent of the YARN block.

Impact

None. Observable runtime behaviour is identical.

Risk Level

none, three-line deletion across three scripts, no functional change.

low - the change is six lines across three scripts, guarded by MULTIHOMED_NETWORK=1. Before the fix the misplaced YARN property did nothing in mapred-site.xml, after the fix the correct JobHistory properties are written.

No behavior change for the default, non-multi-homed case.

Documentation Update

None.

Contributor's checklist

  • Read through contributor's guide
  • Enough context is provided in the sections above
  • Adequate tests were added if applicable

@github-actions github-actions Bot added the size:S PR with lines of changes in (10, 100] label Apr 18, 2026
…int.sh

- The property was inserted into yarn-site.xml twice with the same value in the MULTIHOMED_NETWORK=1 block.
- Duplicates are harmless at runtime (Hadoop's Configuration parser takes the last value for duplicates and both writes use the same value), but the second write is dead code.
- Applies to base, base_java11, and base_java17.
@voonhous voonhous force-pushed the fix-hadoop-entrypoint-bugs branch from 01fa6e3 to 706eb1a Compare April 18, 2026 13:46
@voonhous voonhous changed the title fix(docker): fix Hadoop entrypoint.sh property bugs in all base modules chore(docker): fix Hadoop entrypoint.sh property bugs in all base modules Apr 18, 2026
@github-actions github-actions Bot added size:XS PR with lines of changes in <= 10 and removed size:S PR with lines of changes in (10, 100] labels Apr 18, 2026
@hudi-bot
Copy link
Copy Markdown
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@codecov-commenter
Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.88%. Comparing base (3d0ab80) to head (706eb1a).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18527      +/-   ##
============================================
+ Coverage     68.87%   68.88%   +0.01%     
- Complexity    28272    28279       +7     
============================================
  Files          2464     2464              
  Lines        135594   135594              
  Branches      16447    16447              
============================================
+ Hits          93389    93404      +15     
+ Misses        34815    34807       -8     
+ Partials       7390     7383       -7     
Flag Coverage Δ
common-and-other-modules 44.64% <ø> (+<0.01%) ⬆️
hadoop-mr-java-client 44.77% <ø> (+<0.01%) ⬆️
spark-client-hadoop-common 48.41% <ø> (-0.01%) ⬇️
spark-java-tests 48.93% <ø> (+0.02%) ⬆️
spark-scala-tests 45.44% <ø> (-0.01%) ⬇️
utilities 38.20% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 10 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Contributor

@hudi-agent hudi-agent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 This review was generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.

No reviewable code files in this PR.

cc @yihua

@danny0405 danny0405 merged commit cfb9833 into apache:master Apr 20, 2026
110 of 119 checks passed
@voonhous voonhous deleted the fix-hadoop-entrypoint-bugs branch April 20, 2026 07:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XS PR with lines of changes in <= 10

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants