Skip to content

Add topology provider jar to Hadoop image.#509

Merged
soenkeliebau merged 3 commits intomainfrom
feat/hadoop_topology_provider
Nov 23, 2023
Merged

Add topology provider jar to Hadoop image.#509
soenkeliebau merged 3 commits intomainfrom
feat/hadoop_topology_provider

Conversation

@soenkeliebau
Copy link
Copy Markdown
Member

Description

This adds an extra jar file to the Hadoop image which contains an implementation of the org.apache.hadoop.net.DNSToSwitchMapping interface to generate rack IDs based on Kubernetes labels, so that HDFS knows the underlying Kubernetes architecture and can distribute blocks properly.

Related to stackabletech/hdfs-operator#325

Definition of Done Checklist

  • Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
  • Please make sure all these things are done and tick the boxes
- [ ] Changes are OpenShift compatible
- [ ] All added packages (via microdnf or otherwise) have a comment on why they are added
- [ ] Things not downloaded from Red Hat repositories should be mirrored in the Stackable repository and downloaded from there
- [ ] All packages should have (if available) signatures/hashes verified
- [ ] Does your change affect an SBOM? Make sure to update all SBOMs

adwk67
adwk67 previously approved these changes Nov 23, 2023
Copy link
Copy Markdown
Member

@adwk67 adwk67 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm (testing to come with the operator changes). Just one nit: do we want to change line 147 to use curl (with --fail) in case the jar-fetch fails for some reason?

@soenkeliebau soenkeliebau added this pull request to the merge queue Nov 23, 2023
Merged via the queue into main with commit 4726089 Nov 23, 2023
@soenkeliebau soenkeliebau deleted the feat/hadoop_topology_provider branch November 23, 2023 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants