HDDS-8825. Use apache/hadoop 3.3.5 docker image #4963
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Change-Id: I934094457b8d98ace07cf4c3f43abdd59d49e929 Update Change-Id: Icc5e52f3c0f6a42c65b0b1817e4d26555320a5b3 Fix Change-Id: I5a8c10896172ed9ac2346824aa2d9141c31520dc Revert "Fix" This reverts commit 4a1ceeb4e3f6290feccb15d378f5b9ab8ad507d3. Revert "Update" This reverts commit 83dce5c2cd6baad6b92f981d4758cb0e24932667. Update Change-Id: I37ad5774d0b88f436693348b0f785fa0f293ac65 Old Hadoop versions, use flokkr/hadoop image. Hadoop 3.3 line, use apache/hadoop image. Change-Id: Id70d316c3672efdc7f7452e19d9ab887b62a3ec7
Change-Id: Iad17f2776c003ea3d049aa8e4ce412d23e84bbf0
Change-Id: I6f0429e83c8b0dddcb59fd45f7e18fc691eaf3ea
Change-Id: I122e3b536d7b63f240190e1ceea726a6475d3466
adoroszlai
approved these changes
Jun 22, 2023
There was a problem hiding this comment.
Thanks @jojochuang for the patch. I didn't know about the new Hadoop image.
|
Thanks @jojochuang for the patch, @smengcl for the review. |
errose28
added a commit
to errose28/ozone
that referenced
this pull request
Jun 25, 2023
* master: (96 commits) HDDS-8586 Recon. - API for Count of deletePending keys and amount of data mapped to such keys. (apache#4923) HDDS-8908. Intermittent failure in TestBlockDeletion#testBlockDeletion (apache#4958) HDDS-8910. Replace LockManager with striped lock in ContainerStateManager (apache#4962) HDDS-8917. Move protobuf conversion out of the lock in PipelineStateManagerImpl (apache#4965) HDDS-8825. Use apache/hadoop 3.3.5 docker image (apache#4963) HDDS-8906. Avoid stream when getting in-service healthy nodes (apache#4960) HDDS-8907. Store volume count when storage report is updated (apache#4957) HDDS-8905. PipelineManager metrics should not be synchronized (apache#4959) HDDS-8553. Improve scanner integration tests. (apache#4936) HDDS-8854. Avoid unnecessary DatanodeDetails creation for NodeStateManager lookup (apache#4925) HDDS-8315. [Snapshot] Added unit tests for SnapshotDiffManager (apache#4716) HDDS-7968. [Snapshot] Improve KeyDeletingService to reclaim eligible key blocks in snapshot's deletedTable (apache#4935) HDDS-8838. Update default datanode check empty containter on disk to false (apache#4937) HDDS-8763. Support RocksDB iterator with ByteBuffer. (apache#4942) HDDS-8543. FSO directory should reflect bucket/cluster default replication (apache#4947) HDDS-8898. Replication limit should not be less than reconstruction weight (apache#4954) HDDS-8739. Snapdiff should return complete absolute path in Diff Entry (apache#4823) HDDS-8908. Mark TestBlockDeletion#testBlockDeletion as flaky HDDS-8534. Support asynchronous service logging (apache#4663) HDDS-8879. Cleanup SecurityConfig and related class initialization (apache#4921) ...
errose28
added a commit
to errose28/ozone
that referenced
this pull request
Jun 26, 2023
* tmp-dir-refactor: (99 commits) HDDS-8586 Recon. - API for Count of deletePending keys and amount of data mapped to such keys. (apache#4923) Fix SCM HA finalization compat test HDDS-8908. Intermittent failure in TestBlockDeletion#testBlockDeletion (apache#4958) HDDS-8910. Replace LockManager with striped lock in ContainerStateManager (apache#4962) HDDS-8917. Move protobuf conversion out of the lock in PipelineStateManagerImpl (apache#4965) HDDS-8825. Use apache/hadoop 3.3.5 docker image (apache#4963) HDDS-8906. Avoid stream when getting in-service healthy nodes (apache#4960) HDDS-8907. Store volume count when storage report is updated (apache#4957) HDDS-8905. PipelineManager metrics should not be synchronized (apache#4959) HDDS-8553. Improve scanner integration tests. (apache#4936) HDDS-8854. Avoid unnecessary DatanodeDetails creation for NodeStateManager lookup (apache#4925) HDDS-8315. [Snapshot] Added unit tests for SnapshotDiffManager (apache#4716) HDDS-7968. [Snapshot] Improve KeyDeletingService to reclaim eligible key blocks in snapshot's deletedTable (apache#4935) HDDS-8838. Update default datanode check empty containter on disk to false (apache#4937) HDDS-8763. Support RocksDB iterator with ByteBuffer. (apache#4942) HDDS-8543. FSO directory should reflect bucket/cluster default replication (apache#4947) HDDS-8898. Replication limit should not be less than reconstruction weight (apache#4954) HDDS-8739. Snapdiff should return complete absolute path in Diff Entry (apache#4823) HDDS-8908. Mark TestBlockDeletion#testBlockDeletion as flaky HDDS-8534. Support asynchronous service logging (apache#4663) ...
errose28
added a commit
to errose28/ozone
that referenced
this pull request
Jun 26, 2023
* master: (79 commits) HDDS-8914. Datanode may fail to start due to duplicate VolumeInfoMetrics (apache#4966) HDDS-8921. Add support for EC in Freon SCM block generator (apache#4982) HDDS-8927. Metadata scanner should not scan unhealthy containers. (apache#4976) HDDS-8929. Avoid list allocation for pipeline search (apache#4980) HDDS-8778. Support recursive volume delete using Ozone sh command. (apache#4842) HDDS-8885. Quota repair count enable quota feature for old bucket/volume. (apache#4941) HDDS-8771. Refactor volume level tmp directory for generic usage. (apache#4838) HDDS-8922. Random EC read pipeline ID causes XceiverClient cache churn (apache#4971) HDDS-8586 Recon. - API for Count of deletePending keys and amount of data mapped to such keys. (apache#4923) HDDS-8908. Intermittent failure in TestBlockDeletion#testBlockDeletion (apache#4958) HDDS-8910. Replace LockManager with striped lock in ContainerStateManager (apache#4962) HDDS-8917. Move protobuf conversion out of the lock in PipelineStateManagerImpl (apache#4965) HDDS-8825. Use apache/hadoop 3.3.5 docker image (apache#4963) HDDS-8906. Avoid stream when getting in-service healthy nodes (apache#4960) HDDS-8907. Store volume count when storage report is updated (apache#4957) HDDS-8905. PipelineManager metrics should not be synchronized (apache#4959) HDDS-8553. Improve scanner integration tests. (apache#4936) HDDS-8854. Avoid unnecessary DatanodeDetails creation for NodeStateManager lookup (apache#4925) HDDS-8315. [Snapshot] Added unit tests for SnapshotDiffManager (apache#4716) HDDS-7968. [Snapshot] Improve KeyDeletingService to reclaim eligible key blocks in snapshot's deletedTable (apache#4935) ...
vtutrinov
pushed a commit
to Cyrill/ozone
that referenced
this pull request
Jul 3, 2023
vtutrinov
pushed a commit
to Cyrill/ozone
that referenced
this pull request
Jul 3, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Our tests depend on flokkr/hadoop which has stopped developing. We should switch to apache/hadoop docker images which has the latest Hadoop 3.3.5 docker image. For the tests that rely on older Hadoop versions, keep using flokkr/hadoop because apache/hadoop currently only maintain 3.3.x images.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-8825
How was this patch tested?
Existing tests