Skip to content

Conversation

@zhudebin
Copy link

fix bug HADOOP-13192

umbrant and others added 30 commits September 1, 2015 18:30
… they're not empty. Contributed by Colin McCabe and Eddy Xu.

(cherry picked from commit 50ee8f4)
(cherry picked from commit 2f46ee5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/TestBlockListAsLongs.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestNameNodePrunesMissingStorages.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestTriggerBlockReport.java

(cherry picked from commit 03d4af3)
…block to stay

underreplicated for long periods. Contributed by Nathan Roberts.
(cherry picked from commit 04ee18e)

(cherry picked from commit c4cedfc)
(cherry picked from commit c6b68a8)
… to a reducer. Contributed by Jason Lowe.

(cherry picked from commit eccb7d4)

(cherry picked from commit cacadea)
(cherry picked from commit 2abd4f61075739514fb3e63b118448895be02a30)
…tImpl lock for a very long time (sinago via cmccabe)

(cherry picked from commit 28bebc8)
(cherry picked from commit a827089)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java

(cherry picked from commit c3a3092)
…ile writing block. (Arpit Agarwal)

(cherry picked from commit f032473)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalReplicaInPipeline.java

(cherry picked from commit de21de7)
…Kihwal Lee.

(cherry picked from commit 285b31e)

(cherry picked from commit 7e62207)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ContentSummaryComputationContext.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java

(cherry picked from commit 1ef5e0b)
… token renewal of applications part of a bigger workflow. Contributed by Daryn Sharp.

(cherry picked from commit 9c59112)
(cherry picked from commit 1ff3fd3)
(cherry picked from commit 82c722aae86669325672dd10840447434f15e7fd)
… Li.

(cherry picked from commit 4cec9975c0e2c2fcca090888dc2c5e149c5be1e9)
(cherry picked from commit 3de0bf9)
(cherry picked from commit c3b2b377f19293797a88f4636f07e2ab8e18c78b)
…finalize upgrade. Contributed by Jing Zhao.

(cherry picked from commit fddd552)
(cherry picked from commit 38b031d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java

(cherry picked from commit b4e227e)
…te" when mem settings are changed. (Jian He via wangda)

(cherry picked from commit f65eeb4)
(cherry picked from commit e7cbecd)
(cherry picked from commit 9d47d5aa5bffe427c4a77260f7ccc039d446e1fd)
…en calling getQueue (Jason Lowe via wangda)

(cherry picked from commit f47a576)
(cherry picked from commit 3316cd4)
(cherry picked from commit 24d45ee9544abcfcf9e611ab835ec2f824333670)
… own class. (Chris Trezzo via kasha)

(cherry picked from commit c66c3ac)
(cherry picked from commit f4d6c5e)
(cherry picked from commit 2b9cac26ec5457ad4e18ec418ff8249c37861eb8)
…s which is a regression from MR1 (zxu via rkanter)

(cherry picked from commit d50e8f0)
(cherry picked from commit 756c254)
(cherry picked from commit 4f0fd74880e58958d89ca1cd4b7f665520595b2c)
… up split calculation (gera)

(cherry picked from commit 6d2cf9f)
(cherry picked from commit 1544c63)
… and fail to tell the DFSClient about it because of a network error (cmccabe)

(cherry picked from commit bc9cb3e)
(cherry picked from commit c71b54f)
…DFS-7915 DataNode (cmccabe)

(cherry picked from commit a889844)
(cherry picked from commit 3d0385c)
…here is an I/O error during requestShortCircuitShm (cmccabe)

(cherry picked from commit a0e0a63)
(cherry picked from commit 788b767)
localized. Contributed by Chengbing Liu

(cherry picked from commit 0d6bd62)
(cherry picked from commit a7696b3)
(cherry picked from commit 9e30232004ab7c3c3bfde3b8b27c37fa7065f6be)
…ocalizing all resources. (Zhihai Xu via kasha)

(cherry picked from commit 47279c3)
(cherry picked from commit 4045c41)
(cherry picked from commit 6f2cc0dfa8f21984ecdab59dc087ccf525934930)
…M-RM tokens when they roll-over. Contributed by Jason Lowe.

(cherry picked from commit 9fc32c5)
(cherry picked from commit 32dc13d)
(cherry picked from commit aad56fe3a2f29e73a013b9afa9b44b151a34e0f3)
…Brahma Reddy Battula via gera)

(cherry picked from commit f343f86)
(cherry picked from commit 58970d6)
…erge. Contributed by Gera Shegalov

(cherry picked from commit 7dc3c12)

(cherry picked from commit 87c2d91)
(cherry picked from commit 1da62ba736f5f161a18a52b7ca0d212786f3848c)
…luster restart. (surendra singh lilhore via Xiaoyu Yao)

(cherry picked from commit 0100b15)
(cherry picked from commit e68e8b3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java

(cherry picked from commit b054cb6)
…y command is issued. Contributed by Brahma Reddy Battula.

(cherry picked from commit 6f62267)

(cherry picked from commit 802a577)
(cherry picked from commit 995382c)
…ll block reports from a DN (cmccabe)

(cherry picked from commit baf794d)

(cherry picked from commit 5f3d967)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm

(cherry picked from commit a776ef5)
…en copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du.

(cherry picked from commit f4e2b3c)
(cherry picked from commit fb5b0eb)
(cherry picked from commit 3a3dcf0bb689d6aa8bc524a31799067702814674)
… exceptions happen in stopping NM's sub-services. Contributed by Junping Du

(cherry picked from commit 711d77c)

(cherry picked from commit a81ad81)
(cherry picked from commit aa82b0684554be8d09f6fcd88826f167922280cc)
…n a QJM cluster. Contributed by Weiwei Yang

(cherry picked from commit b0ad644)
(cherry picked from commit 802676e)
(cherry picked from commit 2cadeb9e017c6a75db16e1f23b2accda04f12298)
…lock request is in edit log. Contributed by Rushabh S Shah.

(cherry picked from commit 2d4ae3d)

(cherry picked from commit f264a5a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReplacement.java

(cherry picked from commit 470019e)
mingmasplace and others added 27 commits March 7, 2016 20:52
…(Sangjin Lee via mingma)

(cherry picked from commit 49eedc7)
(cherry picked from commit f01f194)
(cherry picked from commit d6e40fd)
…o in response of RM REST API. (Varun Vasudev via junping-du)
…tributed by Zhe Zhang.

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
…-Chiu Chuang.

(cherry picked from commit ab725cf)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
…REDUCE-6363. Contributed by J.Andreina.

(cherry picked from commit 368c773)
(cherry picked from commit 7a3fd1b)
…OOM. Contributed by Sangjin Lee

(cherry picked from commit 9a09200)
…when rolling log aggregation is disabled. Contributed by Jun Gong
…am with accurate .getPos() (Contributed by kanaka kumar avvaru)

(cherry picked from commit 48ca23d)
(cherry picked from commit d21bc81)

(cherry picked from commit 62da8f6)

 Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
        hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
…ppend (Contributed by Brahma Reddy Battula)

(cherry picked from commit 37e4e45)
Added CHANGES.txt entry

(cherry picked from commit b52ac3c)
(cherry picked from commit fea50c5)
(cherry picked from commit c28e4b0)
(cherry picked from commit bae229a)

 Conflicts:
	LICENSE.txt

(cherry picked from commit b8febb7)
…cies.

(cherry picked from commit 08b7efa)
(cherry picked from commit e16040b)
(cherry picked from commit 22403be)
(cherry picked from commit 7cf625188cbc2b369986bf5c98c3f1bbfb8cd198)
…ontributed by harsh. (harsh)

(cherry picked from commit 54e612b)
… resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan.

Updated the 2.7 branch patch to fix a minor issue in TestRMContainerAllocator.java and a minor fix in the 2.6 patch.

(cherry picked from commit 8d48266)
(cherry picked from commit 9f51a3f)
…UG level instead of INFO level. Contributed by Xiaoyu Yao.

(cherry picked from commit 0942c99)
…er resource requests. Contributed by Wangda Tan

(cherry picked from commit c9bb96f)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/rm/TestRMContainerAllocator.java
…NFO level (Yiqun Lin via cmccabe)

(cherry picked from commit f0252ad)
(cherry picked from commit 43f32d3)
(cherry picked from commit 154332b)
(cherry picked from commit a61e041)
…rate duplicate records. Contributed by Wilfred Spiegelenburg

(cherry picked from commit 9227dfc)
@zhudebin zhudebin closed this May 23, 2016
shanthoosh added a commit to shanthoosh/hadoop that referenced this pull request Oct 15, 2019
Rocksdb LocalStoreMonitor is responsible for clearing up unused local task partition stores. Metrics are required to understand the behavior of this monitor in production(especially when it's clearing up unused rocksdb local stores).

The following two metrics will be emitted from this monitor:
a) Total disk space cleared in bytes.
b) Total number of rocksdb task partition stores cleared.

Author: Shanthoosh Venkataraman <svenkataraman@linkedin.com>

Reviewers: Jacob Maes <jmaes@linkedin.com>

Closes apache#95 from shanthoosh/metrics_into_local_store_monitor
saxenapranav pushed a commit to saxenapranav/hadoop that referenced this pull request Aug 18, 2023
Fix for tracing context in sub-sequential and parallel operations
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.