Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HDFS-16209. Add description for dfs.namenode.caching.enabled #3378

Merged
merged 4 commits into from
Sep 8, 2021
Merged

HDFS-16209. Add description for dfs.namenode.caching.enabled #3378

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9a6e363
HDFS-16209. Set dfs.namenode.caching.enabled to false as default
tomscut Sep 3, 2021
ae0c271
rollback updates to add description information
tomscut Sep 6, 2021
e565fb6
add TODO desc for DFS_NAMENODE_CACHING_ENABLED_DEFAULT
tomscut Sep 6, 2021
1c453e2
empty commit
tomscut Sep 6, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -462,6 +462,7 @@ public class DFSConfigKeys extends CommonConfigurationKeys {
public static final long DFS_NAMENODE_PATH_BASED_CACHE_REFRESH_INTERVAL_MS_DEFAULT = 30000L;
public static final String DFS_NAMENODE_CACHING_ENABLED_KEY =
"dfs.namenode.caching.enabled";
// TODO: Default value to be set false in 4.0.0 release onwards (HDFS-16209)
public static final boolean DFS_NAMENODE_CACHING_ENABLED_DEFAULT = true;

/** Pending period of block deletion since NameNode startup */
Expand Down
4 changes: 4 additions & 0 deletions hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2868,6 +2868,10 @@
maintain a mapping of cached blocks to DataNodes via processing DataNode
cache reports. Based on these reports and addition and removal of caching
directives, the NameNode will schedule caching and uncaching work.
In the current implementation, centralized caching introduces additional
write lock overhead (see CacheReplicationMonitor#rescan) even if no path
to cache is specified, so we recommend disabling this feature when not in
use. We will disable centralized caching by default in later versions.
</description>
</property>

Expand Down
2 changes: 1 addition & 1 deletion hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/CentralizedCacheManagement.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -256,7 +256,7 @@ The following properties are not required, but may be specified for tuning:

* dfs.namenode.caching.enabled

This parameter can be used to enable/disable the centralized caching in NameNode. When centralized caching is disabled, NameNode will not process cache reports or store information about block cache locations on the cluster. Note that NameNode will continute to store the path based cache locations in the file-system metadata, even though it will not act on this information until the caching is enabled. The default value for this parameter is true (i.e. centralized caching is enabled).
This parameter can be used to enable/disable the centralized caching in NameNode. When centralized caching is disabled, NameNode will not process cache reports or store information about block cache locations on the cluster. Note that NameNode will continute to store the path based cache locations in the file-system metadata, even though it will not act on this information until the caching is enabled. The default value for this parameter is true (i.e. centralized caching is enabled). In the current implementation, centralized caching introduces additional write lock overhead (see CacheReplicationMonitor#rescan) even if no path to cache is specified, so we recommend disabling this feature when not in use. We will disable centralized caching by default in later versions.

* dfs.datanode.pmem.cache.recovery

Expand Down