Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 68 additions & 18 deletions hadoop-hdds/common/src/main/conf/ozone-env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,10 @@ export OZONE_OS_TYPE=${OZONE_OS_TYPE:-$(uname -s)}
# helper scripts # such as workers.sh, start-ozone.sh, etc.
# export OZONE_WORKERS="${OZONE_CONF_DIR}/workers"

# A space-separated list of worker host names, used as an alternative to the
# OZONE_WORKERS file. Only one of OZONE_WORKERS or OZONE_WORKER_NAMES may be set.
# export OZONE_WORKER_NAMES=""

###
# Options for all daemons
###
Expand All @@ -168,6 +172,15 @@ export OZONE_OS_TYPE=${OZONE_OS_TYPE:-$(uname -s)}
# non-secure)
#

# Extra Java runtime options for all Ozone server daemons (OM, SCM, DataNode,
# S3 Gateway, Recon, HttpFS, CSI). These get appended to OZONE_OPTS for such
# daemons and are a convenient way to apply common options to all of them.
# export OZONE_SERVER_OPTS=""

# Simple override of the default log level used to build OZONE_ROOT_LOGGER and
# OZONE_DAEMON_ROOT_LOGGER. Defaults to INFO.
# export OZONE_LOGLEVEL=INFO

# Where (primarily) daemon log files are stored.
# ${OZONE_HOME}/logs by default.
# Java property: hadoop.log.dir
Expand Down Expand Up @@ -207,16 +220,6 @@ export OZONE_OS_TYPE=${OZONE_OS_TYPE:-$(uname -s)}
# Java property: hadoop.policy.file
# export OZONE_POLICYFILE="hadoop-policy.xml"

#
# NOTE: this is not used by default! <-----
# You can define variables right here and then re-use them later on.
# For example, it is common to use the same garbage collection settings
# for all the daemons. So one could define:
#
# export OZONE_GC_SETTINGS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps"
#
# .. and then use it when setting OZONE_OM_OPTS, etc. below

###
# Secure/privileged execution
###
Expand All @@ -233,21 +236,17 @@ export OZONE_OS_TYPE=${OZONE_OS_TYPE:-$(uname -s)}
# data transfer protocol using non-privileged ports.
# export JSVC_HOME=/usr/bin

# Extra arguments to pass to jsvc when launching secure/privileged daemons.
# export OZONE_DAEMON_JSVC_EXTRA_OPTS=""

#
# This directory contains pids for secure and privileged processes.
#export OZONE_SECURE_PID_DIR=${OZONE_PID_DIR}

#
# This directory contains the logs for secure and privileged processes.
# Java property: hadoop.log.dir
# export OZONE_SECURE_LOG=${OZONE_LOG_DIR}

#
# When running a secure daemon, the default value of OZONE_IDENT_STRING
# ends up being a bit bogus. Therefore, by default, the code will
# replace OZONE_IDENT_STRING with OZONE_xx_SECURE_USER. If one wants
# to keep OZONE_IDENT_STRING untouched, then uncomment this line.
# export OZONE_SECURE_IDENT_PRESERVE="true"
# export OZONE_SECURE_LOG_DIR=${OZONE_LOG_DIR}

###
# Ozone Manager specific parameters
Expand Down Expand Up @@ -276,6 +275,57 @@ export OZONE_OS_TYPE=${OZONE_OS_TYPE:-$(uname -s)}
#
# export OZONE_SCM_OPTS=""

###
# S3 Gateway specific parameters
###
# Specify the JVM options to be used when starting the S3 Gateway.
# These options will be appended to the options specified as OZONE_OPTS
# and therefore may override any similar flags set in OZONE_OPTS
#
# export OZONE_S3G_OPTS=""

###
# Recon specific parameters
###
# Specify the JVM options to be used when starting Recon.
# These options will be appended to the options specified as OZONE_OPTS
# and therefore may override any similar flags set in OZONE_OPTS
#
# export OZONE_RECON_OPTS=""

###
# HttpFS Gateway specific parameters
###
# Specify the JVM options to be used when starting the HttpFS Gateway.
# These options will be appended to the options specified as OZONE_OPTS
# and therefore may override any similar flags set in OZONE_OPTS
#
# export OZONE_HTTPFS_OPTS=""

###
# CSI server specific parameters
###
# Specify the JVM options to be used when starting the CSI server.
# These options will be appended to the options specified as OZONE_OPTS
# and therefore may override any similar flags set in OZONE_OPTS
#
# export OZONE_CSI_OPTS=""

###
# Client and tool command specific parameters
###
# Specify the JVM options to be used when running the corresponding command
# (ozone sh, fs, admin, debug, freon, vapor). These options will be appended
# to the options specified as OZONE_OPTS and therefore may override any
# similar flags set in OZONE_OPTS
#
# export OZONE_SH_OPTS=""
# export OZONE_FS_OPTS=""
# export OZONE_ADMIN_OPTS=""
# export OZONE_DEBUG_OPTS=""
# export OZONE_FREON_OPTS=""
# export OZONE_VAPOR_OPTS=""

###
# Advanced Users Only!
###
Expand Down
2 changes: 1 addition & 1 deletion hadoop-ozone/dist/src/shell/ozone/ozone
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ function ozonecmd_case
;;
httpfs)
OZONE_SUBCMD_SUPPORTDAEMONIZATION="true"
OZONE_OPTS="${OZONE_OPTS} ${RATIS_OPTS} -Dhttpfs.home.dir=${OZONE_HOME} -Dhttpfs.config.dir=${OZONE_CONF_DIR} -Dhttpfs.log.dir=${OZONE_HOME}/log -Dhttpfs.temp.dir=${OZONE_HOME}/temp -Dlog4j.configuration=file:${OZONE_CONF_DIR}/log4j.properties ${OZONE_MODULE_ACCESS_ARGS}"
OZONE_HTTPFS_OPTS="${OZONE_HTTPFS_OPTS} ${RATIS_OPTS} -Dhttpfs.home.dir=${OZONE_HOME} -Dhttpfs.config.dir=${OZONE_CONF_DIR} -Dhttpfs.log.dir=${OZONE_HOME}/log -Dhttpfs.temp.dir=${OZONE_HOME}/temp -Dlog4j.configuration=file:${OZONE_CONF_DIR}/log4j.properties ${OZONE_MODULE_ACCESS_ARGS}"
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One subtle point: this follows the existing OZONE_S3G_OPTS / OZONE_RECON_OPTS pattern by appending the built-in defaults after the user-provided component opts. That means user-provided duplicate -Dhttpfs.* properties in OZONE_HTTPFS_OPTS will not override the built-in HttpFS properties, because Java uses the last duplicate -D value.

I think this is acceptable for consistency with the other daemons, which is the explicit goal of HDDS-14443. OZONE_HTTPFS_OPTS is meant for adding JVM options; letting users override the built-in httpfs.* paths would be a broader, cross-daemon change and is out of scope here.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 this is good.

OZONE_CLASSNAME='org.apache.ozone.fs.http.server.HttpFSServerWebServer'
OZONE_RUN_ARTIFACT_NAME="ozone-httpfsgateway"
;;
Expand Down