[SPARK-46749][DOCS] Document SPARK_LOG_* and SPARK_PID_DIR

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to document the following three environment variables for Spark Standalone cluster.

• SPARK_LOG_DIR
• SPARK_LOG_MAX_FILES
• SPARK_PID_DIR

Why are the changes needed?

So far, the users need to look at the spark-env.sh.template or spark-daemon.sh files to see the descriptions and the default values. We had better document it officially.

spark/conf/spark-env.sh.template

Lines 67 to 69 in 9a2f393

	# - SPARK_LOG_DIR Where log files are stored. (Default: ${SPARK_HOME}/logs)
	# - SPARK_LOG_MAX_FILES Max log files of Spark daemons can rotate to. Default is 5.
	# - SPARK_PID_DIR Where the pid file is stored. (Default: /tmp)

spark/sbin/spark-daemon.sh

Lines 25 to 28 in 9a2f393

	# SPARK_LOG_DIR Where log files are stored. ${SPARK_HOME}/logs by default.
	# SPARK_LOG_MAX_FILES Max log files of Spark daemons can rotate to. Default is 5.
	# SPARK_MASTER host:path where spark code should be rsync'd from
	# SPARK_PID_DIR The pid files are stored. /tmp by default.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Generate HTML docs.

Was this patch authored or co-authored using generative AI tooling?

No.

[HyukjinKwon]

Merged to master.

[dongjoon-hyun]

Thank you, @HyukjinKwon !