Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-46887][DOCS] Document a few missed spark.ui.* configs to Configuration page #44913

Closed
wants to merge 1 commit into from

Conversation

dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Jan 27, 2024

What changes were proposed in this pull request?

This PR aims to document a few missed spark.ui.* configurations for Apache Spark 4. This PR focuses only public configurations and excludes internal configuration like spark.ui.jettyStopTimeout.

Why are the changes needed?

To improve documentations.

After this PR, I verified the following configurations are documented at least once in Configuration or Security page.

$ git grep 'ConfigBuilder("spark.ui.'
core/src/main/scala/org/apache/spark/internal/config/Status.scala:  val LIVE_ENTITY_UPDATE_PERIOD = ConfigBuilder("spark.ui.liveUpdate.period")
core/src/main/scala/org/apache/spark/internal/config/Status.scala:  val LIVE_ENTITY_UPDATE_MIN_FLUSH_PERIOD = ConfigBuilder("spark.ui.liveUpdate.minFlushPeriod")
core/src/main/scala/org/apache/spark/internal/config/Status.scala:  val MAX_RETAINED_JOBS = ConfigBuilder("spark.ui.retainedJobs")
core/src/main/scala/org/apache/spark/internal/config/Status.scala:  val MAX_RETAINED_STAGES = ConfigBuilder("spark.ui.retainedStages")
core/src/main/scala/org/apache/spark/internal/config/Status.scala:  val MAX_RETAINED_TASKS_PER_STAGE = ConfigBuilder("spark.ui.retainedTasks")
core/src/main/scala/org/apache/spark/internal/config/Status.scala:  val MAX_RETAINED_DEAD_EXECUTORS = ConfigBuilder("spark.ui.retainedDeadExecutors")
core/src/main/scala/org/apache/spark/internal/config/Status.scala:  val MAX_RETAINED_ROOT_NODES = ConfigBuilder("spark.ui.dagGraph.retainedRootRDDs")
core/src/main/scala/org/apache/spark/internal/config/Status.scala:  val LIVE_UI_LOCAL_STORE_DIR = ConfigBuilder("spark.ui.store.path")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_SHOW_CONSOLE_PROGRESS = ConfigBuilder("spark.ui.showConsoleProgress")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:    ConfigBuilder("spark.ui.consoleProgress.update.interval")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_ENABLED = ConfigBuilder("spark.ui.enabled")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_PORT = ConfigBuilder("spark.ui.port")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_FILTERS = ConfigBuilder("spark.ui.filters")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_ALLOW_FRAMING_FROM = ConfigBuilder("spark.ui.allowFramingFrom")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_REVERSE_PROXY = ConfigBuilder("spark.ui.reverseProxy")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_REVERSE_PROXY_URL = ConfigBuilder("spark.ui.reverseProxyUrl")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_KILL_ENABLED = ConfigBuilder("spark.ui.killEnabled")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_THREAD_DUMPS_ENABLED = ConfigBuilder("spark.ui.threadDumpsEnabled")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_FLAMEGRAPH_ENABLED = ConfigBuilder("spark.ui.threadDump.flamegraphEnabled")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_HEAP_HISTOGRAM_ENABLED = ConfigBuilder("spark.ui.heapHistogramEnabled")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_PROMETHEUS_ENABLED = ConfigBuilder("spark.ui.prometheus.enabled")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_X_XSS_PROTECTION = ConfigBuilder("spark.ui.xXssProtection")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_X_CONTENT_TYPE_OPTIONS = ConfigBuilder("spark.ui.xContentTypeOptions.enabled")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_STRICT_TRANSPORT_SECURITY = ConfigBuilder("spark.ui.strictTransportSecurity")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_REQUEST_HEADER_SIZE = ConfigBuilder("spark.ui.requestHeaderSize")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_TIMELINE_ENABLED = ConfigBuilder("spark.ui.timelineEnabled")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_TIMELINE_TASKS_MAXIMUM = ConfigBuilder("spark.ui.timeline.tasks.maximum")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_TIMELINE_JOBS_MAXIMUM = ConfigBuilder("spark.ui.timeline.jobs.maximum")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_TIMELINE_STAGES_MAXIMUM = ConfigBuilder("spark.ui.timeline.stages.maximum")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_TIMELINE_EXECUTORS_MAXIMUM = ConfigBuilder("spark.ui.timeline.executors.maximum")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_VIEW_ACLS = ConfigBuilder("spark.ui.view.acls")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_VIEW_ACLS_GROUPS = ConfigBuilder("spark.ui.view.acls.groups")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val PROXY_REDIRECT_URI = ConfigBuilder("spark.ui.proxyRedirectUri")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val CUSTOM_EXECUTOR_LOG_URL = ConfigBuilder("spark.ui.custom.executor.log.url")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_SQL_GROUP_SUB_EXECUTION_ENABLED = ConfigBuilder("spark.ui.groupSQLSubExecutionEnabled")
core/src/main/scala/org/apache/spark/internal/config/UI.scala:  val UI_JETTY_STOP_TIMEOUT = ConfigBuilder("spark.ui.jettyStopTimeout")

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manual review.

Was this patch authored or co-authored using generative AI tooling?

No.

@dongjoon-hyun
Copy link
Member Author

cc @viirya

@dongjoon-hyun
Copy link
Member Author

Thank you, @viirya !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants