Skip to content

Commit

Permalink
[SPARK-48076][K8S] Promote KubernetesVolumeUtils to DeveloperApi
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

This PR aims to promote `KubernetesVolumeUtils` to `DeveloperApi` from Apache Spark 4.0.0 for Apache Spark Kubernetes Operator.

### Why are the changes needed?

This API was added by the following at `Apache Spark 3.0.0` and has been stable.
- #22959

Since `Apache Spark Kubernetes Operator` requires this, we had better maintain it as a developer API officially from `Apache Spark 4.0.0`.
- apache/spark-kubernetes-operator#10

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #46326 from dongjoon-hyun/SPARK-48076.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
  • Loading branch information
dongjoon-hyun committed May 2, 2024
1 parent e521d3c commit 04f3a93
Showing 1 changed file with 10 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,25 @@ package org.apache.spark.deploy.k8s
import java.lang.Long.parseLong

import org.apache.spark.SparkConf
import org.apache.spark.annotation.{DeveloperApi, Since, Unstable}
import org.apache.spark.deploy.k8s.Config._

private[spark] object KubernetesVolumeUtils {
/**
* :: DeveloperApi ::
*
* A utility class used for K8s operations internally and Spark K8s operator.
*/
@Unstable
@DeveloperApi
object KubernetesVolumeUtils {
/**
* Extract Spark volume configuration properties with a given name prefix.
*
* @param sparkConf Spark configuration
* @param prefix the given property name prefix
* @return a Map storing with volume name as key and spec as value
*/
@Since("3.0.0")
def parseVolumesWithPrefix(sparkConf: SparkConf, prefix: String): Seq[KubernetesVolumeSpec] = {
val properties = sparkConf.getAllWithPrefix(prefix).toMap

Expand Down

0 comments on commit 04f3a93

Please sign in to comment.