Skip to content

Conversation

@dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Jul 2, 2025

What changes were proposed in this pull request?

This PR aims to fix jvmArgs parameter to support multiple arguments.

- name: OPERATOR_JAVA_OPTS
value: {{ .Values.operatorDeployment.operatorPod.operatorContainer.jvmArgs }}

Why are the changes needed?

The previous syntax, "$OPERATOR_JAVA_OPTS", cannot handle multiple Java options.

exec java -cp "./$SPARK_OPERATOR_JAR" "$LOG_CONFIG" "$OPERATOR_JAVA_OPTS" org.apache.spark.k8s.operator.SparkOperator

$ cat test.sh
OPERATOR_JAVA_OPTS="-Dfile.encoding=UTF8 -XX:+ExitOnOutOfMemoryError"
exec java "$OPERATOR_JAVA_OPTS" -XX:+PrintFlagsFinal -version

$ sh test.sh | grep ExitOnOutOfMemoryError
     bool ExitOnOutOfMemoryError                   = false                                     {product} {default}
openjdk version "21.0.7" 2025-04-15
OpenJDK Runtime Environment Homebrew (build 21.0.7)
OpenJDK 64-Bit Server VM Homebrew (build 21.0.7, mixed mode, sharing)

Does this PR introduce any user-facing change?

Yes. This is a bug fix.

How was this patch tested?

Manual review because JVM option check is a little tricky.

Was this patch authored or co-authored using generative AI tooling?

No.

@github-actions github-actions bot added the BUILD label Jul 2, 2025
echo "Starting Operator..."

exec java -cp "./$SPARK_OPERATOR_JAR" "$LOG_CONFIG" "$OPERATOR_JAVA_OPTS" org.apache.spark.k8s.operator.SparkOperator
exec java -cp "./$SPARK_OPERATOR_JAR" "$LOG_CONFIG" $OPERATOR_JAVA_OPTS org.apache.spark.k8s.operator.SparkOperator
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous syntax cannot handle multiple Java options.

$ cat test.sh
OPERATOR_JAVA_OPTS="-Dfile.encoding=UTF8 -XX:+ExitOnOutOfMemoryError"
exec java "$OPERATOR_JAVA_OPTS" -XX:+PrintFlagsFinal -version

$ sh test.sh | grep ExitOnOutOfMemoryError
     bool ExitOnOutOfMemoryError                   = false                                     {product} {default}
openjdk version "21.0.7" 2025-04-15
OpenJDK Runtime Environment Homebrew (build 21.0.7)
OpenJDK 64-Bit Server VM Homebrew (build 21.0.7, mixed mode, sharing)

@dongjoon-hyun
Copy link
Member Author

cc @viirya , @peter-toth , @jiangzho

@dongjoon-hyun
Copy link
Member Author

Thank you, @viirya .
Merged to main.

@dongjoon-hyun dongjoon-hyun deleted the SPARK-52647 branch July 2, 2025 03:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants