Skip to content

[SPARK-37407][FOLLOW-UP][PYTHON][ML] Add not None assertions for active context and its _jvm#35022

Closed
zero323 wants to merge 1 commit intoapache:masterfrom
zero323:SPARK-37407-FOLLOW-UP
Closed

[SPARK-37407][FOLLOW-UP][PYTHON][ML] Add not None assertions for active context and its _jvm#35022
zero323 wants to merge 1 commit intoapache:masterfrom
zero323:SPARK-37407-FOLLOW-UP

Conversation

@zero323
Copy link
Copy Markdown
Member

@zero323 zero323 commented Dec 25, 2021

What changes were proposed in this pull request?

This PR add is not None assertions for Optional _active_spark_context and _jvm

Why are the changes needed?

To address failures in CI:

python/pyspark/ml/functions.py:69: error: Item "None" of "Optional[SparkContext]" has no attribute "_jvm"  [union-attr]
python/pyspark/ml/functions.py:69: error: Item "None" of "Optional[Any]" has no attribute "org"  [union-attr]
python/pyspark/ml/functions.py:104: error: Item "None" of "Optional[SparkContext]" has no attribute "_jvm"  [union-attr]
python/pyspark/ml/functions.py:104: error: Item "None" of "Optional[Any]" has no attribute "org"  [union-attr]

caused by divergence between #34678 and changes introduced by #34892

Does this PR introduce any user-facing change?

No.

How was this patch tested?

dev/lint-python

@zero323
Copy link
Copy Markdown
Member Author

zero323 commented Dec 25, 2021

I am going to merge to it to address current failure on master.

@zero323 zero323 closed this in d424eca Dec 25, 2021
@zero323 zero323 deleted the SPARK-37407-FOLLOW-UP branch December 25, 2021 22:14
@zero323
Copy link
Copy Markdown
Member Author

zero323 commented Dec 25, 2021

Merged into master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant