[SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io #32964
Conversation
python/pyspark/pandas/frame.py
Outdated
| See :meth:`pyspark.pandas.DataFrame.spark.to_spark_io`. | ||
|
|
||
| .. deprecated:: 3.2.0 | ||
| Use :func:`spark.to_spark_io` instead. |
There was a problem hiding this comment.
Does this link correctly in the built documentation? I thought it has to be :func:DataFrame.spark.to_spark_io
There was a problem hiding this comment.
@HyukjinKwon Thanks for the review. Updated it.
|
ok to test |
|
Test build #140007 has finished for PR 32964 at commit
|
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
|
Test build #140015 has finished for PR 32964 at commit
|
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
| See :meth:`pyspark.pandas.DataFrame.spark.to_spark_io`. | ||
|
|
||
| .. deprecated:: 3.2.0 | ||
| Use :func:`DataFrame.spark.to_spark_io` instead. |
There was a problem hiding this comment.
What's diff here? :meth:`pyspark.pandas.DataFrame.spark.to_spark_io` vs :func:`DataFrame.spark.to_spark_io`?
There was a problem hiding this comment.
Updated to pyspark.pandas.spark.accessors.SparkFrameMethods.to_spark_io. Please correct me if I'm wrong.
|
Test build #140036 has finished for PR 32964 at commit
|
|
Kubernetes integration test starting |
|
Kubernetes integration test status success |
|
Merged to master. Thanks @pingsutw |
| .. deprecated:: 3.2.0 | ||
| Use :func:`DataFrame.spark.to_spark_io` instead. | ||
| """ | ||
| warnings.warn("Deprecated in 3.2, Use spark.to_spark_io instead.", FutureWarning) |
There was a problem hiding this comment.
shall we use DataFrame.spark.to_spark_io ?
There was a problem hiding this comment.
Sounds good! I'll address it, thanks :)
### What changes were proposed in this pull request? This PR is followup for #32964, to improve the warning message. ### Why are the changes needed? To improve the warning message. ### Does this PR introduce _any_ user-facing change? The warning is changed from "Deprecated in 3.2, Use `spark.to_spark_io` instead." to "Deprecated in 3.2, Use `DataFrame.spark.to_spark_io` instead." ### How was this patch tested? Manually run `dev/lint-python` Closes #33631 from itholic/SPARK-35811-followup. Authored-by: itholic <haejoon.lee@databricks.com> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
### What changes were proposed in this pull request? This PR is followup for #32964, to improve the warning message. ### Why are the changes needed? To improve the warning message. ### Does this PR introduce _any_ user-facing change? The warning is changed from "Deprecated in 3.2, Use `spark.to_spark_io` instead." to "Deprecated in 3.2, Use `DataFrame.spark.to_spark_io` instead." ### How was this patch tested? Manually run `dev/lint-python` Closes #33631 from itholic/SPARK-35811-followup. Authored-by: itholic <haejoon.lee@databricks.com> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org> (cherry picked from commit 3d72c20) Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
What changes were proposed in this pull request?
Deprecate the
DataFrame.to_spark_ioWhy are the changes needed?
We should deprecate the
DataFrame.to_spark_iosince it's duplicated withDataFrame.spark.to_spark_io, and it's not existed in pandas.Does this PR introduce any user-facing change?
Yes, users will get warning while using
DataFrame.to_spark_ioapi.How was this patch tested?
Pass the CIs