Skip to content

[SPARK-45168][PYTHON][FOLLOWUP] test_missing_data.py Code Cleanup#42942

Closed
zhengruifeng wants to merge 1 commit intoapache:masterfrom
zhengruifeng:inc_pd_clean_up
Closed

[SPARK-45168][PYTHON][FOLLOWUP] test_missing_data.py Code Cleanup#42942
zhengruifeng wants to merge 1 commit intoapache:masterfrom
zhengruifeng:inc_pd_clean_up

Conversation

@zhengruifeng
Copy link
Contributor

What changes were proposed in this pull request?

remove unreachable code path

Why are the changes needed?

code cleanup

Does this PR introduce any user-facing change?

no

How was this patch tested?

CI

Was this patch authored or co-authored using generative AI tooling?

no

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-45168][PYTHON][FOLLOWUP] Code Cleanup [SPARK-45168][PYTHON][FOLLOWUP] test_missing_data.py Code Cleanup Sep 15, 2023
@zhengruifeng
Copy link
Contributor Author

after this PR,

(spark_dev_310) ➜  spark git:(inc_pd_clean_up) ag --py 'pandas\.__version' python
python/pyspark/sql/pandas/utils.py
37:    if LooseVersion(pandas.__version__) < LooseVersion(minimum_pandas_version):
40:            "your version was %s." % (minimum_pandas_version, pandas.__version__)
(spark_dev_310) ➜  spark git:(inc_pd_clean_up) ag --py 'pd\.__version' python
python/pyspark/pandas/supported_api_gen.py
102:    if LooseVersion(pd.__version__) != LooseVersion(pandas_latest_version):
105:            "however, your version was %s" % (pandas_latest_version, pd.__version__)

python/pyspark/pandas/tests/series/test_compute.py
490:        LooseVersion(pd.__version__) >= LooseVersion("2.0.0"),

python/pyspark/pandas/tests/test_stats.py
169:        LooseVersion(pd.__version__) >= LooseVersion("2.0.0"),

python/pyspark/pandas/tests/test_dataframe_spark_io.py
102:            if LooseVersion(pd.__version__) > LooseVersion("1.5.0"):

python/pyspark/pandas/tests/test_series_conversion.py
53:        LooseVersion(pd.__version__) >= LooseVersion("2.0.0"),

python/pyspark/pandas/tests/test_ops_on_diff_frames.py
497:        LooseVersion(pd.__version__) >= LooseVersion("2.0.0"),

python/pyspark/pandas/tests/test_series_datetime.py
201:        LooseVersion(pd.__version__) >= LooseVersion("2.0.0"),
208:        LooseVersion(pd.__version__) >= LooseVersion("2.0.0"),

python/pyspark/pandas/tests/groupby/test_split_apply.py
44:        LooseVersion(pd.__version__) >= LooseVersion("2.0.0"),

python/pyspark/pandas/tests/groupby/test_index.py
44:        LooseVersion(pd.__version__) >= LooseVersion("2.0.0"),

python/pyspark/pandas/tests/groupby/test_stat.py
248:        LooseVersion(pd.__version__) >= LooseVersion("2.0.0"),

python/pyspark/pandas/tests/computation/test_corrwith.py
62:        if LooseVersion(pd.__version__) == LooseVersion("1.5.0") and isinstance(pobj, pd.Series):

python/pyspark/pandas/tests/test_dataframe_conversion.py
206:        LooseVersion(pd.__version__) >= LooseVersion("2.0.0"),

python/pyspark/pandas/tests/diff_frames_ops/test_cov_corrwith.py
133:        if LooseVersion(pd.__version__) == LooseVersion("1.5.0"):
149:        if LooseVersion(pd.__version__) == LooseVersion("1.5.0"):

python/pyspark/pandas/tests/indexes/test_datetime.py
101:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):

python/pyspark/pandas/tests/indexes/test_category.py
205:        if LooseVersion(pd.__version__) >= LooseVersion("1.5.0"):

python/pyspark/pandas/tests/indexes/test_base.py
202:        if LooseVersion(pd.__version__) < LooseVersion("1.5.0"):
284:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):
459:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):
1517:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):
1590:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):
1604:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):
1618:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):
1632:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):
1646:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):
1661:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):
1675:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):
1683:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):
1694:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):
1702:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):
1710:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):
1718:        if LooseVersion(pd.__version__) >= LooseVersion("2.0.0"):

python/pyspark/pandas/utils.py
580:                    % (pd.__version__, param.name, pandas_func.__name__)

@zhengruifeng
Copy link
Contributor Author

@zhengruifeng
Copy link
Contributor Author

thanks @dongjoon-hyun , merged to master

@zhengruifeng zhengruifeng deleted the inc_pd_clean_up branch September 15, 2023 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants