Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-45450][PYTHON] Fix imports according to PEP8: pyspark.pandas and pyspark (core) #43257

Closed
wants to merge 2 commits into from

Conversation

HyukjinKwon
Copy link
Member

What changes were proposed in this pull request?

This PR proposes to fix imports according to PEP8 in pyspark.pandas and pyspark.* (core), see https://peps.python.org/pep-0008/#imports.

Why are the changes needed?

I have not been fixing them as they are too minor. However, this practice is being propagated across the whole PySpark packages, and I think we should fix them all so other users do not follow the non-standard practice.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Existing linters and tests should cover.

Was this patch authored or co-authored using generative AI tooling?

No.

@HyukjinKwon
Copy link
Member Author

cc @itholic @zhengruifeng FYI

@dongjoon-hyun
Copy link
Member

Could you fix flake8 check?

flake8 checks failed:
./python/pyspark/pandas/internal.py:22:1: F401 'typing.TYPE_CHECKING' imported but unused
from typing import Any, Dict, List, Optional, Sequence, Tuple, Union, TYPE_CHECKING, cast
^
./python/pyspark/pandas/internal.py:1284:56: F821 undefined name 'Series'
        scols_or_pssers: Sequence[Union[PySparkColumn, "Series"]],
                                                       ^
./python/pyspark/pandas/internal.py:1369:54: F821 undefined name 'Series'
    def with_filter(self, pred: Union[PySparkColumn, "Series"]) -> "InternalFrame":
                                                     ^
1     F401 'typing.TYPE_CHECKING' imported but unused
2     F821 undefined name 'Series'

@holdenk
Copy link
Contributor

holdenk commented Oct 10, 2023

Also could we add automated checking for the import order change so that we don't have to do this again?

@HyukjinKwon
Copy link
Member Author

Yeah, I think we should add this to a linter ..

@HyukjinKwon
Copy link
Member Author

Merged to master.

@HyukjinKwon HyukjinKwon deleted the SPARK-45450 branch January 15, 2024 00:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants