Skip to content

[TYPING] Add type overloads for inplace dataframe operations#48662

Closed
tigerhawkvok wants to merge 4 commits into
apache:masterfrom
tigerhawkvok:overloads
Closed

[TYPING] Add type overloads for inplace dataframe operations#48662
tigerhawkvok wants to merge 4 commits into
apache:masterfrom
tigerhawkvok:overloads

Conversation

@tigerhawkvok
Copy link
Copy Markdown

Add type overloads for inplace dataframe operationsctly show a return of None or DataFrame based on flag state.

Additionally adds one small type check function, validate_strict_bool() to wrap validate_bool_kwargs() for inplace, as validate_bool_kwargs() accepts bool or None; but inplace can only be bool.

Aside from that very minor function edit, all other changes are exclusively to type hints.

Simple test case:

from typing import Any, cast
from pyspark.pandas import DataFrame

foo = cast(DataFrame, None).sort_index()
bar = cast(DataFrame, None).sort_index(inplace= True)

bam = bool(cast(Any, None))
baz = cast(DataFrame, None).sort_index(inplace= bam)

Before:

The union is returned regardless of state of inplace

image
image
image

After:

  • inplace is implicitly or explicitly True returns a DataFrame
    image

  • inplace is explicitly False returns None
    image

  • inplace is ambiguous and either True or False, still returns Union[DataFrame, None]
    image
    image

…ctly show a return of None or DataFrame based on flag state
@tigerhawkvok
Copy link
Copy Markdown
Author

The lint error for this PR is an open MyPy bug:

python/mypy#11165

@HyukjinKwon
Copy link
Copy Markdown
Member

cc @itholic

@github-actions
Copy link
Copy Markdown

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!

@github-actions github-actions Bot added the Stale label Feb 10, 2025
@github-actions github-actions Bot closed this Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants