Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FEAT-#7146: Use BaseQueryCompiler, BasePandasDataset, DataFrame or Series type hints at a high level #7147

Merged
merged 2 commits into from
Apr 4, 2024
Merged

FEAT-#7146: Use BaseQueryCompiler, BasePandasDataset, DataFrame or Series type hints at a high level #7147

merged 2 commits into from
Apr 4, 2024

Conversation

anmyachev
Copy link
Collaborator

@anmyachev anmyachev commented Apr 3, 2024 •
edited
Loading

What do these changes do?

To facilitate navigation, allows to automatically find places where functions and attributes are defined.

  • first commit message and PR title follow format outlined here

    NOTE: If you edit the PR title to match this format, you need to add another commit (even if it's empty) or amend your last commit for the CI job that checks the PR title to pick up the new PR title.

  • passes flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
  • passes black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
  • signed commit with git commit -s
  • Resolves Use BaseQueryCompiler, BasePandasDataset, DataFrame or Series type hints at a high level #7146
  • tests added and passing
  • module layout described at docs/development/architecture.rst is up-to-date

github-advanced-security[bot]
github-advanced-security bot found potential problems Apr 3, 2024
View reviewed changes
modin/pandas/groupby.py Fixed Show fixed Hide fixed
modin/pandas/indexing.py Fixed Show fixed Hide fixed
@anmyachev anmyachev force-pushed the issue7146 branch 4 times, most recently from 1ac15a6 to 84dc9d1 Compare April 3, 2024 18:12
@anmyachev
FEAT-modin-project#7146: Use BaseQueryCompiler, BasePandasDataset, Da…
7855110 
…taFrame or Series type hints at a high level

Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@anmyachev
fix mypy
9170401 
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
@anmyachev anmyachev marked this pull request as ready for review April 3, 2024 18:44
YarShev
YarShev reviewed Apr 3, 2024
View reviewed changes
modin/distributed/dataframe/pandas/partitions.py
@@ -90,7 +90,7 @@ def unwrap_partitions(
f"Only API Layer objects may be passed in here, got {type(api_layer_object)} instead."
)

modin_frame = api_layer_object._query_compiler._modin_frame
modin_frame = api_layer_object._query_compiler._modin_frame # type: ignore[attr-defined]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is # type: ignore[attr-defined] needed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mypy requires that attribute _modin_frame be specified for _query_compiler.

modin/pandas/accessor.py
@@ -208,7 +215,9 @@ class ModinAPI:
Object to operate on.
"""

def __init__(self, data):
_data: Union[DataFrame, Series]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did this become a class attribute?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a practice that allows to specify exactly what attributes all instances have and, in addition, makes it possible to specify its type.

What is written here applies to all of your similar comments.

modin/pandas/accessor.py
@@ -45,20 +51,21 @@ class BaseSparseAccessor(ClassLogger):
Object to operate on.
"""

_parent: Union[DataFrame, Series]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did this become a class attribute?

modin/pandas/base.py
@@ -174,6 +177,7 @@ class BasePandasDataset(ClassLogger):
# Pandas class that we pretend to be; usually it has the same name as our class
# but lives in "pandas" namespace.
_pandas_class = pandas.core.generic.NDFrame
_query_compiler: BaseQueryCompiler
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this as a class attribute?

modin/pandas/groupby.py
Comment on lines +94 to +95
_df: Union[DataFrame, Series]
_query_compiler: BaseQueryCompiler
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need these as class attributes?

modin/pandas/iterator.py
@@ -30,7 +36,9 @@ class PartitionIterator(Iterator):
The function to get inner iterables from each partition.
"""

def __init__(self, df, axis, func):
df: DataFrame
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this as a class attribute?

modin/pandas/resample.py
Comment on lines +37 to +38
_dataframe: Union[DataFrame, Series]
_query_compiler: BaseQueryCompiler
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

modin/pandas/series_utils.py
Comment on lines +42 to +43
_series: Series
_query_compiler: BaseQueryCompiler
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

@YarShev YarShev merged commit c1865b8 into modin-project:master Apr 4, 2024
37 checks passed
@anmyachev anmyachev deleted the issue7146 branch April 4, 2024 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@YarShev YarShev YarShev approved these changes

@aregm aregm Awaiting requested review from aregm aregm is a code owner

@gshimansky gshimansky Awaiting requested review from gshimansky gshimansky is a code owner

@ienkovich ienkovich Awaiting requested review from ienkovich ienkovich is a code owner

@Garra1980 Garra1980 Awaiting requested review from Garra1980 Garra1980 is a code owner

@vnlitvinov vnlitvinov Awaiting requested review from vnlitvinov vnlitvinov is a code owner

@dchigarev dchigarev Awaiting requested review from dchigarev dchigarev is a code owner

@AndreyPavlenko AndreyPavlenko Awaiting requested review from AndreyPavlenko AndreyPavlenko is a code owner

@devin-petersohn devin-petersohn Awaiting requested review from devin-petersohn devin-petersohn is a code owner

@mvashishtha mvashishtha Awaiting requested review from mvashishtha mvashishtha is a code owner

@RehanSD RehanSD Awaiting requested review from RehanSD RehanSD is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Use BaseQueryCompiler, BasePandasDataset, DataFrame or Series type hints at a high level
2 participants
@anmyachev @YarShev