-
-
Notifications
You must be signed in to change notification settings - Fork 17.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TYP: indexes #37224
TYP: indexes #37224
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few comments
pandas/core/reshape/concat.py
Outdated
@@ -463,6 +464,7 @@ def get_result(self): | |||
if self.bm_axis == 0: | |||
name = com.consensus_name_attr(self.objs) | |||
cons = self.objs[0]._constructor | |||
assert issubclass(cons, Series) # for mypy |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of the assert I think we can just annotate the _constructor
property (?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it already is annotated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I recall a discussion about this. There is no requirement that _constructor has to be type(self). This makes typing intractable in many places so we assume that _constructor is type(self).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could this raise AssertionError for users with subclassed Series and DataFrames?
probably safer to use casts here. (anywhere where _constructor is involved)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using cons = cast(Type[Series], cons)
here leads to errors:
pandas/core/reshape/concat.py:467: error: Incompatible types in assignment (expression has type "Type[Series]", variable has type "Type[FrameOrSeries]") [assignment]
pandas/core/reshape/concat.py:472: error: Unexpected keyword argument "index" for "NDFrame" [call-arg]
pandas/core/generic.py:213: note: "NDFrame" defined here
pandas/core/reshape/concat.py:472: error: Unexpected keyword argument "name" for "NDFrame" [call-arg]
pandas/core/generic.py:213: note: "NDFrame" defined here
pandas/core/reshape/concat.py:472: error: Unexpected keyword argument "dtype" for "NDFrame" [call-arg]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm, early in the history of adding type hints it was decided that Aliases should be TypeVars. This can cause many issues.
here it maybe that objs
on L298 should use FrameOrSeriesUnion.
Anyway, working with the TypeVar for now. there are two issues.
- mypy cannot perform type narrowing with
if self._is_series
- FrameOrSeries is bounded by NDFrame, so mypy cannot know that if not a Series it is a DataFrame (in the else).
Since you've added from pandas import DataFrame, Series
could do...
diff --git a/pandas/core/reshape/concat.py b/pandas/core/reshape/concat.py
index e54ad177b3..9dba9be0ff 100644
--- a/pandas/core/reshape/concat.py
+++ b/pandas/core/reshape/concat.py
@@ -3,7 +3,7 @@ Concat routines.
"""
from collections import abc
-from typing import TYPE_CHECKING, Iterable, List, Mapping, Union, overload
+from typing import TYPE_CHECKING, Iterable, List, Mapping, Type, Union, overload
import numpy as np
@@ -457,14 +457,15 @@ class _Concatenator:
def get_result(self):
from pandas import DataFrame, Series
+ cons: Type[FrameOrSeriesUnion]
+
# series only
- if self._is_series:
+ if isinstance(self.objs[0], Series):
# stack blocks
if self.bm_axis == 0:
name = com.consensus_name_attr(self.objs)
cons = self.objs[0]._constructor
- assert issubclass(cons, Series) # for mypy
arrs = [ser._values for ser in self.objs]
@@ -478,7 +479,6 @@ class _Concatenator:
# GH28330 Preserves subclassed objects through concat
cons = self.objs[0]._constructor_expanddim
- assert issubclass(cons, DataFrame) # for mypy
index, columns = self.new_axes
df = cons(data, index=index)
@@ -487,6 +487,7 @@ class _Concatenator:
# combine block managers
else:
+ assert isinstance(self.objs[0], DataFrame) # for mypy
mgrs_indexers = []
for obj in self.objs:
indexers = {}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since you've added
from pandas import DataFrame, Series
could do...
casts would be my preference but also needs a few hoops to avoid Incompatible types in assignment
with TypeVars.
diff --git a/pandas/core/reshape/concat.py b/pandas/core/reshape/concat.py
index e54ad177b3..e894c2a373 100644
--- a/pandas/core/reshape/concat.py
+++ b/pandas/core/reshape/concat.py
@@ -3,7 +3,7 @@ Concat routines.
"""
from collections import abc
-from typing import TYPE_CHECKING, Iterable, List, Mapping, Union, overload
+from typing import TYPE_CHECKING, Iterable, List, Mapping, Type, Union, cast, overload
import numpy as np
@@ -30,7 +30,7 @@ import pandas.core.indexes.base as ibase
from pandas.core.internals import concatenate_block_managers
if TYPE_CHECKING:
- from pandas import DataFrame
+ from pandas import Series, DataFrame
from pandas.core.generic import NDFrame
# ---------------------------------------------------------------------
@@ -455,16 +455,17 @@ class _Concatenator:
self.new_axes = self._get_new_axes()
def get_result(self):
- from pandas import DataFrame, Series
+ cons: Type[FrameOrSeriesUnion]
+ sample: FrameOrSeriesUnion
# series only
if self._is_series:
+ sample = cast("Series", self.objs[0])
# stack blocks
if self.bm_axis == 0:
name = com.consensus_name_attr(self.objs)
- cons = self.objs[0]._constructor
- assert issubclass(cons, Series) # for mypy
+ cons = sample._constructor
arrs = [ser._values for ser in self.objs]
@@ -477,8 +478,7 @@ class _Concatenator:
data = dict(zip(range(len(self.objs)), self.objs))
# GH28330 Preserves subclassed objects through concat
- cons = self.objs[0]._constructor_expanddim
- assert issubclass(cons, DataFrame) # for mypy
+ cons = sample._constructor_expanddim
index, columns = self.new_axes
df = cons(data, index=index)
@@ -487,6 +487,7 @@ class _Concatenator:
# combine block managers
else:
+ sample = cast("DataFrame", self.objs[0])
mgrs_indexers = []
for obj in self.objs:
indexers = {}
@@ -509,7 +510,7 @@ class _Concatenator:
if not self.copy:
new_data._consolidate_inplace()
- cons = self.objs[0]._constructor
+ cons = sample._constructor
return cons(new_data).__finalize__(self, method="concat")
def _get_result_dim(self) -> int:
pandas/core/reshape/concat.py
Outdated
@@ -476,6 +478,7 @@ def get_result(self): | |||
|
|||
# GH28330 Preserves subclassed objects through concat | |||
cons = self.objs[0]._constructor_expanddim | |||
assert issubclass(cons, DataFrame) # for mypy |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same comment
@@ -161,6 +161,24 @@ def to_timestamp(self, freq=None, how="start") -> DatetimeIndex: | |||
arr = self._data.to_timestamp(freq, how) | |||
return DatetimeIndex._simple_new(arr, name=self.name) | |||
|
|||
# error: Decorated property not supported [misc] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also need to remove from inherit_names?
and maybe update comment on L151
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
updated comment; removing from inherit_names is a PITA since it isnt there directly. harmless to have it there.
@jbrockmendel can you respond to #37224 (comment) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @jbrockmendel lgtm
Thanks! |
black pandas
git diff upstream/master -u -- "*.py" | flake8 --diff