TYP: indexes #37224

jbrockmendel · 2020-10-18T18:15:48Z

…p-gb

WillAyd

A few comments

pandas/core/indexes/datetimelike.py

pandas/core/indexes/period.py

WillAyd · 2020-10-19T22:38:45Z

pandas/core/reshape/concat.py

@@ -463,6 +464,7 @@ def get_result(self):
            if self.bm_axis == 0:
                name = com.consensus_name_attr(self.objs)
                cons = self.objs[0]._constructor
+                assert issubclass(cons, Series)  # for mypy


Instead of the assert I think we can just annotate the _constructor property (?)

it already is annotated

I recall a discussion about this. There is no requirement that _constructor has to be type(self). This makes typing intractable in many places so we assume that _constructor is type(self).

Could this raise AssertionError for users with subclassed Series and DataFrames?

probably safer to use casts here. (anywhere where _constructor is involved)

Using cons = cast(Type[Series], cons) here leads to errors:

pandas/core/reshape/concat.py:467: error: Incompatible types in assignment (expression has type "Type[Series]", variable has type "Type[FrameOrSeries]") [assignment] pandas/core/reshape/concat.py:472: error: Unexpected keyword argument "index" for "NDFrame" [call-arg] pandas/core/generic.py:213: note: "NDFrame" defined here pandas/core/reshape/concat.py:472: error: Unexpected keyword argument "name" for "NDFrame" [call-arg] pandas/core/generic.py:213: note: "NDFrame" defined here pandas/core/reshape/concat.py:472: error: Unexpected keyword argument "dtype" for "NDFrame" [call-arg]

hmm, early in the history of adding type hints it was decided that Aliases should be TypeVars. This can cause many issues.

here it maybe that objs on L298 should use FrameOrSeriesUnion.

Anyway, working with the TypeVar for now. there are two issues.

mypy cannot perform type narrowing with if self._is_series

FrameOrSeries is bounded by NDFrame, so mypy cannot know that if not a Series it is a DataFrame (in the else).

Since you've added from pandas import DataFrame, Series could do...

diff --git a/pandas/core/reshape/concat.py b/pandas/core/reshape/concat.py index e54ad177b3..9dba9be0ff 100644 --- a/pandas/core/reshape/concat.py +++ b/pandas/core/reshape/concat.py @@ -3,7 +3,7 @@ Concat routines. """ from collections import abc -from typing import TYPE_CHECKING, Iterable, List, Mapping, Union, overload +from typing import TYPE_CHECKING, Iterable, List, Mapping, Type, Union, overload import numpy as np @@ -457,14 +457,15 @@ class _Concatenator: def get_result(self): from pandas import DataFrame, Series + cons: Type[FrameOrSeriesUnion] + # series only - if self._is_series: + if isinstance(self.objs[0], Series): # stack blocks if self.bm_axis == 0: name = com.consensus_name_attr(self.objs) cons = self.objs[0]._constructor - assert issubclass(cons, Series) # for mypy arrs = [ser._values for ser in self.objs] @@ -478,7 +479,6 @@ class _Concatenator: # GH28330 Preserves subclassed objects through concat cons = self.objs[0]._constructor_expanddim - assert issubclass(cons, DataFrame) # for mypy index, columns = self.new_axes df = cons(data, index=index) @@ -487,6 +487,7 @@ class _Concatenator: # combine block managers else: + assert isinstance(self.objs[0], DataFrame) # for mypy mgrs_indexers = [] for obj in self.objs: indexers = {}

Since you've added from pandas import DataFrame, Series could do...

casts would be my preference but also needs a few hoops to avoid Incompatible types in assignment with TypeVars.

diff --git a/pandas/core/reshape/concat.py b/pandas/core/reshape/concat.py index e54ad177b3..e894c2a373 100644 --- a/pandas/core/reshape/concat.py +++ b/pandas/core/reshape/concat.py @@ -3,7 +3,7 @@ Concat routines. """ from collections import abc -from typing import TYPE_CHECKING, Iterable, List, Mapping, Union, overload +from typing import TYPE_CHECKING, Iterable, List, Mapping, Type, Union, cast, overload import numpy as np @@ -30,7 +30,7 @@ import pandas.core.indexes.base as ibase from pandas.core.internals import concatenate_block_managers if TYPE_CHECKING: - from pandas import DataFrame + from pandas import Series, DataFrame from pandas.core.generic import NDFrame # --------------------------------------------------------------------- @@ -455,16 +455,17 @@ class _Concatenator: self.new_axes = self._get_new_axes() def get_result(self): - from pandas import DataFrame, Series + cons: Type[FrameOrSeriesUnion] + sample: FrameOrSeriesUnion # series only if self._is_series: + sample = cast("Series", self.objs[0]) # stack blocks if self.bm_axis == 0: name = com.consensus_name_attr(self.objs) - cons = self.objs[0]._constructor - assert issubclass(cons, Series) # for mypy + cons = sample._constructor arrs = [ser._values for ser in self.objs] @@ -477,8 +478,7 @@ class _Concatenator: data = dict(zip(range(len(self.objs)), self.objs)) # GH28330 Preserves subclassed objects through concat - cons = self.objs[0]._constructor_expanddim - assert issubclass(cons, DataFrame) # for mypy + cons = sample._constructor_expanddim index, columns = self.new_axes df = cons(data, index=index) @@ -487,6 +487,7 @@ class _Concatenator: # combine block managers else: + sample = cast("DataFrame", self.objs[0]) mgrs_indexers = [] for obj in self.objs: indexers = {} @@ -509,7 +510,7 @@ class _Concatenator: if not self.copy: new_data._consolidate_inplace() - cons = self.objs[0]._constructor + cons = sample._constructor return cons(new_data).__finalize__(self, method="concat") def _get_result_dim(self) -> int:

WillAyd · 2020-10-19T22:38:51Z

pandas/core/reshape/concat.py

@@ -476,6 +478,7 @@ def get_result(self):

                # GH28330 Preserves subclassed objects through concat
                cons = self.objs[0]._constructor_expanddim
+                assert issubclass(cons, DataFrame)  # for mypy


Same comment

…p-gb

simonjayhawkins · 2020-10-24T20:36:57Z

pandas/core/indexes/period.py

@@ -161,6 +161,24 @@ def to_timestamp(self, freq=None, how="start") -> DatetimeIndex:
        arr = self._data.to_timestamp(freq, how)
        return DatetimeIndex._simple_new(arr, name=self.name)

+    # error: Decorated property not supported  [misc]


also need to remove from inherit_names?

and maybe update comment on L151

updated comment; removing from inherit_names is a PITA since it isnt there directly. harmless to have it there.

…p-gb

simonjayhawkins · 2020-10-27T08:55:08Z

@jbrockmendel can you respond to #37224 (comment)

…p-gb

simonjayhawkins

Thanks @jbrockmendel lgtm

WillAyd · 2020-10-29T15:51:30Z

Thanks!

jbrockmendel added 3 commits October 17, 2020 17:33

TYP: plotting._matplotlib.converter

4668fb8

Merge branch 'master' of https://github.com/pandas-dev/pandas into ty…

26414f1

…p-gb

TYP: indexes

0e2c353

WillAyd reviewed Oct 19, 2020

View reviewed changes

WillAyd added the Typing type annotations, mypy/pyright type checking label Oct 19, 2020

jreback added this to the 1.2 milestone Oct 19, 2020

jbrockmendel mentioned this pull request Oct 22, 2020

TYP: is_dtype_compat #37340

Merged

jbrockmendel added 2 commits October 22, 2020 15:27

Merge branch 'master' of https://github.com/pandas-dev/pandas into ty…

f55eb60

…p-gb

Merge branch 'master' of https://github.com/pandas-dev/pandas into ty…

9e5c552

…p-gb

jbrockmendel mentioned this pull request Oct 23, 2020

TYP: DatetimeIndex, TimedeltaIndex #37365

Merged

jbrockmendel added 3 commits October 24, 2020 11:08

Merge branch 'master' of https://github.com/pandas-dev/pandas into ty…

1a10e74

…p-gb

use @doc for PeriodIndex props

7ef7c56

revert

96ed502

simonjayhawkins reviewed Oct 24, 2020

View reviewed changes

jbrockmendel added 4 commits October 25, 2020 08:20

dummy commit to force CI

ff2bcca

Merge branch 'master' of https://github.com/pandas-dev/pandas into ty…

9b1784d

…p-gb

update comment

a3c4b99

Merge branch 'master' of https://github.com/pandas-dev/pandas into ty…

05e2590

…p-gb

jbrockmendel added 2 commits October 27, 2020 08:20

Merge branch 'master' of https://github.com/pandas-dev/pandas into ty…

6e666ef

…p-gb

suggested updates

3ca562c

simonjayhawkins approved these changes Oct 29, 2020

View reviewed changes

WillAyd merged commit 5532ae8 into pandas-dev:master Oct 29, 2020

jbrockmendel deleted the typ-gb branch October 29, 2020 15:55

kesmit13 pushed a commit to kesmit13/pandas that referenced this pull request Nov 2, 2020

TYP: indexes (pandas-dev#37224)

324dd6c

ukarroum pushed a commit to ukarroum/pandas that referenced this pull request Nov 2, 2020

TYP: indexes (pandas-dev#37224)

dd8f701

simonjayhawkins mentioned this pull request Nov 10, 2020

CI: unpin numpy for CI / Checks github action #36092

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TYP: indexes #37224

TYP: indexes #37224

jbrockmendel commented Oct 18, 2020

WillAyd left a comment

WillAyd Oct 19, 2020

jbrockmendel Oct 19, 2020

simonjayhawkins Oct 23, 2020

simonjayhawkins Oct 23, 2020

simonjayhawkins Oct 23, 2020

jbrockmendel Oct 27, 2020

simonjayhawkins Oct 27, 2020 •

edited

Loading

simonjayhawkins Oct 27, 2020

WillAyd Oct 19, 2020

simonjayhawkins Oct 24, 2020

jbrockmendel Oct 29, 2020

simonjayhawkins commented Oct 27, 2020

simonjayhawkins left a comment

WillAyd commented Oct 29, 2020

TYP: indexes #37224

TYP: indexes #37224

Conversation

jbrockmendel commented Oct 18, 2020

WillAyd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

simonjayhawkins Oct 27, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

simonjayhawkins commented Oct 27, 2020

simonjayhawkins left a comment

Choose a reason for hiding this comment

WillAyd commented Oct 29, 2020

simonjayhawkins Oct 27, 2020 •

edited

Loading