[call-overload] `Series.name` is incompatible with `DataFrame.__getitem__`

[randolf-scholz]

Describe the bug

It should be type-safe to put Series.name into DataFrame.__getitem__, but Series.name is defined as None | Hashable and DataFrame.__getitem__ does not accept Hashable.

pandas-stubs/pandas-stubs/core/frame.pyi

Lines 510 to 523 in d2e3f74

	@overload
	def __getitem__(self, idx: Scalar | tuple[Hashable, ...]) -> Series: ...
	@overload
	def __getitem__(self, rows: slice) -> DataFrame: ...
	@overload
	def __getitem__(
	self,
	idx: Series[_bool]
	| DataFrame
	| Index
	| np_ndarray_str
	| np_ndarray_bool
	| list[_ScalarOrTupleT],
	) -> DataFrame: ...

pandas-stubs/pandas-stubs/core/series.pyi

Lines 326 to 327 in d2e3f74

	@property
	def name(self) -> Hashable | None: ...

To Reproduce

from pandas import DataFrame, Series

foo = Series([1, 2, 3], name="foo")
bar = DataFrame(columns=["foo", "bar", "baz"])
bar[foo.name]  # [call-overload]
# No overload variant of "__getitem__" of "DataFrame" matches argument type "Hashable" 

Please complete the following information:

• pandas 1.5.3
• pandas-stubs 1.5.3.230304

[Dr-Irv]

We can change

 def __getitem__(self, idx: Scalar | tuple[Hashable, ...]) -> Series: ... 

to

 def __getitem__(self, idx: Hashable| tuple[Hashable, ...]) -> Series: ... 

but you'd still have to write bar[cast(Hashable, foo.name)] because foo.name could be None

[randolf-scholz]

@Dr-Irv None is an instance of Hashable so it should work without casting.

[Dr-Irv]

@Dr-Irv None is an instance of Hashable so it should work without casting.

Fair point - so we should change the return type of Series.name as well.

[randolf-scholz]

@Dr-Irv From a semantic point of view I think it makes sense to still type hint it as Optional[Hashable] because it conveys important information in that there is a default value (usually None itself / or None. by convention). This is why I personally still prefer Optional[T] in such cases instead of None | T. The latter I use when e.g. a function needs an arg that could be None, but has no default.

[Dr-Irv]

@Dr-Irv From a semantic point of view I think it makes sense to still type hint it as Optional[Hashable] because it conveys important information in that there is a default value (usually None itself / or None. by convention).

That makes sense, and I think my comment above was incorrect about requiring a cast, so this is then pretty easy to fix as suggested #570 (comment)

This is why I personally still prefer Optional[T] in such cases instead of None | T. The latter I use when e.g. a function needs an arg that could be None, but has no default.

Both the pandas source and pandas-stubs have made a unilateral decision to use sometype | None versus Optional[sometype]

[Dr-Irv]

This seems to be fixed as of version 2.0.1.230501.