Describe the bug
Here:
|
_R = TypeVar("_R", bound=pa.DataType) |
we bind _R to be a subtype of pyarrow.DataType
And in these two places we require that functions return a value of type _R (ie. they should return a pyarrow datatype):
|
def __init__( |
|
self, |
|
name: str, |
|
func: Callable[..., _R], |
|
input_fields: list[pa.Field], |
|
return_field: _R, |
|
volatility: Volatility | str, |
|
) -> None: |
|
@overload |
|
@staticmethod |
|
def udf( |
|
func: Callable[..., _R], |
|
input_fields: Sequence[pa.DataType | pa.Field] | pa.DataType | pa.Field, |
|
return_field: pa.DataType | pa.Field, |
|
volatility: Volatility | str, |
|
name: str | None = None, |
|
) -> ScalarUDF: ... |
which seems incorrect, as functions should return arrays that contain values of that type
To Reproduce
datafusion-python ((53.0.0)) $ mypy examples/python-udf.py
examples/python-udf.py:27: error: Value of type variable "_R" of function cannot be "Array[Any]" [type-var]
Found 1 error in 1 file (checked 1 source file)
Expected behavior
mypy should pass
Additional context
Describe the bug
Here:
datafusion-python/python/datafusion/user_defined.py
Line 36 in e0284c6
we bind
_Rto be a subtype ofpyarrow.DataTypeAnd in these two places we require that functions return a value of type
_R(ie. they should return a pyarrow datatype):datafusion-python/python/datafusion/user_defined.py
Lines 123 to 130 in e0284c6
datafusion-python/python/datafusion/user_defined.py
Lines 166 to 174 in e0284c6
which seems incorrect, as functions should return arrays that contain values of that type
To Reproduce
Expected behavior
mypy should pass
Additional context