Skip to content
This repository has been archived by the owner on Nov 22, 2022. It is now read-only.

Review Callable usage #125

Closed
zero323 opened this issue Feb 5, 2019 · 1 comment
Closed

Review Callable usage #125

zero323 opened this issue Feb 5, 2019 · 1 comment

Comments

@zero323
Copy link
Owner

zero323 commented Feb 5, 2019

At least in some cases we can replace Callable with more precise Protocols.

@zero323
Copy link
Owner Author

zero323 commented Aug 28, 2019

#193 notwithstanding we have following ellipsed Callable:

def registerFunction(self, name: str, f: Callable[..., Any], returnType: DataType = ...) -> None: ...

def registerFunction(self, name: str, f: Callable[..., Any], returnType: DataType = ...) -> None: ...

def register(self, name: str, f: Callable[..., Any], returnType: DataTypeOrString = ...) -> None: ...

func: Callable[..., Any]

def __init__(self, func: Callable[..., Any], returnType: DataTypeOrString, name: Optional[str] = ...) -> None: ...

def udf(f: Callable[..., Any], returnType: DataTypeOrString = ...) -> UserDefinedFunctionLike: ...

def transform(self, dstreams: List[DStream[Any]], transformFunc: Callable[..., RDD[T]]) -> DStream[T]: ...

where all, but the last one, are legacy udf related. Theoretically we can narrow things down a bit by enumerating all possible external types on argument and return type side (this wouldn't cover UDTs though), but it is probably not worth the effort, as application cannot be validated anyway.

The last option cannot be optimized, due to it's signature.

The remaining applications have been already addressed by 16c4a46

@zero323 zero323 closed this as completed Aug 28, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant