-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Handle untyped interfaces in _check
#18
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With this change, the following should be a failure (non-unique types):
import nnbench
@nnbench.benchmark
def increment(value):
return value + 1
@nnbench.benchmark
def double(value: int) -> int:
return value * 2
Is this expected? Are we fine with this?
(We should check that the resulting type in the error message is still sensible, though. Is it <class 'empty'>
by any chance?)
src/nnbench/runner.py
Outdated
if not param.annotation == inspect.Parameter.empty and not issubclass( | ||
param_types[name], param.annotation | ||
): # only check type match if type supplied |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if not param.annotation == inspect.Parameter.empty and not issubclass( | |
param_types[name], param.annotation | |
): # only check type match if type supplied | |
# skip the check if benchmark interface is untyped | |
if param.annotation == inspect.Parameter.empty: | |
continue | |
if not issubclass(param_types[name], param.annotation): |
It is |
It's fine for now. We can think about making it show up as |
This reverts commit bab9b63.
i would argue that your example above should throw an error. Untyped and typed might very well be non-unique types. We cannot handle different types for the same variable name and if untyped cannot ensure it is the right one. So it's either on the user to keep everything untyped and ensure the right value is passed. Or the absence of the type is an oversight in which case the error is a welcome reminder. |
Co-authored-by: Nicholas Junge <n.junge@appliedai.de>
Guard match checks of types in function signature and supplied via the
params
kwarg in thebenchmark
decorators_check
method against untyped function signatures.If untyped, we do not check for matching types. We also log a warning to the
stdout
of thelogger
indicating the untyped parameter and function name.