New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make lasagne.utils.create_param() get callable that returns Theano expressions or shared variables #695
Conversation
BTW, I have a question about naming in the PR that isn't yet resolved. What should I do with the name parameter if the callable method returns a shared variable or a Theano expression? |
I have no idea why there are too many failures 😕 |
There seem to be a bunch of failures, and not just in tests for Regarding the name parameter: our assumption is that, when a shared variable is passed in, it may or may not already have been named. If it already has a name, we do not want to overwrite it, so we do nothing. We could technically check if the specified variable has a name, and then insert a name if it doesn't have one, but that would get a bit hairy, I think. Also it wouldn't work for expressions. |
I have only run the test locally for the parts of the code that I have changed. Maybe the failures are affected by the update to the Yes, it's written in the comments. My question was specifically for the case that a callable has been passed to the |
@benanne One particular thing I encountered when running tests locally vs. Travis CI is that some tests won't pass when Theano is configured to use GPU. What should be done with these test? Some of the tests that previously were in the repository (not added by this PR) are prone to this problem, too. |
@benanne I wonder how these two tests passed before:
In both tests, a scalar is passed to Now the build failures are down to these two and a doctest error on the EmbeddingLayer ( |
You changed the behaviour of if spec is callable:
spec = spec(shape)
if spec is Python scalar:
spec = floatX(spec)
if spec is numpy array:
check for correct shape
spec = theano.shared(spec)
if spec is theano variable:
check for correct dimensionality
add name if unset
else:
complain that the spec is unsupported Note that the Instead of setting a boolean flag to spit out the correct error message in the end, you could gradually build up a prefix string for the error message inside the if cases ("the numpy array", "the Theano variable", "the numpy array returned by the callable", ...). |
Ok, @f0k thanks for the suggestions. All tests pass now except the |
I can't reproduce the doctest failure in |
spec = spec(shape) | ||
callable_prefix = "cannot initialize parameters: the provided " \ | ||
"callable did not return a value with the " \ | ||
"correct shape or dimensions." |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just to make the errors a little nicer:
err_prefix = "cannot initialize parameter %s: " % name
if callable(spec):
spec = spec(shape)
err_prefix += "the %s returned by the provided callable"
else:
err_prefix += "the provided %s"
...
if isinstance(spec, np.ndarray):
if spec.shape != shape:
raise ValueError("%s has shape %s, should be %s") % (err_prefix % "numpy array", spec.shape, shape)
...
if isinstance(spec, theano.Variable):
if spec.ndim != len(shape):
raise ValueError("%s has %d dimensions, should be %d" % (err_prefix % "Theano variable", spec.ndim, len(shape))
else:
if "callable" in err_prefix:
raise TypeError("%s is not a numpy array or a Theano expression" % (err_prefix % "value"))
else:
raise TypeError("%s is not a numpy array, a Theano expression, or a callable" % (err_prefix % "value"))
Note that I've changed RuntimeError
to ValueError
or TypeError
(we need to clean this up elsewhere as well), the tests need to change accordingly.
Changed the error handling to what @f0k suggested. Tests are also updated accordingly and all pass. Only the |
""" | ||
shape = tuple(shape) # convert to tuple if needed | ||
if any(d <= 0 for d in shape): | ||
raise ValueError(( | ||
"Cannot create param with a non-positive shape dimension. " | ||
"Tried to create param with shape=%r, name=%r") % (shape, name)) | ||
|
||
err_prefix = "cannot initialize parameter %s: " % name | ||
if hasattr(spec, '__call__'): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd prefer if callable(spec)
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok. I'll change this, too.
This is still a mystery. Does the new |
err_prefix += "the provided %s" | ||
|
||
if np.isscalar(spec): | ||
spec = floatX(spec) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
np.isscalar("asdf")
returns True. We need to explicitly check for numbers.
All tests are passing now. @f0k could you please review the code once more? |
err_prefix += "the provided %s" | ||
|
||
if isinstance(spec, numbers.Number): | ||
spec = np.asarray(spec) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice find! This seems slightly too strict, though (I'm sorry that this PR turns out to be much more intricate than it seemed at first glance):
>>> x = np.float32(123)
>>> isinstance(x, numbers.Number)
False
Maybe:
if isinstance(spec, numbers.Number) or isinstance(spec, np.generic) and spec.dtype.kind in 'biufc':
Although this is getting silly. Let's leave it at what you have.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(I'm sorry that this PR turns out to be much more intricate than it seemed at first glance)
No, it's ok. Sorry that I have been slow in implementing the correct code. This is my first PR for Lasagne 😄
>>> x = np.float32(123) >>> isinstance(spec, numbers.Number) False
@f0k I think this code isn't completely correct. numbers.Number
in fact correctly handles the case you have provided.
>>> x = np.float32(123)
>>> isinstance(x, numbers.Number)
True
As far as I tested multiple cases, numbers.Number
correctly handles scalars of multiple types and kinds.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this code isn't completely correct.
numbers.Number
in fact correctly handles the case you have provided.
Interesting. Not for me on Python 2.7.6 or 3.4.2, numpy 1.8.2.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You've passed spec
as argument to isinstance
, instead of x
. I'm working on Python 2.7.11, NumPy 1.11.0.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, I had just written it wrongly, I passed x
. It depends on the numpy/Python version than. In any case, it indicates we don't need to do such a stretch here and can keep it as you have it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But it won't work correctly in those combinations of NumPy/Python.
So I should keep them as is?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But it won't work correctly in those combinations of NumPy/Python.
It's a corner case. Using scalars was not actually documented, it just happened to be used in one of the tests. Using numpy scalars instead of Python scalars is even more undocumented.
So I should keep them as is?
You can also add my modified if
statement, as you please. Either way is fine by me!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I think that's a more robust solution. I'll update the if statement.
Either of the following: | ||
|
||
* a numpy array with the initial parameter values | ||
* an scalar or a numpy array with the initial parameter values |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
*a scalar (word starts with consonant)
Very cool! Just three minor requests. In the end, could you please squash this into a single commit? It should go like this:
|
@f0k squashed as you said, and all tests are passing. |
Great, thanks a lot, everything looks fine! Will merge in the evening if there are no further comments. |
Merged, thanks again! |
👍 |
This issues comments raised in #693