-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
missing implementations on SpatialMaxPooling_updateGradInput #12
Comments
Awesome!
I sort of implement stuff as and when it becomes necessary. If you have a moment to implement one of the commented out cases that would be great! :-)
In max-pooling, we take a square of pooling width ( When we back-propagate, we take the gradient from the gradOutput pixel, and send it back to the one that produced the The pooling areas can be exactly contiguous, which is the easy situation. In this case each gradOutput value maps to exactly one gradInput pixel. Well, that's always true. But the point is, each gradInput pixel will take input from exactly 0 or 1 gradOutput pixels. But rather than making the pooling areas contiguous, we can make them overlap. And then some gradInput pixels need to be updated from the sum of several gradOutput pixels. This is a bit tricky to do, if done from parallel threads, so we consider this case separately. The stride, Actually, you might not need to handle this, because:
Spatial max pooling is over a 2d image, so that is 2 dimensions, Currently, implementation is done for mini-batches, but not for single non-batched examples.
As far as this bit:
To handle overlapping pools, I noticed that mostly we only have
But no more than 2 pools overlap for any input/gradInput pixel. So, we can do the pooling in two batches. In the first batch, we do the odd pools:
These dont overlap :-) Then we do the even pools:
... dont overlap either. And we add their results to the gradInput results from the first batch of pools. In 2 dimensions, we'll need 4 such batches, like we will have these pools first:
(where 'X' are the pools we are calculating, and '.' are the ones we skip for now) Next batch will be:
Then:
Finally
So, 4 times. This will actually generalize to any case where the pooling size is no smaller than half the stride, or something like this, hence that weird looking It would be easy enough to handle the fully general case though. eg, if stride is 1, and pooling size is 3, then we'd simply need to do the backpropagation 9 times, adding the results to the output of the earlier updateGradInput batches. |
@hughperkins |
@fmassa Ah, good info. Thanks! |
Hi Gustavo, Several people in the neural-style project encountered the same issue, so I've copy/pasted an implementation for contiguous non-batched pools, and if you update your clnn, In terms of testing clnn, I basically run the following tests currently, on a single OS (ubuntu 14.04), and a single GPU (NVIDIA 940M). This is quite quick to do, so please feel free to change things in whatever way you think is beautiful :-) Ideally not diverging too much from cunn, but I've changed tons of stuff, so fairly flexible. The tests I do are:
|
Hi @hughperkins, thanks for the detailed explanation. Cool, one of the pieces of code I was trying to get running with I'll try this week to get it running with some copy/pasting (and or factorization). Otherwise, re-porting the |
Cool :-) By the way, if you re-port cunn code, you might want to try the following:
This will show you the diff between automatically ported cunn files (in |
Hi Gustavo, please note that cunn spatialmaxpooling has been ported across now, since |
Oh, sweet. I was in the middle of the process porting the kernel from |
Ok. I can close this issue, right? |
Yep, thanks a lot! |
I'm porting some cunn code over to clnn and stumbled over the following error:
Browsing through
SpatialMaxPooling.cpp
I found out that there's actually some commented code just above the line that throws that exception. Is there any plans on implementing those cases?Also, some CNN noob questions: what those cases stand for? I'd be happy to implement those if anyone points out some reference to what actually this method is doing.
The text was updated successfully, but these errors were encountered: