-
Notifications
You must be signed in to change notification settings - Fork 18.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Generalize bilinear interpolation filler to N-D multilinear/multicubic/lanczos filler #4198
base: master
Are you sure you want to change the base?
Generalize bilinear interpolation filler to N-D multilinear/multicubic/lanczos filler #4198
Conversation
add base class InterpolationFillerBase that implements common interpolation function calculations the derived fillers support N dimensions and different integer scaling factors per dimension change 'linear' filler to 'multilinear' filler add generic interpolation weight filler add base class InterpolationFillerBase that implements common interpolation function calculations change 'linear' filler to 'multilinear' filler add 'cubic', 'lanczos2', 'lanczos3' and 'lanczos4' fillers all fillers support N dimensions and different integer scaling factors per dimension remove everything except multilinear
This jupyter notebook tests the new fillers: |
Can this PR be used to downsample a feature map with a fractional factor? |
@Coldmooon The fillers can also be used to downsample a feature map with fractional factors (1/2, 1/3, 1/4,...), as they work with Deconvolution and Convolution layers, where the strides define the up- and downsampling factors, respectively. |
@christianpayer Thank you very much~! Recently, I've been studying resampling by caffe. Your PR helps me a lot. I modified
Using
A further question is what if non-integer factor? For example, 1.4286 for upsample and 0.7143 for downsample. Fractional stride will be rounded down in caffe. In these cases, I use another way to compute
|
Really nice work! Could you add some unit tests, following the existing examples in |
@Coldmooon Your changes for downsampling look OK. You are right, there is a difference between the downsampling of this PR and the reference created with skimage. I don't know exactly, how skimage performs downsampling, as it can be implemented in multiple ways. Even if you compare the outputs of skimage's linear/cubic resampling and opencv's, you will see many differences. Regarding your question of non-integer factors, this is not possible with a caffe convolution. With an integer factor, you will get the same kernel for every x/y-coordinate of the image. With non-integer factors, this is not the case. You would need different kernels for different coordinates of the image, which is not possible with the convolution layer. If you want to get more insights into this, I suggest you to debug Matlab's imresize and look for the function 'contributions' and compare the outputs for integer and non-integer factors. |
@ajtulloch Thanks! I can add some unit test, but I don't know, what should be tested. I could check, whether the kernels sum up to a fixed value. Or I could compare with hardcoded weights (possibly created from Matlab?) and check for differences. |
@christianpayer I tested upsampling and downsampling with non-integer factors. The resampled images for the two cases look like noise and look like the downsampled image with an integer factor but without appending the Maybe it's possible to implement a new |
This branch implements an n-dimensional generalization of the bilinear filler (#2213) and adds cubic and lanczos fillers.
It makes #3984 obsolete, as it uses a new common base class for all interpolation fillers. The base class InterpolationFillerBase calculates the weight values by calling the virtual interpolation functions of its derived classes. This PR implements linear, cubic and lanczos interpolation fillers, while additional interpolation functions (e.g. Hermite, Mitchell, Gaussian) may be implemented easily by deriving the class InterpolationFillerBase and implementing the virtual functions f() and support().
I have written a simple test script in python, which tests the fillers with various scaling factors in x and y. I will upload it after some cleanup.
If you have comments/suggestions, let me know!