ND Crop layer

[BlGene]

Something along the lines of what a ND crop layer would look like.
Based on: PR #1976 -> PR #3565 -> this

Todo

• Test for D > 4
• Test backwards
• GPU kernel copy instead of memcopy (?)
• const indices and offsets
• GPU backwards/forwards
• CPU backwards
• parameter size checks
• lint

Please see PR #3613 for how to set the crop parameters.

[BlGene]

@longjon So, my attempt at a ND crop layer, if this is approximately what you are looking for I can try to fix the GPU part.

Edit: For the GPU part just implementing the 4D case would seem like a reasonable compromise.

[longjon]

Actually I'm confused about the parameter interface here... what I had in mind is something mirroring the conv layer interface, with a scalar (i.e., optional) param axis alongside a repeated param of crop values (as you have currently).

The way it would work would be the same way conv layer works: if only one crop value is specified, all the dimensions after the specified axis would be cropped (by the same amount); otherwise the number of crop values specified must be equal to the number of dimensions following axis, and the trailing dimensions would be cropped accordingly.

The reason for doing it this way is because the crop values need to be determined from the net specification, and the net specification doesn't know how many dimensions the data will have! If the layers we are "cropping through" include conv/pool layers with non-singleton kernel sizes, then we know exactly how many dimensions to crop; but if not, all we can do is propagate the axis param and determine how many dimensions to crop at run time. (At least, I think that should work.)

It would be nice to have an ND GPU kernel, but it's fine to punt that to get the basic functionality in.

Thanks!

[BlGene]

@longjon The parameters should do what you want now.

I substituted the copy kernel with repeated calls to caffe_copy, which if I understood correctly will call cudaMemcpy, and thus perform the copy operation on the GPU. My intuition is that calling this thousands of times is probably less efficient than writing a custom kernel.

[longjon]

The parameters look right now.

caffe_copy will perform the copy on the GPU, but (as you suggest) has overhead -- even in the 2D case I found the overhead unacceptable, which is why the kernel exists. A fine solution for now would be to keep the existing kernel around, and use it only for the innermost two dimensions, looping around the outer ones. That should keep the performance the same in the 2D case while supporting ND.

[BlGene]

@longjon The crop_copy_gpu function now calls the kernel for the innermost two dimensions. This should hopefully fix the speed problem.

[bjosv79]

It seems to a few problems with the backward pass of crop_layer.cu:

• A typo src_outer_stride instead of src_inner_stride in line 94.
• Incorrect number of elements in caffe_gpu_set in line 89.
• Is the caffe_gpu_set really necessary? It is already done in line 119?

[BlGene]

@bjosv79 Thanks. All those things seemed correct, I changed them ( but haven't tested yet ).

[shelhamer]

Needs ID++ after #3211 -- sorry for the rebase.

[shelhamer]

@BlGene I pushed a rebase on the latest master to my fork https://github.com/shelhamer/caffe/tree/crop-nd with a few other additions -- feel free to integrate my changes if the commits save you time.

[shelhamer]

Let's make axis mirror the axis arg of other layers to allow for negative indexing through an int32 field and the CanonicalAxisIndex() helper method? See for instance https://github.com/BVLC/caffe/blob/master/src/caffe/layers/flatten_layer.cpp#L12-L13

[BlGene]

@shelhamer OK, I fixed the commit messages.

I'm happy to have FCN functionality merged too. Thanks for the attention to detail. 👍

[elezar]

Just a quick question from my side, can the ND crop be used to implement a rectangular crop as in #1980?

[BlGene]

This layer crops blob B to have the same shape as Blob A. So its a bit hacky, but you could create a dummy blob (A) that has the desired shape and crop the (B) blob to this size.

BR, Max