implement double backwards for MaxPool3d #5328
Conversation
|
Could you run some benchmarks to see if the memory usage increase from changing MaxUnpool3d to not pack indices is significant? I think someone (@apaszke?) mentioned Resnet as a network that uses MaxUnpool3d; it would be nice to see if the memory usage increases significantly or not under resnet. edit: Ignore this comment, the memory usage is the same |
|
Oh, resnets use the 2d version, and I thought we’re modifying that. Will this make 3d consistent with the other pooling layers? |
|
Yeah this makes it more consistent. Also, I don't see any reason for this to use more memory than the old implementation. The way indices are being stored is changed, but we're still using the same number of bits in the tensor. |
| THIndex_t idx = k*oT*oW*oH + oH*oW*(start_t+maxz) + oW*(start_h+maxy) + (start_w+maxx); | ||
| if (start_t+maxz<0 || start_h+maxy<0 || start_w+maxx<0 || start_t+maxz>=oT | ||
| || start_h+maxy>=oH || start_w+maxx>=oW) | ||
| maxp = ind_p_k[ti * iH * iW + i * iW + j] - TH_INDEX_BASE; /* retrieve position of max */ |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| THIndex_t *ind_p_k = ind_p + k * iT * iH * iW; | ||
|
|
||
| int ti, i, j; | ||
| THIndex_t maxp; | ||
| for (ti = 0; ti < iT; ti++) |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| THIndex_t *ind_p_k = ind_p + k * iT * iH * iW; | ||
|
|
||
| int ti, i, j; | ||
| THIndex_t maxp; | ||
| for (ti = 0; ti < iT; ti++) |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
|
|
||
| if (maxp != -1) { | ||
| /* update gradient */ | ||
| gradInput_p_k[maxp] += gradOutput_p_k[index]; |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| @@ -127,6 +127,7 @@ static void THNN_(VolumetricDilatedMaxPooling_updateOutput_frame)( | |||
| { | |||
| /* loop over output */ | |||
| int64_t i, j, ti; | |||
| real *ip = input_p + k * itime * iwidth * iheight; | |||
| for (ti = 0; ti < otime; ti++) | |||
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| { | ||
| for (x = 0; x < kernel_w; x++) | ||
| for (x = start_w; x < end_w; x += dilationW) |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| @@ -381,16 +372,13 @@ static void THNN_(VolumetricDilatedMaxPooling_updateGradInput_frame)( | |||
| for (j = 0; j < owidth; j++) | |||
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
|
|
||
| int t, i, j, index; | ||
| THIndex_t maxp; | ||
| for (t = 0; t < iT; t++) | ||
| { | ||
| for (i = 0; i < iH; i++) | ||
| { | ||
| for (j = 0; j < iW; j++) |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
|
|
||
| int t, i, j, index; | ||
| THIndex_t maxp; | ||
| for (t = 0; t < iT; t++) | ||
| { | ||
| for (i = 0; i < iH; i++) | ||
| { | ||
| for (j = 0; j < iW; j++) |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
fixes #4497.
test plan: added double backwards check for maxpool3d tests