Upscale2D or Conv2D produces NaN values

[nkoppel]

My application using dfdx: https://github.com/nkoppel/generals-bot is currently failing to train a unet to play generals.io because the policy network produces NaN values during training. This is probably a bug in Conv2D or Upscale2D, as the NaN values only occupy a small area of the output, and this program has successfully trained networks without Upscale2D. The NaN values occur about every 100-500 batches, and appear only in the last image of a batch, in the following shape:

0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 NaN
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 NaN
NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN

(along the bottom and the bottom 3 of the right edge, for all 4 output channels)

Due to its relative rarity and the consistency of the shape of the NaN values, I suspect that this is a bug due to Conv2D or Upscale2D reading uninitialized memory or failing to write to uninitialized memory, but I have been unable to locate the bug thus far. It is probably more likely to be a bug in Upscale2D, because I have tried using the new cudnn implementation of Conv2D, and experienced the exact same behavior.

This error is reproduced here: https://gist.github.com/nkoppel/db07f684222a7169d2c39b205542d4c5. I have no idea why, but using Sgd or inspecting the value of out both prevent the bug from occurring.

[coreylowman]

Is this with cudnn or cuda?

Edit: nvm i see it happens with both

[coreylowman]

Do you happen to have the image sizes that upscale is scaling to? I'm wondering if

float h_scale = ((float)op.h_in-1)/(op.h_out-1);
float w_scale = ((float)op.w_in-1)/(op.w_out-1);

is producing nans, but that would require h_out/w_out to be 1

Edit: but I guess that would result in ALL the values in the image being Nan, not just the edge

[nkoppel]

I suspect that this bug is memory-related, and it being difficult to reproduce is due to the pseudo-randomness in cuda's malloc algorithm.

[coreylowman]

Also, this happens in the forward call right? Not backward?

[nkoppel]

Yes, at least within my application, it happens in the forward call. If it had happened in the backward call, parameters would have been set to NaN and caused the entire output to be NaN.

[coreylowman]

The weird thing is since upscale2d calls let mut out = inp.device.try_zeros_like(&(chan, out_height, out_width))?;, this will memset all the data in the output to 0. Meaning the kernel is definitely setting the nan values.

[nkoppel]

Yes, and it seems impossible for upscale2d to read out of bounds as long as the data in op and inp_strides is correct.

[coreylowman]

Oh I figured it out - there's a bug in the upscale2d kernel. Will push to the above PR