Gathered from https://discourse.julialang.org/t/batchnorm-only-track-stats-true-supported-on-gpu/62091.
This would most likely require changes in NNlibCUDA as well. I'm not sure how interchangeable the various cudnnBatchNormalizationForward* functions are, so putting a pin in this until someone more knowledgeable can comment.