Regarding the normalization step

[DahyeYoon]

Hi Matt,

First, thanks for sharing your nice code which helped me a lot for figuring out details of the paper.
I came up with a question regarding the normalization step.
What is the reason for making images to be in the range [0,1] in line 128~133 of 'd_model.py'? Was it due to the undesired interpolation effect occurring during running skimage.transform.resize? I tried by myself with two options: with and without taking your normalization step before resizing; it seems like there is difference.

One more thing, when running your code with UCF101 dataset, there is an error "Images of type float must be between -1 and 1." I checked with the debugging mode, and found that sometimes skimage.transform.resize results violate [-1, 1] range. For a case you need a quick modification, I guess following code will help you some,

sknorm_img = np.minimum(sknorm_img, 1)  
sknorm_img = np.maximum(sknorm_img, -1)

Best,
Da-Hye

[fedden]

Thanks, hopefully this will fix some similar issues I had. If placing your code just after the following:

# for skimage.transform.resize, images need to be in range [0, 1], so normalize to
# [0, 1] before resize and back to [-1, 1] after
sknorm_img = (img / 2) + 0.5

Should the maximum not be 0 instread, rather than -1?

sknorm_img = np.minimum(sknorm_img, 1)  
sknorm_img = np.maximum(sknorm_img, 0)

[DahyeYoon]

Thanks for your reply!

I think you're right and there might have been an oversight.
But, I can't still understand the reason for making images to be in the range [0,1].
Would it be possible for you perhaps to explain your normalization step for me? I would greatly appreciate it.

Best,
Da-Hye

[fedden]

Hey Da-Hye,

No problem. Don't take my word as gospel; I could definitely be wrong on this. Really I'm just following the comments in the code about keeping the images in the desired ranges as quoted above.

Essentially images need to have a minimum and maximum of zero and one respectively. So the np.arrays are taken from the range -1 to 1 with this line:

sknorm_img = (img / 2) + 0.5

Then sometimes the images might be still be outside the range of zero to one (usually not by much when you print it out) so I just clamped them using these methods.

sknorm_img = np.minimum(sknorm_img, 1)  
sknorm_img = np.maximum(sknorm_img, 0)

I'm not sure whether I'm being a muppet or not but this seemed to get rid of the errors I was getting.

Hopefully it helps!

[PerloDaniele]

Hi All,
I have figured out how not generate the annoying images "Images of type float must be between -1 and 1."
The issue seams to be in the input frames generation.
In utils.py ->
def normalize_frames(frames):
[...]
#the double division is prone to numeric errors
new_frames /= (255 / 2)
#replace it with new_frames /= 255 * 0.5
[...]

It worked well.
I hope it may help
Daniele