Outlier 3-px error value in last batch of validation set

[mzy97]

When I use finetune.py to train, when the code reach the last batch of validation set, if I set validation batch size to 3 (which can't divide validation set size), the last batch validation error (3-px error) is very large compared to all the previous validatioin error. Why is it happen? Thanks.
batch size setting:
TestImgLoader = torch.utils.data.DataLoader( DA.myImageFloder(test_left_img, test_right_img, test_left_disp, False), batch_size=2, shuffle=False, num_workers=4, drop_last=False)
result image (see iter 13, use your pretrained KITTI2015 model):

But when i use batch_size=2, which can divide validation set size, the result is normal. (use your pretrained KITTI2015 model)

And when i use my own trained model, the situation even worse in both validation batch_size=2 and 3.

Iter 0 3-px error in val = 0.979
Iter 1 3-px error in val = 0.495
Iter 2 3-px error in val = 2.987
Iter 3 3-px error in val = 1.483
Iter 4 3-px error in val = 1.028
Iter 5 3-px error in val = 0.730
Iter 6 3-px error in val = 2.347
Iter 7 3-px error in val = 2.339
Iter 8 3-px error in val = 3.413
Iter 9 3-px error in val = 1.755
Iter 10 3-px error in val = 1.139
Iter 11 3-px error in val = 15.801
Iter 12 3-px error in val = 1.836
Iter 13 3-px error in val = 0.706
Iter 14 3-px error in val = 2.358
Iter 15 3-px error in val = 0.646
Iter 16 3-px error in val = 1.741
Iter 17 3-px error in val = 2.123
Iter 18 3-px error in val = 1.530
Iter 19 3-px error in val = 33.141

Another question: how much the training batch size will affect the result? batch_size=12 will do much better than batch_size=2?

[JiaRenChang]

@mzy97
I thought it's normal because we used 40 images to valid.
While batch size is 3, the last iteration only contained 1 image pair, and the error can not be averaged with other "good performance image pairs".
We suggest the training batch size can be set to 8~16.
In our experiments, yes, batch size =12 is much better than batch size =2.

[mzy97]

@JiaRenChang
Very glad to see your reply.
But in your code to calculate validation error, i can't see why this "bad average" happens.
This code will let the last batch_size=1. And the dimension of pred_disp and true_disp variable in my test is (1,368,1232).
TestImgLoader = torch.utils.data.DataLoader( DA.myImageFloder(test_left_img, test_right_img, test_left_disp, False), batch_size=3, shuffle=False, num_workers=4, drop_last=False)
This code will get the right number of total not-zero point- len(index[0]).
index = np.argwhere(true_disp > 0)
And this code should work nomarlly.
1 - (float(torch.sum(correct)) / float(len(index[0])))

Can you tell me more detail about what things happened in code which lead to this behaviour?

Thank you!

[JiaRenChang]

@mzy97
index = np.argwhere(true_disp > 0)
This code line finds all the valid pixels in a batch.
If batch_size = 3, the len(index[0]) indicates the number of valid pixels in these three image pairs.