You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
That square grids will occurs when the output is affected by data other than input.
For example, using zero-padding in the convolution layer, it spoils the accuracy in the vicinity of an edge of the output image. Using 3x3 convolution with 1px padding, It can be seen that the edge 1px of the output image becomes unstable. So when using padding, it needs to calculate unstable area and cut out it. I believe batch normalization also causes similar problems.
So, in waifu2x, padding and batch normalization are not used.
Why we should care
Many architectures use padding during training, so if waifu2x use padding , then it can use others' model check points to initialize training. And the models can concatenate layers (e.g. DCSCN ) or easily add up residuals.
Re-training my models are expensive.
But padding makes the edge values unstable, especially when splitting a large image into pieces and merge them back. For example, the grids in the follow image always show up.
I first pad the whole image boarder with its replication values, then I split it into pieces. Each piece has overlapping boarders. After rescaling the pieces, I cut the overlapped parts and merge them back to an image. I also cut the padded boarder in the final image.
Overlapping 3 pixels seems enough, though larger value might be better.
Here is a naive and buggy example. An image is sliced from the top left. If a slice's width is smaller than the padded width, then the code will raise an error.