developing SMAA shader

[prepare]

SMAA for real time AA shader

see http://www.iryoku.com/smaa/

https://github.com/iryoku/smaa

---

Aims: provider real-time AA like these pictures

I test our lion images with the original (iryoku) program(SMAA-DX10-v2.8.exe)
I think the results are impressive and fit my need.

This is test image.

pic 1: test image

pic 2: disable SMAA

---

pic 3: enable SMAA

---

pic 4: no SMAA, our lcd-effect text look clear

---

pic 5: custom setting SMAA, our lcd-effect text look blur, but the lion is good

---

pic 6: with preset SMAA (ultra) , color edge detection

---

pic 7: right side (AA) image is post SMAA filter of the lion on the left

pic 8: small lion, rendered with Agg (software rendering), with AA

At small side, our Agg is finer.

---

so , Let implement this!

[prepare]

Implementation

The 2 images from WebGL port of SMAA (see more at
https://github.com/mrdoob/three.js/blob/dev/examples/js/shaders/SMAAShader.js,
https://github.com/mrdoob/three.js/blob/dev/examples/js/postprocessing/SMAAPass.js)

pic 1: getTextureArea(), png, depth 24 bits

pic 2: getSearchTexture(), png, depth 8 bits

pic3 : modified version of pic 2, rgba32 bits

[prepare]

Apply SMAA's edge detection filter (step 1)

pic 1: test input data

pic 2: just draw the img on GLES2 surface, no other filter

---

the original code, this steps alpha value always = 0.0
(so you will not see any thing, you will see just a red rect.)

so I modified the shader for debugging purpose.

pic 3: On GLES2, SMAA edge detection

on red background,

black => discard color
yellow line => full edges (ignore original weight, just an opaque yellow, ) with alpha = 1.0,

---

lets discard color as original,
and let the edges have weight as original (vary yellow) with alpha =1.0

pic 4: GLES2, SMAA edge detection

---

and when turn it to the original code

pic 5: just a red rect

[prepare]

since pic 3-4-5 have red background.

here, the left one is stage 1 output image without red background
compare with the original output(right).

pic 6: you can see the red and green pixels, but this is not correct

[prepare]

Early experiment with errs : (

pic 1: attemp1, err

pic 2: pic 1 (above) zoom

---

pic 3: lion, err

---

pic 4: attemp 4

pic 5: zoom on pic 4, (1) anti-alias at left leg of the lion compare to the original alias(2)

You can see that there are some anti-alias results.
We come into the nearly correct way now!

---

pic 6: fix SMAA_AREATEX_SELECT, still err

developing ...

[prepare]

attempt 2: step1 edge detect
I think it is pixel perfect !

pic 4: Color Edge detection, PixelFarm (left) vs original (right)

---

pic 5: pic 4 zoom in, lion's forehead , PixelFarm (top) vs original (bottom)

---

pic 6: pic 4 zoom in, text part, PixelFarm (top) vs original (bottom)

[prepare]

I'm working on this ...

[prepare]

Attempt 3, not correct.

[prepare]

pic 1: edges

pic 2: blend weight, not correct

top-to-left_bottom is ok

errs

attemp 5, errs

[prepare]

MLAA's L, Z, U shapes

from RESHETOV A. : Morphological antialiasing. In Proceedings
of the Conference on High Performance Graphics 2009

---

---

from SMAA paper

[prepare]

Area Texture

Calculates the area for a given pattern and distances to the left and to the
right, biased by an offset:

---

Orthogonal Pattern 0

-0.375

-0.25

-0.125

+0

0.125

+0.25

+0.375

---

Orthogonal Pattern 1

-0.375

-0.25

-0.125

+0

+0.125

+0.25

+0.375

---

Orthogonal Pattern 2

-0.375

-0.25

-0.125

+0

+0.125

+0.25

+0.375

---

Orthogonal Pattern 3

-0.375

-0.25

-0.125

+0

+0.125

+0.25

+0.375

---

[prepare]

Orthogonal Pattern 4

-0.375

-0.25

-0.125

+0

+0.125

+0.25

+0.375

---

Orthogonal Pattern 5

-0.375

-0.25

-0.125

+0

+0.125

+0.25

+0.375

---

Orthogonal Pattern 6

-0.375

-0.25

-0.125

+0

+0.125

+0.25

+0.375

---

Orthogonal Pattern 7

-0.375

-0.25

-0.125

+0

+0.125

+0.25

+0.375

---

Orthogonal Pattern 8

-0.375

-0.25

-0.125

+0

+0.125

+0.25

+0.375

[prepare]

Orthogonal Pattern 9

-0.375

-0.25

-0.125

+0

+0.125

+0.25

+0.375

---

Orthogonal Pattern 10

-0.375

-0.25

-0.125

+0

+0.125

+0.25

+0.375

---

Orthogonal Pattern 11

-0.375

-0.25

-0.125

+0

+0.125

+0.25

+0.375

---

Orthogonal Pattern 12

-0.375

-0.25

-0.125

+0

+0.125

+0.25

+0.375

[prepare]

Orthogonal Pattern 13

-0.375

-0.25

-0.125

+0

+0.125

+0.25

+0.375

---

Orthogonal Pattern 14

-0.375

-0.25

-0.125

+0

+0.125

+0.25

+0.375

---

Orthogonal Pattern 15

-0.375

-0.25

-0.125

+0

+0.125

+0.25

+0.375

[prepare]

Diagonal Patterns

from SMAA paper, fig 8 , Diagonal patterns map (left) and their precomputed
area texture (right)

Diagonal Pattern 0

-0.25, 0.25

-0.125, 0.125

0,0

0.125,-0.125

0.25,-0.25

---

Diagonal Pattern 1

-0.25, 0.25

-0.125, 125

0,0

0.125,-0.125

0.25,-0.25

---

Diagonal Pattern 2

0.25,0.25

-0.125,0.125

0.0

0.125,-0.125

0.25,-0.25

---

Diagonal Pattern 3

-0.25,0.25

-0.125,0.125

0,0

0.125,-0.125

0.25,-0.25

---

Diagonal Pattern 4

-0.25,0.25

-0.125,0.125

0,0

0.125,-0.125

0.25,-0.25

---

Diagonal Pattern 5

-0.25,0.25

-0.125,0.125

0,0

0.125,-0.125

0.25,-0.25

---

Diagonal Pattern 6

-0.25,0.25

-0.125,0.125

0,0

0.125,-0.125

0.25,-0.25

---

Diagonal Pattern 7

-0.25,0.25

-0.125,-0.125

0,0

0.125,-0.125

0.25,-0.25

---

Diagonal Pattern 8

-0.25,0.25

-0.125,0.125

0,0

0.125,-0.125

0.25,-0.25

---

Diagonal Pattern 9

-0.25,0.25

-0.125,0.125

0,0

0.125,-0.125

0.25,-0.25

---

Diagonal Pattern 10

-0.25,0.25

-0.125,0.125

0,0

0.125,-0.125

0.25,-0.25

---

Diagonal Pattern 11

-0.25,0.25

-0.125,0.125

0,0

0.125,-0.125

0.25,-0.25

---

Diagonal Pattern 12

-0.25,0.25

-0.125,0.125

0,0

0.125,-0.125

0.25,-0.25

---

Diagonal Pattern 13

-0.25,0.25

-0.125,0.125

0,0

0.125,-0.125

0.25,-0.25

---

Diagonal Pattern 14

-0.25,0.25

-0.125,0.125

0,0

0.125,-0.125

0.25,-0.25

---

Diagonal Pattern 15

-0.25,0.25

-0.125,0.125

0,0

0.125,-0.125

0.25,-0.25

[prepare]

Orthogonal Pattern Location

pic 1: offset 0, 16 patterns with location

pic 2: left, original and right my re-arranged to show to changes

pic 3: zoom upper part, offset -0.375,-0.25,-0.125

pic 4: zoom lower part, offset +0.125, +0.25, +0.375

[prepare]

pic 4: bilinear func

---

pic 5: This dict returns which edges are active for a certain bilinear fetch. (it's the reverse lookup of the bilinear function), see pic 5, red point => sampling position

---

[prepare]

Checking texture coordinates

pic 8: observing , the nature of texture coord

---

pic 9: observing , texture coord is +0.5,0.5 from grid coord

---

pic 10: texture coord offset back -0.5,-0.5 compare diff with floor(tex_coord), diff nearly 0

---

pic 11: texture coord offset back +0.5,+0.5 compare diff with floor(tex_coord),, diff nearly 1

[prepare]

Texture examples

pic 1: original bmp

---

pic 2: result of step 1 (find edges)

---

pic 3: pic 2, various offset

[prepare]

X-offset only, Zoom In

pic 4 :from SMAA step 1(edge detection)

pic 5: offset x +0.125 and -0.125

---

pic 6: zoom in, observe diff at arrow marks

A = 255/255 => 1
B = 223/255 => 0.8745
C = B =E=G=I

D= A= F =H

N=Q= S =A = 255/255 => 1
M=O=P=R = 223/255 => 0.8745

---

when we offset sampling point to left side 1/8 of pixel
the leftmost pixel loss 'color energy' = 1/8 pixel
remaining = 7/8 of original
so remaining = (7/8)255 = 0.875 * 255 => 233.125

---

notes:
When we offset sampling point X to +0.125 side (to right side) =>
the px take more right-side color (eg black) into itself.

When we offset sampling point X to -0.125 side (to left side) =>
the px take more left-side color (eg black) into itself.

[prepare]

Y-offset only, Zoom In

pic 7 : from step 1, edge filter

pic 8: zoom in

notes:
When we offset sampling point Y to +0.125 side (to lower side) =>
the px take more lower -side color (eg black) into itself.

When we offset sampling point X to -0.125 side (to upper side) =>
the px take more upper -side color (eg black) into itself.

---

Analysis on vertical axis

pic 9: left img=> offset=(+0.0px,+0.125px), right img => offset =(+0.0px,-0.125px)

pic 10: zoom, left img=> offset=(+0.0px,+0.125px), right img => offset =(+0.0px,-0.125px)

A, C= 223/255 => 0.8745
B, D = 255/255 => 1

N, P, R = 223/255 => 0.8745
M, O, Q = 255/255 => 1

notes:
When we offset sampling point Y to +0.125 side (to lower side) =>
the px take more lower -side color (eg black) into itself.

When we offset sampling point X to -0.125 side (to upper side) =>
the px take more upper -side color (eg black) into itself.

---

when we offset sampling point to top side 1/8 of pixel
the topmost pixel loss 'color energy' = 1/8 pixel
remaining = 7/8 of original
so remaining = (7/8)255 = 0.875 * 255 => 233.125

---

[prepare]

X Offset only, -0.25 px

pic 11: zoom

B = 255/255 = 1
A , C , D = 191/255 = 0.749

*so remaining = (3/4)255 = 0.75 * 255 => 191.25

[prepare]

Y Offset only -0.125

pic 12: zoom

A = 223/255 => 0.8745

so remaining = (7/8) * 255 = 0.875 * 255 => 223.125

[prepare]

X offset -0.25px and Y offset -0.125 px

pic 13: offset x -0.25px and offset y -0.125px

A = 167/255 => 0.6549
B = 167/255 => 0.6549
C = 223/255 => 0.8745
D = 223/255 => 0.8745
E = 167/255 => 0.6549
F = 191/255 => 0.7490
G= 175/255 => 0.6863

---

Lets do some calculation at A pixel

It value 167/255 can be calculated from bilinear interpolation

1). see pic 11, only X offset -0.25px => A pixel value= (3/4) *255 = 191.25
then bring 191.25 to 2) below
2. see pic 12, only Y offset -0.125px => (7/8) * 191.25 => 167.34375

so A = 167 :)

OR
A= (3/4) * (7/8) * 255
= (0.65625) * 255
= 167.34375

---

if we use 'normalized' pixel value (where value range starts from 0.0 to 1.0)
in this case, A pixel value = (3 / 4 ) * (7 / 8)= 0.65625

if you look at pic 5 (above), 0.65625 is here, exact the same as real specimen.

---

Think back...

If you sampling texture at position offset (-0.25px,-0.125px)
and you get sampling value= 0.65625

you can assume that you are at the Left-Upper corner

[prepare]

Z shape and its area

---

Sampling around Z-shape

[prepare]

Z shape and its area

pic 17: z-shapes

---

pic 18: area under the line

[prepare]

U shape and its area